Discussion
Infinite Extensibility means changes of any kind can be injected into a system indefinitely, without having to start over. This means without restarting the system or starting a new version. Thus Infinite Extensibility can reduce versionitis and legacy systems, and better support continuous evolution, dynamic changes, a more productive process, and parts. High Extensibility is widely seen as needed but hard to achieve. Some feel extensibility is the most important feature of a design, me included. Extensibility is the most important requirement. After all, 75% of software development is maintenance and Hyper Change is the driving force in today's Information Age.
Sometimes there will be a complete change of heart in a system's design. Then a new version should be done. But the reason will be massive change, not that changes to the old system are hard to do.
If we pull this off, we will have slowed down the Law of Software Entropy, which says that systems invariably "run down" with age, which is the number, size and complexity of changes made to them. They won't run down as easily because change will be so easy to do. But then again, they will run down somewhat due to awkward design decisions which may pile up, breaking the original conceptual architecture. This is more of an organizational skill issue than a technical issue, but is very real. As the need for developers increases but the supply of those capable of really good design remains somewhat constant, software entropy unfortunately tends to appear more than it should. This trend may be reversed when visual tool use replaces coding, and tools are smart enough to enforce good design.
Infinite Extensibility is achieved by pushing loose coupling to the max, plus management of dependencies. Dependencies are what makes extensibility hard.
BA Use Cases
From the BA's standpoint, we need to support these Use Cases while the system is running or not. Note that a part can be a "container part" with a bunch of child parts.
- Add a part, either uninstantiated or already instantiated.
- Remove a part.
- Replace a part with a new version.
- Replace a part's DK.
- Add, change or remove a part's public "work" methods.
- Direct method calls between parts instead of Messages when high speed is needed.
- Privide custom implementations of any microkernel class.
- Use factories and item IDs in containers, in addition to part class names.
- A part needs to use the kernel.
- A tool needs to use the kernel.
We are not supporting these Use Cases:
- Treating kernel objects as parts. This introduces too much complexity.
- Multiple versions of the same part. This introduces inconsistency.
(Do we have all Use Cases?)
Use Case Solutions
1. Add a part, either uninstantiated or already instantiated.
"Uninstantiated" already works by adding a section for an item in a container's parex file. It can be improved by not closing and restarting the container, and instead the container being smart enough to just add the part.. "Already instantanted" could be done with Basket.addItem(Item item) combined with a parallel change to the container's parex file. This could be done just by code, or with the user dragging a part from the Part Shop and dropping it on a System Tree container.
This allows moving live parts among systems and Part Shops.
2. Remove a part.
Do with Basket.removeItem(String itemName). If in use by other parts or running, the system should complain. If used by other parts, something should alert the user. If running, the part should be stopped first.
3. Replace a part with a new version.
A replacable part needs to declare that ability. It also needs to keep track of who is using it internally (its own non-BA subsystem). A two phase commit style transaction is required. In BeginReplacement, the system and all instances of the part determine if the part can be replaced, the instances stop receiving new clients, and they may request their clients to pause using the part. In CommitReplacement, the part and system replaces all references with the new part and the old part passes its state to the new part. AbortReplacement is used if problems are encountered. This is complex and not full designed.
4. Replace a part's DK.
Already works with ParamDriven.applyNewParam().
5. Add, change or remove a part's public "work" methods.
This is very hard in Java due to strong typing compilation. The crux of the problem is the list of methods in an interface (or class) is static, not dynamic. Thus static multiple method interfaces cannot evolve. While language level interfaces are great for defining object responsibilities, they are terrible when it comes time to add, modify or remove a method. To introduce change, one must change the interface, thus breaking all implementations. While this is no problem for closed or small systems, it's a show stopper for large systems or use of lots of third party parts, which is becoming the norm. OO languages or IDLs have done their best implementing polymorphism and contracts. We now need to proceed on to the next challenge - How can objects support Infinite Extensibility with public methods?
They can't, so we require all parts to collaborate with Messages and provide a list of Messages they support. The intent and result is the same as interface methods, but supported Messages can be added, changed or removed at run time.
Most parts are subsystem entry points. Within a non-BA subsystem, direct method calls are fine, because that is encapsulated, is more clear than Messages, is more compact codewise, is more easily documented, and is faster. Most inter-object calls are within a part if large or medium grained parts are used.
The first BA iteration uses direct method calls in containers a lot, for establishing relationships and for minor configuration. The first will be replaced by Messages. The second can be replaced by Messages or parex files in most cases, and by "Unimethods" as defined below. This whole area is a risk and will need exploration.
6. Direct method calls between parts instead of Messages when high speed is needed.
This conflicts with (5), so we introduce an exception to the rule. A part may expose a public method by:
- Providing a reference to itself in an "Acquire" Message.
- Providing an interface for each single method exposed.
This allows high speed collaboration without getting into extensibility problems or creating a system that cannot be validated for dependencies. We define this as the Unimethod Interface pattern, which allows flexibility through fine grain desgin. If a method is added to a high speed part, nothing breaks, unlike normal interfaces.
The first time PartA needs to call a high speed method, it sends an "Acquire" Message which returns a reference to the requested PartB. PartA then casts the PartB reference to the correct Unimethod Interface, saves the reference, and starts making calls. Interface examples are SendCommMessage, GetNextDatabaseRow and CalcCustomerItemPrice.
7. Privide custom implementations of any microkernel class.
We support this by using an interface for each kernel class, with standard implementations. Each container inherits its parent's implementations. The class to use per interface can be specified in a container's parex file.
We use interfaces instead of Messages due to the role the kernel plays in a BA system. It needs to be small, fast, simple, understandable and extensible. The last is achieved by the paragraph above, but due to the nature of the problem is not achieved 100%. This is inherent to foundational layers. Down the road, we may discover a better approach. If there's only one kernel implementation (like the Linux kernel), we have no problems, and so recommend against additional implementations at first, except by the BA developers for things such as container proxies.
8. Use factories and item IDs in containers, in addition to part class names.
"BeanClassName" is already supported. To support container factories, allow the ItemFactory to be set in the container parex this config lines. The default ItemFactory only knows about class names. A custom one can handle the BeanClassName any way it wants to, such as by mapping semantic names to class names known only by the factory.
This is the Abstract Factory pattern. Factories allow extreme extensibility by converting intent to decision at runtime, a form of Declarative Knowledge. This feature is based on work by Steve Alexander, and was not in BA iteration one.
9. A part needs to use the kernel.
We expose only kernel BasketServices and ContainerServices to parts, to encapsulate the kernel. A part receives the appropriate reference if it implements BasketServicesUser or ContainerServicesUser. The part then calls interface methods to use kernel services.
10. A tool needs to use the kernel.
Tools are very different from parts. Tools need to start BA systems, close systems, navigate through all a system's parts, receive system events such as when nodes are opened, add parts, remove parts, replace parts, etc. Therefore we expose much more of the kernel to tools. This is done by:
- Starting a system returns the root Item. This allows access to the entire system.
- System navigation is done by Item and Basket.
- System events are handled by ItemSystem. (not in first iteration)
- Basket handles dynamic add, remove, replace part. (remove, replace are new features) via applyNewParam(), which supports container Param changes of any kind. However, the Basket is restarted. We need to improve on this, for example support altering just one part without restarting.
Message Subsystem
This is the most important subsystem the kernel uses, more so than the Param subsystem. It replaces the "method call" mechanism of classical OO with Messages. In a BA system dependencies are defined by the Message structure. Here's a rough review of the new Message subsystem, starting with the two main interfaces parts implement to collaborate.
public interface MessageListener extends MessageRegistryUser {
public void processMessage(Message message);
public MessageListenerInfo loadMessageListenerInfo();
}
public interface MessageSource extends MessageRegistryUser {
public void addMessageListener(String eventName, MessageListener listener);
public void removeMessageListener(String eventName, MessageListener listener);
public MessageRouter loadMessageRouter(); // For tool use only
public MessageSourceInfo loadMessageSourceInfo();
}
public interface MessageRegistryUser {
public void setMessageRegistry(MessageRegistry registry);
}
The MessageRegistry is provided by the container and may be custom. It contains MessageDefs. A MessageDef defines the name and properties of a Message, plus documentation. MessageDefs are just like method signatures. The registry allows the same Message to be sent or received by multiple parts, without each having to create a MessageDef, as in the first BA iteration. We expect that in most cases there will be one registry per system, until systems get large or systems get reused. Message names will need collision avoidance, probably by using the same scheme class names use.
MessageListenerInfo and MessageSourceInfo are extensible definitions of supported behavior. MessageListenerInfo contains the Message names the part is interested in, which are required, and which are optional, which are conditional, and future info. MessageSourceInfo contains the Message names to part fires, including which are required and such. The Info objects can be acquired by parts for dynamic behavior. They follow the ObjectInfo pattern, which replaces many getter methods with a single getter returning a Datatron.
loadMessageRouter() is used by tools to examine the Message structure of a system, such as for showing a Message chain or validating Message dependencies. It is not to be used by the kernel or parts, since it violates encapsulation and could introduce unwanted dependencies that would be unknown to Message Relational Integrity. A read only version of the source's MessageRouter is returned to prevent the obvious.
Comments:
"load" is used instead of "get" to avoid including these methods in the Inspector's property list. This speeds up Inspector use, because these methods are lots of work.