Architecture - UHR Generation 2

Architecture is many things to many people. To us it's the choice of key high level abstractions, partitions and their relationships, in that order of importance. The rest is low level design such as component selection, class identification, interface definitions, data structures and the UI.

Rationale

Rationale is the reasoning behind action. We can reduce ours to just a few lines of deduction. The line in bold is the conclusion.

Focus on reuse
1. Observation shows software productivity has huge room for improvement.
2. Reflection shows that most development work should only be done once.
3. Thus the unifying theme for improvement should be reuse.

Use reusable parts and infrastructure
1. The place to start is reusable parts. (a bit of a leap)
2. The less parts know about each other the greater the reuse.
3. The less parts have to do the greater the reuse.
4. Domain parts cannot do this themselves.
5. Thus domain parts need reusable infrastructure to maximize reuse.

Small domain neutral infrastructure
1. The power of an abstraction varies inversely with its domain neutrality.
2. Perceived complexity varies with size more than anything else.
3. Thus infrastructure should be as small and domain neutral as possible.

Separate what from how
1. A line in the sand needs to be drawn between what to do and how to do it.
2. Parts should specialize in how to do things.
3. People should specialize in deciding what to do.
4. Thus a medium such as DK should carry what to do to parts that know how to do it.

Anonymous collaboration
1. Usually PartA must collaborate with some PartB to get its job done.
2. If PartA knows about PartB then PartA can only be reused with PartB.
3. All forms of collaboration can be expressed as routing data between parties.
4. Thus parts should collaborate anonymously through a medium such as datatrons.

To summarize, we can achieve higher productivity through higher reuse by using reusable infrastructure to assemble reusable parts into systems. People should express their intentions with DK, not code. DK should drive the behavior of reusable parts that collaborate anonymously with datatrons.

Now that we've determined our key high level abstractions, what are our key high level partitions? Consider Occham's Razor, "Entities are not to be multiplied beyond necessity". Ideally our partitions should be the same as our 4 abstractions. But a severe lesson learned from generation one (and examination of other frameworks) was that infrastructure itself quickly becomes the dominant, intractable, limiting aspect. What if we could vanish it into the other three, or morph the other three into the infrastructure? Why must infrastructure be separate from domainness in form?

Indeed it need not be. Taking a cue from life forms, an experiment led to the conclusion that self-referential systems can be built from parts where the infrastructure uses the same 3 partitions (parts, DK, datatrons) that domain behavior does. This vanishes reusable infrastructure as a separate partition. Boom! The separate kernel of generation one is gone. If perceived complexitity of a system varies with the square of the number of partitions, we have just reduced it from 16 to 9.

What really happened to the infrastructure partition? It didn't vanish. It changed from a separate abstraction to an emergent property of the 3 partitions. Emergent properties are highly desirable because they leverage off of existing entities. They cause higher output to input ratios, and can dramatically increase the efficiency of systems.

The final question is what are our key high level relationships between these 3 partitions? Due to the small number and clarity of partitions, this falls into place easily as:

These are the three essential elements of UHR. They form the foundation for everything else. Not shown is the trivial but closuresque relationship that DK itself is a type of datatron.

How The Three Essential Elements Work Together

When a part is created it is automatically given its DK. This is done with the ParamDriven interface using the setParam(Param) method. The part doesn't do anything at all to get its DK - it just magically appears, greatly simplifying system design.

The part then reads its DK for whatever it needs to do to initialize itself for this particular reuse case.

Usually this is very simple. Every time it needs to do something, it consults its DK for the key/value(s) that has the information about what to do.

In more complex cases the part will perform lots of work when given its DK. For example it may be a cache registry for data from many places. It would collect the data the DK told it to from where the DK said to get it, in the priority and amounts in the DK. It would advertise what it had using DK to where it should using DK. Refresh policy would be determined by DK. What to do is determined by Dk and the part knows how to do it. The CacheRegistry part could be widely reusable, because it was designed to configure itself with DK. Stop and think a minute - this is powerful stuff....

The same part can be reused many places in the same system. Each part instance gets its own separate bundle of DK. That DK can be custom for each part reuse, or DK can itself be reused for many parts. You can even reuse DK for different kinds of parts that are compatable enough for that. The Declarative Knowledge abstraction has a powerful, flexible, simple, understandable mechanism in UHR implementations that allows the part inventory manager, system architect, system assembler, part designer, etc to achieve very high reuse rates that exceed their own and customer expectations.

Now we see how parts and datatrons work together:

After parts are created and initialize themselves with DK, they start working. Unless a system is built with one part, parts always work together to accomplish the system's total behavior. How parts work together (collaborate) without entangling dependencies that destroy the loose coupling needed to achieve high reuse is what datatrons are for.

In the above illustration (All by my loving wife, Martha. I'm a terrible artist. JH :-) PartA never knows about PartB. It's as if there's a cloud between them. To collaborate they send and receive datatrons that fly through the clouds between parts.

The particular datatrons used here are messages. However, to keep the principle of datatrons and Anonymous Collaboration firmly in our frontal lobes, we call them datatrons, because that's what they really are. A message is merely a datatron with a name for understanding and routing purposes. Message name examples are "CloseRequested", "GetUserSecurityLevel" and "DataAdded".

A datatron is a bundle of data. The most common datatron currently is an Infotron. This is a tree data structure of keyed values. It can carry most types of data easily. For example a message named "FileDeleted" would have the name of the file deleted.

To preserve loose coupling, datatrons can only carry primitives, Strings or other datatrons. This takes care of most collaboration needs.

In some cases a complex object needs to be transferred between parts. That introduces tight coupling. We are still working out ways to minimize that. One promising way is "stream datatrons" that allow high speed reads and writes between parts of primitives and Strings. Another way is Jini style service lookups, using a matching template to find the needed service, getting it, and then casting to an interface. If we can standardize on these interfaces we will have reasonable loose coupling. Either of these mechanisms is done by domain parts, not the core, which remains small and simple. The core transports datatrons and never knows what's in them.

Architectural Model

The model reflects what was learned from generation one, two prototypes and six architecture candidates, plus frequent communication with Steve Alexander.

We use layers strictly. A layer can only use a class at its level or below. If an upward call is needed an event must be used. This is why the core parts themselves use Messages.

"Core parts" provide the infrastructure all parts need, including themselves in a self-referential manner. "Domain parts" are for specific domain areas like Visual Tools, communications or business logic. All layers above Part Role are all built with core parts.

Core Types

The layers support various types of cores, such as:

1. Minimal Core - No DK or Messages. Only the Datatron, Part Role, Plugpoint Structure and Lifecycle layers are needed. Since CellLifecycle has no Message events, it must use direct calls on other parts.

2. DK Core - Add the DK layer and use a different CellLifecycle part that supplies DK to ParamDriven parts.

3. Full Core - Add the Circuit Based layer. Use a different CellLifecycle part that handles Messages.

More types are possible. This allows users of UHR to take their own approach, instead of
feeling locked into a monolithic framework. While core variations is not in the UHR Technology Grid, we have designed the core to be very flexible, and tested that idea by producing multiple cores. This will make it easier for others to customize the core as needed, or to observe how architectural extensibility can be achieved.

One goal of generation two was to avoid a monolithic framework. The design kept trying to go that direction, and we had to try real hard to keep multiple cores possible. It was like a slippery fish that kept squirming out of your hand, a bear of a puzzling problem. The final trick was to first analyze the layer dependencies. This failed at first, because for CellLifecycle to use messages to alert upward layers, and to be parameter driven, the structure layer had to be on top. But that made only one core possible. The solution was to remove lifecycle responsibilities from the structure layer, create the Lifecycle layer, and use different CellLifecycle parts for different cores. See the final diagram.

Certainly the surprise here was the heavy dependencies Lifecycle has with other layers. This indicates the importance of isolating this abstraction, which, now that we think about it, has nothing to do with structure. :-)

Part Types

Finally we model the main different types of parts, showing the meta-architecture we've been working up to. Part inventories take lots of careful planning, management and investment. However, unlike most inventories, you only build a part once.

These can also be considered partitions, but from the perspective of the part inventory manager. The partitions of parts, DK and datatrons was from the perspective of the software designer.

Part Dependencies

An important principle we are following is all part dependencies should be published. This allows the designer or other parts to better understand, use and reuse parts. This is a tricky area we have not yet fully resolved. A central regisry may emerge, with each part having a PartID to allow lookup of it's DK definition, Messages in and Messages out, services, documentation, use history, defect history, etc. The Part Shop would issue PartIDs and manage the part inventory and registry.

All part dependencies are published by the part or are available in a central registry or such. The key policies leading directly to benefits are:

1. A tool should be able to validate that all required dependencies are achieved.
2. Automatic hookup of most needed dependencies should be possible.
3. A human should be able to easily understand all a part's dependencies.
4. Dynamic dependencies should be minimized.
5. Horizontal dependencies should be minimized.

Dependencies are static or dynamic. Dynamic ones are much harder to understand, validate, automatically hookup, or have correctly available. Static ones give a system a certain rock solid foundation, and give the system designer greater peace of mind. :-)

Vertical dependencies are better than horizontal tree dependencies because they are easier to validate and hookup automatically. A horizonal dependency (between parts A an B in different sub containers) can be converted to two vertical dependencies by introducing an intermediary part above parts A and B. A and B have dependencies on the intermediary.

Open Issues

These are technical and hard to understand. We will remove, resolve or rewrite them.

PartID

How should we handle separation of part specification DK from mission DK? The former describes in messages, out messages, the mission DK structure, etc. The latter describes only what it takes to vary a part's behavior for a particular reuse mission. A part's Spec DK never changes (except new versions) but Mission DK varies for each mission.

One possibity is associating Part Definition IDs with data about the part, such as Spec DK, change history, use history, searchable attributes, implementations available, etc. This is similar to product model IDs.

One way to handle this is all parts implement a standard interface like:

    public interface Part {
        public String  getPartID(); // Really a definition ID

        public void    setPartInfo(PropMap);
        public PropMap getPartInfo();
    }

For example a PartID might be "org.jcon.inv.io.FileOpener". This looks like a class name, but is really a globally unique ID. The actual class name could be anything. There will bw multiple implementations for some PartIDs.

The PropMap is set for the part by getting the PartID, finding the Spec DK for that PartID, converting the Spec DK to a PropMap property called "Spec", and then calling setPartInfo(PropMap). Additional data can be put in the PropMap by the part or other parts. PartInfo is used at runtime for things like tools editing Part DK, parts querying parts about what they can do, or the designer trying to understand a part.

An alternative is using only the getPartID() method and using it to lookup data as needed. This is simpler and more stable.

I shudder to use an interface all parts must implement. A lesson learned from other frameworks is no matter what interfaces they started with in requiring parts to use, these interfaces turned out to be very confining later. Perhaps our solution is a required interface for a single abstraction (ID) and optional interfaces for other abstractions.

Part DK Definition

We do not expect Part DK definition to be dynamic. Thus it can be looked up with PartID, and ParamDrivenInfo is not needed from a ParamDriven part.

(more issues later)