Framework Patterns

Created 9/29/99 - Last Updated 2/20/01 - Jack Harich

A Framework Pattern tends to appear only in frameworks. Many other patterns will also appear, but Framework Patterns are of extreme interest because they form the prime pallet for the framework designer, and offer a starting point for drastically better frameworks by trying to improve the patterns themselves. The idea is the better our nuts and bolts are, the better our contraptions. :-)

These patterns lean towards "Total System" frameworks. We expect to add to this collection. Send us your suggestions!

Circuit Based - This occurs when parts publish all their discrete input and output, and the framework takes care of the input and output hookups among parts. This creates "circuits" just like the wires connecting electronic devices on a circuit board.. All part input and output travels along circuits. The input is normal methods. The output is usually handled as an event. Parts have no idea who they are collaborating with. (This was named Connection Based in an earlier document version)

This pattern deviates from the tradition of parts publishing only their detailed input (method signatures) and not their detailed output (method calls to other objects). The deviation immediately encounteres language limitations to publishing and sending output, and so clever mechanisms are required. For publishing output, there seems to be no clear standards. For sending the output, most have settled on standard events. However there are numerous complications in the object sent in the event and sending it. How is strong typing enforced? How is the event id generically handled and name space conflicts avoided? How are complex objects in the event standardized? Is event object mutation by listeners permitted? How are asynchronous events handled? What about exceptions? The industry is wrestling with these issues, including yours truly. :-)

Personally, I expect to see language support for Circuit Based in a few years, once the needs and benefits become clear.

This pattern is young. See the book "Component Software" by Szyperski, 1997, page 148 for a discussion of "Connection Oriented Programming". It makes the interesting observation that connections provides greater symmetry. We prefer the term "Circuit" to "Connection" because "Circuit" reuses the concept that has worked so well in electronics, and emphasises the route in addition to the connection at the part. We prefer "Based" because "Oriented" has become a buzzword, because this pattern requires such strong framework support that the framework becomes based on circuits, and because "Interface Based" is widely accepted. Should this pattern be called "Circuit Board" instead? Send us your votes! ;-)

Then again, this pattern is old. Circuit Based has strong parallels with electronics. Circuit boards consist of circuits and devices, with electrons as the uniform signal between devices. This simple pattern, circuits, devices and electrons, has allowed a host of industry improvements, starting with integrated circuits and riding Moore's Law for decades now. The same thing can happen to software if we begin building upon the correct simple patterns. For example UHR uses Message chains, parts and datatrons, but is groping around for what the circuit board should be. :-).

We will see "connection" used as a synonym for "circuit" in the literature, a pity because it confuses the issue. The precise shade of difference is connection refers to the hookup between the part and the path traveled, while circuit is the path. This is an important distinction, since in the future many of us will become circuit designers, an art in itself. Connections are trivial, while circuits commit parts to specific collaborations, induce multicasts, and the longer the circuit the longer the output takes to happen. Circuit layout and device selection, not device construction, will be the challenge as system sizes grow.

The advantages of Circuit Based include:

1. Very loose coupling is achieved.
2. A system's circuits become 100% exposed.
3. Any circuit can be made or changed at runtime.
4. Dynamic system behavior of nearly any kind becomes possible.
5. Part functionality and dependencies are clearer, so how to use a part is clearer.
6. Investment in large part inventories becomes practical.
7. The use of Visual Tools for system assembly becomes much easier.
8. "Push" models become easier.
9. Stateless parts become easier.
10. System designers think in more of a "hookup" and "black box" mode. This is a higher level than traditiional object oriented design, because in OO hookups occur within objects.
11. Gone are the statically compiled short connections of one part to another that make systems so hard to work with.
12. Output is far more flexible, offering multcasting, monitoring, filtering, dynamic redirection, logging, counting, visual animation and all kinds of wonderful things.

Disadvantages include lower speed and increased complexity, compared to traditional coding. We see both of these as surmountable. For example "acquire events" can get a direct interface based link for repeated high speed calls. Hardware will get faster. The circuit mechanism will get faster. Better Visual Tools can eliminate the increased complexity at the code level by code generation, use of DK for hookup definitions, autochecks for broken event links, and animation to make it all look so simple, even at the system design stage.

Container - A container serves as a plugpoint by containing reusables. In the case of a tree, it may also contain containers. The advantages are containers can be used to clearly organize a system, and can serve a number of ways in hitting higher reuse as they manage their members.

Context - The Context Pattern provides a "context" for a software component to use. This context is often the component's only view to the world outside of itself. This better encapsulates the component. The benefit is greater modularity, because the component is coupled to the rest of the system only through the context. By changing the context the component can work within different systems or different places in the same system, which increases reuse. Note the component may be a class or subsystem. The component must have a method to allows setting its context, such as:

public void setContext(Context context) {
    this.context = context;
}

This pattern is usually used with containers or a tree of components where some are containers and some are leafs. A common example of this pattern is AppletContext. For a complete example using production code see the NodeContext class. This is a very common, easy and powerful pattern. It was added to this document on February 2001 and so is not in the pattern analysis.

Exposed Delegation - This allows a class to expose its delegates, which carry out some of its responsibilities. This allows a slew of radical framework improvements. Here's an example:

public interface Delegator {

    // Returns an array of delegate names
    public String[] getDelegateNames();

    // Returns the delegate or null if none
    public Object getDelegate(String delegateName);

    // For optional use if the class doesn't provide the delegate
    public void setDelegate(String delegateName, Object delegate);
}

When a client needs a class implementing Delegator to do something, it may call a normal method, or it may get a delegate, cast it, and call a delegate method. The getting and casting takes extra time, and so will often be done in client initialization. The class should document what delegates it offers.

Advantages include avoiding large numbers of methods per class, avoiding high class complexity, allowing large class responsibilities without design and implementation problems, greater reuse, more dynamic behavior options, easier delayed object activation and greater flexibility in general. The main effect is likely to be cleaner, more extensible framework architecture.

Hierarchial Composition - This involves composing a system in a strict tree style. The root is a container. The containers contain containers and/or parts. Both the containers and parts are plugpoints, though usually only the part are. The advantages are clear system organization, ease of framework architecture design, ease of delayed subsystem activation, ease of mapping the internal structure to a visual one, ease of supporting Hierachial Lookup, ease of supporting a layered design or other types of partitions, and lots more. See earlier writeup.

Hierarchial Lookup - This occurs when an object is retreived by searching upward in a hierarchy, such as a tree of containers or a tree of registries. The advantages are shadowing is easily supported, and it supports Service Architecture very well.

Inverted Layers - In Layered Architecture, a layer calls lower layers directly. A layer cannot call upward directly, but can send events. Thus lower layers make far fewer assumptions about upper layers, and so tend to be more domain neutral and reusable.

As the diagram of typical layers shows, variations are possible. The software industry started out with applications, then had them use various reusables, which needed to use even more reusables. This resulted in the Classic Layers variation, which is the most common one today. Note the higher the layer, the greater its size, and the lower a layer, the greater its reuse. This pattern is very useful but has three main problems:

1. The higher the layer, the less the reuse because the more domain specific it is.
2. The higher the layer, the less the reuse because the more assumptions made about lower layers.
3. Many things (upper layers) develop dependencies on a few (lower layers). This inhibits lower layer evolution, flexibility and reusability. This is similar to the Fragile Base Class Problem.

The Classic Layer Pattern has built in limits to growth. Application parts are hard to reuse, and the frameworks that lower layers consist of find evolution terribly hard. This has been choking framework design for years, and is one of the main reasons why we see so many framework failures and short life spans.

These problems are elegantly resolved by inverting the pyramid of layers, giving the Inverted Layers variation. Now lower layer parts are fairly ignorant of the upper layers, which are more free to vary, allowing greater reuse of the upper layers. Because so much is done for them by upper layers (lifecycle management, initialization, hookup, etc), application parts can be mostly small, simple, and dumb, making them wildly easy to design, program and compose with. Upper layers, such as a framework "core", can vary more independently of parts, allowing fantastic evolution where it matters most - frameworks. This resolves problems B and C, and ironically turns the pyramid right side up. :-)

If this pattern is used with Circuit Based, application parts become more reusable because they are ignorant of each other, and more of them can be designed to be less domain dependent. This resolves problem A. It's conceivable that combining these two patterns will increase application part reuse by an order of magnitude.

Inversion completely changes the architecture. Application parts must now communicate with other layers using events (such as Messages) whereas before they used calls. This new mindset may seem awkward at first, but soon becomes A Welcome Better Way. :-)

One problem remains - The upper layers have strong dependencies on lower layers. This can be resolved if those dependencies become few and generic, such as by limiting them to part life cycle management and standard "framework user" patterns. When this happens, all layers can achieve high reuse easily. In particular, application part reuse goes sky high, and there is great joy in Mudville. :-)

A small percentage of parts will need to use the upper layers for system management work, such as dynamically adding parts at runtime or Visual Tool use. We shall see how to best handle this.

Lifecycle Controller - This occurs when a resuable has a lifecycle that is controlled by something else. For example the reusable may have setParameters(), init(), start(), sleep(), awake() and destroy() methods, all called by the Lifecycle Controller. These methods would best be portions of Optional Plugpoint Interfaces.

The advantage is resuables can be very standardized, clearly dedicated to a single task, simpler and smaller. All this results in much higher reuse. When this is combined with other patterns such as Circuit Based, Loose Coupling Mechanism and Parameter Driven, it's like a part says, "You take care of the rest of the kingdom, and I'll take care of this."

Loose Coupling Mechanism - This allows collaborating reusables to be more loosly coupled that what a language directly provides, such as interfaces. For example a Blackboard allows clients to collaborate without ever knowing about each other. Events and Messages are other examples. Generally the mechanism dominates the framework architecture. The advantages, of course, are looser coupling. The extreme of this pattern is Anonymous Collaboration.

Object User - This occurs when an element advertises its needs for certain objects, and is automatically supplied with them during initialization. We have identified two variations, shown in these two sample interfaces:

public interface ContainerServicesUser {
    public void setContainerServices(ContainerServices services)
}

public interface PartUser {
    public String[] getPartNeeds();
    public void setPart(String partName, Object part);
}

The first example handles only one needed object, while the second handles many and is more reusable, though at the cost of strong typing. The second also has dynamic potential - the reusable can determine its exact needs at runtime, such as from its initialization data. The advantage of this pattern is what a reusable needs is provided with no muss, no fuss - in other words, loose coupling and less code.

In the case where the reusable is needed to get arbitrary objects after initialization, other means besides this pattern are required, or a further variation. A simple way to handle this need is to use the PartUser variation to get a PartFinder, and then get parts as needed. However this introduces unexposed dependencies inside the part getting the other parts, greatly increasing complexity. Better would be to advertise what parts might be used, or use a Loose Coupling Mechanism.

Optional Plugpoint Interfaces - Many frameworks use one required interface per plugpoint, such as an Applet. Far more flexible is a number of optional interfaces per plugpoint. For example a part could optionally implement PartUser, ContainerUser and ParameterDriven. During part initialization the framework would check to see which interfaces were implemented, and behave accordingly. The various interfaces are a technique for "advertising" the part's needs.

The advantage of this pattern is simplicity per part, extreme flexibility and extensibility. For example in a framework with ten optional interfaces, PartA implements Interface1, PartB implements Interface2 and Interface3, and PartC implements Interfaces1, 3, 8, 9 and 10. After the framework is released a customer needs Interface11 supported. No problem. That feature is added to the PartInterfaceOptions section of the framework, either as hard code or preferrably as an InterfaceOption.

Parameter Driven - This is the practice of determining behavior from configurable parameters, with the goal of greater reuse. The most frequent example is "table driven", where various values are stored in text files or databases. The values are looked up and supplied to what needs them. This separates "what" to do from "how" to do it.. The "what" varies per reuse instance. This type of reuse has proven to be far more useful than other types such as inheritance.

Generally the reusable element receives its parameters during initialization. The parameters can be simple, like a list of key values. Or they can be complex, like XML. The important thing is they declare what to do, and are independent of how to do it or what element does it. Parameters are also known as Declarative Knowledge.

The advantages include greater reuse, lower costs, faster developement, higher quality, ease of allowing end users to compose or configure their own systems, and happier developers and users. See earlier writeup.

Plugpoint - A predesigned point in a system for plugging in a desired element. All frameworks have plugpoints, and will not run or will provide only default behavior unless some plugpoints are filled. Interfaces, subclasses, parameters and configuration files are common plugpoint mechanisms. An outstanding plugpoint example is an Applet (fills a plugpoint) in a browser (the framework).

The advantages of this pattern include carefully controlled exposure of how to vary the behavior of a framework, easy determination of where to go to vary a framework's behavior, and the possibility of creating a Plugpoint Structure once a plugpoint strategy is determined.

Plugpoint Interface - The use of a specific interface to define a plugpoint protocol. This is extremely common. The advantages include easier system design, easier implementation, the use of variable implementations for a single plugpoint, and easy support of third party implementations.

Plugpoint Structure - The basic structure(s) exposed by a framework that resuable elements plug into. For example an initialization file may have a section for optional subsystems entry point class names, or a Preferences window may allow adding plugins or custom extensions, or a website file structure can receive html, image and directory files. The first two are crude Plugpoint Structures, the third is more capable because it allows endless composition, a feature of tree structures.

One perspective is to view a framework's Plugpoint Structure as an arrangement of "docking sites" on the body of the framework. Reusable parts "dock" to compatable sites when added and "undock" when removed. See the terrific book "The Way Life Works" by Hoagland and Dodson, page 48 for one way nature uses docking sites:

"Life can't get by on energy alone. The simple chemistry of random motion and collision we've seen so far could not begin to maintain life in all its complexity. Things can't be left to chance: life needs a way of making chemical events happen more surely and rapidly. Getting molecules into perfect position and then pushing them to react is the job of enzymes.

"Enzymes are catalysts - speeder-uppers and facilitators of chemical reactions. Each enzyme has docking sites on its surface into which simple molecules fit precisely. Once it has a grip on the molecules, the enzyme chemically interacts with them, forcing them to react."

The Plugpoint Structure is the backbone of a framework, so a primary task of the framework designer is designing a structure of docking sites. There will usually be a base structure, such as a tree, layers or files with lists. To this the designer must add finer and finer degrees of docking sites, because whatever is added to a site must often itself be configurable or compositional. If this can all be done in a clean, extensible, flexible manner, the framework skeleton will be solid.

One advantage of this pattern is it forces the framework designer to focus on the interface between reusables and the framework, and get that really right. Another advantage is there are only a few good Plugpoint Structures. Once identified and perfected, the framework designer will do inherently better work.

If there is no Plugpoint Structure in a framework it will inevitably have a hard time accommodating change. For example it may be a classical OO approach with one class containing others, this partition using that partition, this type of reusable plugged in here, that type plugged in there, and in general a web of relationships and arbitrary plugpoints. When young such frameworks appear clean, but as versions with new plugpoints drag by they suffer from entrophy quickly, because they lack an extensible "skeleton" to provide a consistent, rational way to extend functionality.

Primordial - This pattern is a little odd. All systems have a lifecycle. When they start up there are some "chicken and egg" issues that are often best resolved by some "primordial" behavior. For example a framework may create and initialize a bunch of parts when it starts up. Now how do we tell a particular part or parts to start, now that the system is ready? One way is to designate one as the primordial part, and call start() on it using a Startable interface. Another way would be to send a SystemStarted event to a few primordial listeners, possibly in a known sequence.

The advantage of this pattern is there is a clean mechanism for what to start when the system starts up. Another advantage is decoupling of what is started from what starts it, allowing greater reuse. This preserves the conceptual integrity of the framework.

Public Registry - This pattern consists of a collection of things (usually objects) that is populated by one portion of the universe and used by other portions. That universe may be a single system or many systems. "Public" means the registry is available to others, as opposed to being internal to a class or such. A registry's contents may be mixed (many types) or uniform (a single type). A registry entry can be retreived by an id (the usual way) or attributes (trader style). A registry can have events, security, defaults, Hierarchial Lookup, etc.

This pattern is so useful it appears in CORBA, RMI and JINI, just to name a few. The advantages include loose coupling, higher reuse, ease of architecture design, ease of Parameter Driven and ease of sharing objects.

Service Architecture - A service is something available for widespread use. When many services have their own logical partition in a system, we have the Service Architecture Pattern. This is frequently done with a Public Registry. A subtle way is to not even let parts know whether they are dealing with another normal or service part, and let them both be retreived the same way, but be actually stored in different places. For example the call getPart(partName) would cause the normal part partition to be searched first, and then the service partition.

It is considered mandatory to use interfaces for all services, so the actual implementations can be varied.

The advantages of this pattern are clean architecture partitioning, widespread reuse, ease of changing a service implementation without breaking its clients, ease of architecture design and simplification of what the non-service parts do.

These are speculative possible future framework patterns:

Auto JIT Provision - This allows a reusable to be automatically provided with whatever it needs without even asking for it. The framework senses when and what object will be needed, and provides it just-in-time. The advantages are simplicity on the part of the part (pun intended), easy dynamic behavior and delayed activation. This is an infrequent pattern. If we can figure out how to do this astronomically well, another advantage is easy implementation of any conceivable requirement. :-)

Pattern Ranking Grid

When two or more concepts are combined, a powerful multiplier effect oftern makes the result far greater than the sum of the concepts. An example is the Assembly Line combined with Interchangable Parts.

A challenge for the framework designer is how to most effectively weave these powerful patterns into a greater whole. How do you know where it's best to start? What should be the foundational patterns? One way to approach this problem is to first select the patterns with the greatest multiplier effect, combine them into a framework foundation, and then add the next most powerful, and so on. To make that magical task humanly possible, we offer up an admittedly subjective Pattern Ranking Grid.

• Reuse Impact - How much the pattern will help reuse.
• Architecture Impact - How much the pattern will affect the framework architecture.
• Synergy With Other Patterns - The other patterns that can work synergistically with this one in a major way. These were hard to pick.
• Rank - A resulting subjective ranking of the net usefulness of the pattern.

Reuse for Parameter Driven is so high it goes off the scale. The Rank for Plugpoint is not applicable, because Plugpoints are required for all frameworks. To make the results more understandable, here are the rankings. These can serve as a guide to framework designers, who will probably create their own grids.

There were some surprises. For example I didn't expect Container and Lifecycle Controller to be so high. There is some awkwardness, such as if you choose a Hierarchy, it probably has to be designed with your Plugpoint Structure, and Circuit Based requires Loose Coupling Mechanism.

Our conclusion is the A and B patterns are mandatory for mature "Total System" frameworks. This is a bit of a status quo shift, since few frameworks are Circuit Based, Parameter Driven or use Inverted Layers.

The Exposed Delegation pattern was added after the grid was created. It's a minor pattern compared to the others.