September 4, 1998 - Jack Harich - Document Map
Configurability is that quality allowing system behavior to be varied by a small amount of user input. The more configurable a system is:
- The easier it is to build and modify.
- The more of this the user can do.
- The higher reuse is.
- The cheaper the system is.
- The faster it can be built.
- The more defect free it tends to be.
We use the definition:
Configurability - The ratio of system variation to input effort.
Thus it's no surprise that many shrewd developers are pursuring high configurability as a key goal. Simply put, configurable design leverages a developer's efforts to the utmost. Think about it this way - What's the smallest amount of effort I can do to control the greatest amount of system behavior? Isn't this what productivity is all about?
Mechanism examples are drag-and-drop tools to build the desired result, parameter dialogs to gather settings, initialization files, system preferences or options, and styles. The market is in a severe state of churn here, as the rush to find better ways to configure systems causes one experiment after another. No clear winner has emerged yet.
Note we said "input effort" and not "input amount". Suppose we have 100,000 lines of code (LOC) in a domain framework A that averages 10,000 custom LOC per deployment, ie 90% reuse. Framework B has 100,000 LOC and averages 2,000 parameters that need to be set via dialogs, which generate 50,000 additional LOC, ie 66% reuse. By this measure A is better. But since A requires a highly skilled hotshot programmer and B can be done by any sharp domain expert faster than A, B requires less effort and is the more configurable system.
Different technical configuration mechanisms include:
- Subclassing to "fill out" behavioral portions of a framework
- Factory patterns with the type as a parameter
- "Resource files" such as those created by GUI builders like Visual Basic, C++
- Code generators of many kinds such as drag-and-drop GUI builders
- Hand coding to fill in various behaviors
- Initialization files
- Repository settings
- "Federation Properties" such as in style sheets
- Parameters in databases
Note we are trying to find both user input mechanisms and internal design patterns to leverage input.
How in the world are we going to deduce the best configurable designs? Consider the typical use case:
- Desire - The user has a vague, hot burning desire for something.
- Concept - The user or someone else has a clear concept of what they want.
- Directive - This is expressed physically in enough detail to tell what is wanted.
- Builder - Someone or something builds the system, given the directive.
- Behavior - The system behaves as the user originally desired.
We now compare two of the leading ways to build systems:
Object Oriented Code
Parameter Driven Components
|1. Desire||Nutshell Vision||Nutshell Vision|
|2. Concept||Requirements||Select and arrange components|
|3. Directive||Design||Set parameters per component|
|4. Builder||Code custom behavior||Pass parameters to components|
|5. Behavior||Running code with 20% to 80% code reuse||Running code with nearly 100% code reuse|
Not listed is Code Driven Components. This is really just a high reuse variation of Object Oriented Code. We reject both because in practice most groups are not hitting over 20% reuse. OO has turned out to be difficult for millions of procedural programmers to adopt.
We see the bottleneck is in steps 2, 3 and 4 with the current popular best practice, Object Oriented Code. This bottleneck nearly vanishes in the future best practice, Parameter Driven Components, but only if we can figure out a clean parameter mechanism.
We must avoid resorting to code to set component parameters or our gains will be less than astounding. Right now tools such as PowerBuilder, Visual Basic, Dephi, JBuilder, Visual Cafe, etc generate code to set component parameters. This is real problem, since such code cannot be shared across tools, is usually a bear to fiddle with manually, and really only defines a small portion of large system behavior. Such tools leave the rest to hand coding and clever reuse practices such as libraries, frameworks, subclassing, etc. There are high end tools that do better.
And there are frameworks such as SAP and PeopleSoft that are parameter driven to the core, and are sweeping up customers fast. We call these Enterprise Frameworks (EF).
EFs have the drawback of not handling most user concepts. They are also rigid - your business processes must conform to what the EFs call "best practices". They may or may not evolve as your needs do. They are also domain specific.
Having rejected code generation and EFs, how do we capture a user's intentions and turn that into behavior? We can look to the field of Artificial Intelligence for a hint. They call this "declarative knowledge", where the user declares what they want to do, and the system handles how to do it. This massive decoupling suddenly gives us great leverage and focus.
As they say in AI, the hard part is representation, representation and representation. Some of our options for how to express declarative knowledge are:
- English like expressions such as "Edit the UserSys table in the ATSDR database, sorted by..."
- A language such as Prolog or Lisp.
- A particular format depending on the task type.
- Lots of relational tables of parameters.
- A markup language such as XML.
- A new language designed just for expressing Directives per component.
After numerous experiements we have settled on the last option. We combine this with a hierarchical organization of components. With these two simple mechanisms and a powerful implementation language (Java) we can now proceed. (As XML and related DTDs and tool mature, we may move to XML)
Best Practice for System Configurability
Parameter Driven Components
1. Desire Nutshell Vision 2. Concept Select and arrange components 3. Directive Set parameters per component 4. Builder Pass parameters to components 5. Behavior Running code with nearly 100% code reuse
Select and arrange components - Components are selected from a reusable repository containing high quality components. They are arranged in a visual hierarchy that represents the desired system. This is all very simple and intuitive. We keep our declarative knowledge as simple as possible.
In the case of a "missing" component the user merely uses a dummy component and proceeds. The component developers will create it once its parameters are designed.
Set parameters per component - The user edits the parameters with a visual tool that insures parameter integrity. In the case of GUIs a typical drag-and-drop WYSIWYG can be used. A developer or trained user may edit parameter text directly.
In the case of a "missing" component a component developer assists the user by editing parameter text until the desired behavior has been declared. After you've seen a few dozen parameter designs this becomes suprisingly easy. See Parameter Examples. The developer then uses the parameters as the component's requirements, creates it, gets user feedback, deploys it, and adds it to the component repository. This is a tight, focused, productive process, fitting right in with our Continuous Change Process.
Pass parameters to components - Each component that needs to be parameter driven implements ParamDriven. When the component is initialized its paremeters are converted into a data structure we call a Param and is passed to the component via setParam(Param param). We use a single method here, not lots of Bean properties, which fail to work for more than simple properties. Use of the ParamDriven interface also allows the method applyNewParam(Param param) so that modified parameters can be applied at runtime. While this can also be done with Bean properties, the code is overly intricate if any fancy behavior is involved, because the class must have special logic for many setters, not just one.
We have achieved our goal - Configure systems, code reusables. The above Parameter Driven approach uses no code whatsoever to assemble a system from components. Notice the seamless transition from one step to the next by using parameters, not code.
We have also hit very high configurability. With parameter driven components organized in a hierarchical tree we can vary 100% of a system's behavior. With decent visual tools and high quality components, sharp users, domain experts or assembly specialists can assemble and configure node behavior about as fast as they can decide what they want. Input effort is very low, and will not become lower until we have AI assisted tools. The simple paradigm of System Trees and Node Parameters will take us very far.
It is more work to create high quality parameter driven components (most are subsystems) than one shot code. We estimate this as 2 to 5 times as much work. It pays off very quickly. It also requires very skillful, experienced developers who can learn how to do it by studying existing good and bad examples.
We have simplified the above a bit by leaving out further detail such as container parameters. These are only a little more complex than component parameters and use the same language.
Change Point Theory
There's always another way to look at things. All systems have change points. These are the main system input locations to effect change. Thoughtful system design includes minimizing and rationally locating change points. In general the fewer the change points, the more efficient the system, because the absence of a change point indicates a process step is automatic and not manually driven. The type of change input must also be well designed.
In the flowchart below, examine the change point locations. The Code Driven system has two change points while the Parameter Driven system has one. This indicates the powerful gains of abstracting systems into Declarative Knowledge, a Builder and System Behavior, which allows us to apply the concept of Configurability.
If efficiency varies inversely with the square of the number of change points, then a Parameter Driven system will be four times as efficient as a Code Driven one. This formula is a reasonable guess of course, but is probably in the ballpark or even too low. My personal hunch is Parameter Driven Components represent the next generation of software, and will boost productivity by an order of magnitude once we get the kinks worked out, and stop using it for just the GUI and data edits like in most IDEs.
Let's examine the pipeline of abstractions. The old way of thinking was "Capture what we need in documents and throw it over the wall to the designers." The new way is "Capture what we need any way we want, and ultimately express it as pure Declarative Knowledge, so that the next steps can be as automatic as possible." This means:
In a Parameter Driven system the Requirements step culminates in Declarative Knowledge, a radical notion.
So we no longer call the first step Requirements, but Parameters, since that is the end product.
Now that we have expressed WHAT to do with parameters, HOW do we do it? The old way was "Let's write Lots of Clever Code to turn these Requirements into System Behavior." The new way is "Let's write all that Clever Code once by partitioning the Clever Code into a System Engine that reads the Parameters and uses them to drive Parameter Driven Components." This means:
In a Parameter Driven system Custom Programming is replaced by a reusable System Engine which drives Parameter Driven Components.
Egads! Zounds! Is this possible? The author can assure you it's been done many times, in many ways. We are merely formalizing the new concept here, to disperse it to the winds blowing through the turbulent and seed receptive minds of the new crop of developers who are seeking A Better Way, which can be described with a mere three principles:
Now that we have explained the magic of Change Points and Configurability, if you are striving for Configurability a potentially better term is Config Points. Think about it. Where in the systems you build are your CPs? How many are there? Can they be reduced? Can they be reorganized? Would changing the type of CP input help anywhere?
Actually we lean towards Change Points since it has one less syllable. :-)
Note - The above flowchart is highly simplified to emphasize the abstraction employed. Actually all process steps have change points, but with good design most process steps can have minor change points. For example people need to be recruited, retired and replaced. Components will need to be acquired, built and modified. But this is minor compared to the continuous, massive change occuring at the major change points.