The BA Project
Created 4/1/99 - Last Updated 8/27/99 - Jack Harich - Go Back
The Bean Assembler (BA) project is the reference implementation of the Ultra High Reuse (UHR) vision. This is an Open Source project whose end product is the BA kernel, visual tools, parts, educational materials and related items. Strive To Be Architecture Driven
Preserving the Conceptual Integrity of the BA is our greatest challenge. Executing the UHR Technology Grid is another. Thus our software and process is architecture driven, instead of code driven with no mature process.
This page has Conceptual Models of the entire BA Project infrastructure, process and architecture, for understanding and navigation at the highest level. The models express the goals we wish to achieve and how to get there. Click the model for summary information and then drilldown to project areas.
Deep study of these models will give you an intuitive understanding of exactly what we are doing and how we are doing it. This is exciting work, on the leading edge of software technology. It is so difficult that we have taken the time to keep oursleves well managed, as you can see from these models. We don't want to get bogged down in formality, so we're keeping all this as simple as possible. Everyone will be continually using these models, so we will all be thinking at the same high level of abstraction as we go, working to be beat of the same drummer....
Adjust the splitter so only the model shows. Hold the mouse over a model area. High level info will appear here, in the lower frame. You can also drag the splitter to increase the size of the lower frame, and read all the text at once.
Much of this document is designed to be reusable. Businesses using UHR or the BA Project can use the same or similar items. We try to be generic or cover both. We use the terms users and customers interchangably.
(Only model mouseover works now. Model click not yet implemented because we have no other pages to refer to yet. Not all items will have click pages. Clickable items will have a clickable indicator.)
Infrastructure
Good infrastructure is invisible. It helps when you need it and then vanishes. It doesn't slow you down. It works so smoothly you rarely notice it, except when it fails.
Communication - The easier, clearer and faster people can communicate, the better their collective output. Communication is the spreading of knowledge and related data, using a variety of mediums such as personal (verbal, visual and auditory) and physical (text, audio, webpages). The less we have to ask each other, the better, so the quality of physical material is very important.
What do you communicate? Everything others need to do their job well. This includes Mini Process documents, models, strategy, architecture, product documentation, etc. It's really all your "artifacts". As Joshua Marinacci observed, "A lot of communicatin is just recording what you've done." We have identified these communication areas so far:
Website - The biggie for communication. The BA Project has these:
Michael Ivey's server and website, just started.
A business using UHR will have an intranet, internet, extranet, etc.
JSL - For general discussion about Java, OOAD and advanced issues such as reuse. This is not a beginner list. To join, email Greg Kreis, and say why you are seriously interested in these topics.
ba-general - For general discussion about the BA and the BA Project. To join this list or any of the following lists, email Michael Ivey or Paul Reavis, and mention your particular interests.
(More lists. We need to review which are best.)
The Improvement Pipeline is our main process. This is centered around sound artifact and codebase management. We divide this up into Work and Product items. The Product items form a pipeline runnning from objectives to stable releases, with change resulting from the Work items. The steps in the pipeline are well organized and defined, giving us an honest to goodness Defined Process to follow every day. The main repeating cycle is Stable Release.
Release Management is the most important subprocess. It supports the Active Changes, Unstable Releases and Stable Releases steps. It includes version control, configuration management, version builds and validation, version distribution. There is a beehive of activity here.
Our most important result is not just a good product done well and fast, but a reusable process. We hope much of our process and associated infrastructure can be reused by those adopting UHR.
Improvement Pipeline
The big idea here is to continuously improve what we've got, by following our strategy, using our process and listening to input. We go for a long series of small incremental improvements, avoiding big bangs that are too often big flops. This is exactly what biological evolution does. There of course is an occasional mutation. :-)
A really good pipeline lets you "drop improvements into the pipeline" as easily as dropping a piece of paper into a folder. Ahhhh, we've finally explained that animation! :-)
The following are the pipeline items. Size is relative to effort. Note that nothing happens in Product items - Things are only stored there. The Product items represent the Product you are building, with the thick blue arrows showing the conceptual flow.
Perceived Value - The responsibility of this group is to "optimize perceived value" of the product. Often called "Quality" or such, this group tends to appear in stages. At first, everybody or management is responsible for value, and testing is done by developers, with too much of it at the end. Later a dedicated Testing or Readiness group appears. Much later the organization realizes that most problem cost occurs in requirements and design, and creates a Perceived Value group (or any good name) that looks over the whole process to make sure it's done well, with the customer in mind. In a mature organization that group (or the equivalent) dominates product development. Really everyone should think in terms of "optimize perceived value".
We are deliberately going beyond the usual idea of quality here. We want to include continual readiness, learnability, usability, likeability, cost/benefit, long term impact, flexibility, compatability with other products, really all the stuff it takes to exceed the customer's expectations. The idea is to get into a mindset where you're thinking like a user, wondering how the product will be perceived, what value will be seen. Notice it doesn't matter what the developers think of the product - It only matters what the users perceive.
The end result of Perceived Value is the same as Stable Releases - happy customers.
The end result of Strategic Input is a current Strategic Plan.
Strategic Plan - At the architecture level we have the UHR Technology Grid, a 5 to 10 year phased plan from the software design viewpoint. This is the driving force behind all work. As more and more developers start using UHR, we expect their joint improvements to this grid to make it even better. Businesses can supplement this grid with their own versions.
For the BA Project - At the business plan level we have nothing in writing. We're working on that. It's expected to include objectives, priorities, narrative, schedule and staffing. We are hoping to soon formalize UHR development with a legal entity.
The purpose of the Strategic Plan is to guide Mini Process work.
A Mini Process document starts its life in this step. The goal is to formally define a significant work area, such as a subsystem, rewrite, or tool. We only "define" what needs to be done enough to size the work, nail down all high level issues, and give a complete description of what needs to be done. This is pretty much the Nutshell Vision, initial Narrative and initial Requirements. In some cases high level Design is required. For someone experienced with the Mini Process and the BA, this takes a few hours to a few days in most cases. The important thing is to capture all decisions and plans in writing.
Once the High Level work is done, it's ready for a handoff to Working Groups. The document is added to the Change Request backlog. It may be seen as a single change or a change set.
Some groups will have a schedule associated with their work. Since scheduled work is proactive, it enters the pipeline in this step as a Mini Process document or part of one. Don't get scared about too much work here. As little as a one sentence Nutshell Vision is all it takes to start a Mini Process. Most scheduled tasks will have a Nutshell Vision and at least a one paragraph narrative. Anything less than this implies lazyness and a subconscious wish to become a dinosaur. ;-)
The end result of High Level Input is a Mini Process document ready for further work leading directly to implementation.
The purpose of the Mini Process is to drive most of the identification and prioritization of changes in the Change Backlog, and serve as high quality input for Working Groups.
Change Status
Stored in Pipeline Step
Submitted Change Backlog In Review Change Backlog Approved Change Backlog Disapproved (Inactive) Combined (Inactive) Started Active Changes Completed Unstable Releases Tested Unstable Releases Released Stable Releases Changes are an abstraction that flows through the pipeline with different statuses. Bugs are just another Change type, and need the exact same handling. Bugs often get a higher priority. A Change may be combined with another, in which case the first Change is ended.
There will be the usual form for making a Change Request, aka CR. In the case of bugs, the developer may have already fixed it and is submiting the fix as a change to be reviewed and made to the master copy. Some improvements may happen this way too. For improvements, whoever reviews the CR will translate it to a new Mini Process document or modify an existing one.
The end result of Change Requests is an addition to the Change Backlog.
Let us pause to consider that "The main reason a change is hard (or one fix causes another bug) is extensibility wasn't designed in". Thus the bigger a problem your Change Request Backlog is, the poorer your extensibility probably is.
The purpose of the Change Backlog is to give Working Groups clear input.
When work is started the Change status goes to "Started" and the Change becomes an Active Change. When done it goes to "Completed" and the change enters an Unstable Release. Unit Tests must have been run, along with necessary integration tests. The developer should be super confident their work is complete, meets requirements and design well, and is defect free. There will be some defects and improvements noticed in the running system, which are fixed in Testing.
The preliminary result of Working Groups is Active Changes. The final result is Unstable Releases.
One of the Department of Defense's top ten Best Practices is "Pebble Inch Milestones". This means something is not considered done until it's 100% complete. This avoids the classic, "When you think you're 90% complete, you're half done."
The purpose of Active Changes is to allow better management of work in progress.
Frequent releases make versions widely available. With good infrastructure, this is easy and requires no work. All our users do is download or get a release, which includes code and related files, parts, documentation, educational materials, samples, etc as needed.
The purpose of Unstable Releases is to frequently make new usefulness available and allow widespread testing.
Testing usually decides a release is not yet stable, and so prepares another Unstable Release. These are each a little better, and the final one is a Stable Release.
Testing is done in a variety of ways. The most important is peer review, at either the Mini Process, code or running system level. Often Beta Tests are needed. Whatever approaches are taken, an extensive formal testing program is required.
Please note we are not trying to "test in quality," which cannot be done. We are only doing the usual extensive integration testing all large software requires. Lots of preliminary quality work will have already been done by Perceived Value and Working Groups.
The preliminary result of Testing is Unstable Releases. The final result is a Stable Release.
Stable Releases are hard to do, so following in the finest tradition of Open Source, we don't do them very often. Only after an Unstable Release has gone through widespread peer review by banging away on it real hard, plus dedicated full regression testing, plus other types of testing, and the new defect discovery rate is extremely low, do we consider a release stable.
The only possible purpose of Stable Releases is happy customers.
Layered Architecture
We take a Layered Architecture approach to enhance understanding, reuse, stability, flexibility and simultaneous development. These layers contain our prime deliverables.
What is Layered Architecture? You partition a system into layers with the most reused items at the bottom. Any item can use other items in its own layer or lower layers. If upward calls are required then events must be used. This arrangement yields systems and reusables of amazing value. It's resilent to change and growth. Gone is the old juggling act of "Where does this get done? How do I change A without affecting B?" In its place is a standard, highly organized, loosely coupled architecture suitable for all non-trivial systems. Here's the BA Architecture.
System Layer - The purpose of our work is better productivity in building systems, so we provide samples for study and production systems for use. The Visual Tools are BA systems themselves and were the first such systems created.
Eventually Visual Tool construction will dominate the entire BA Project, because of the huge productivity gains they will give us. Advanced innovative Visual Tools are very, very hard to design, but our time is better spent in such leveragable domain neutral reusables than in domain specific systems. A watershed will be when we start using one Visual Tool to create another with no code. At that point productivity will leap an order of magnitude. Five to ten years from now we will look upon today's BA Visual Tools as quaint old fashioned collector's items, or worse. :-)
All Plugpoint Frameworks have Sample Systems showing how to use them. Close study of these will also help you to learn how to design your own frameworks.
Message Subsystem - Iteration One, Iteration Two.
Other - Hmmmm, curious to see what accumulates here....