Process Thoughts...

October 17, 1998 - Jack Harich - Home Page

(document role - new concept developement)

There's a tremendous rush towards another Silver Bullet lately - formal process. And there's a tremendous struggle to comprehend and digest the popular processes available, such as Spiral, Objectory, Capablity Maturity Model, ISO 9000, etc. It's so difficult that most get consultants to assess the organization, prescribe a plan and provide training and checkups to use the process well. It's also so difficult that most small organizations use a home grown process, since the formal processes available are hard to understand and adopt for a small outfit. We also see some large organizations rejecting any popular formal process althgether, such as Microsoft.

This seems much like the recent rush to become Object-Oriented. We have now seen that the average shop has been unable to make the transition to OO. The reasons, IMHO, seem to be perceived complexity and the inertia of years of a procedural mindset. Doesn't this sound much like the current rush to adopt a process?

Why are these formal processes such a hard pill to swallow? I started asking myself that question in early October 98, as I was reading a few new books on the subject in preparation for working with a new client on guess what, process. Here's my reaction:

Project versus Evolution = Batch versus Continuous

The main reason we are struggling with process adoption is we are (usually) trying to use the wrong process type. Most processes are project oriented, while most of software development is evolution oriented.

For example, the focus of Level 2 of the Capability Maturity Model is Project Management, while the soon to be published Unified process (aka Objectory) is built around iterations using the steps Inception, Elaboration, Construction and Transition, with each cycle typically several months to a year long. Both get complex very quickly as details and maturity is discussed.

Approximately 75% of all software development is maintenance. Why shove a project oriented process on organizations who are not even project oriented?

There's got to be a Better Way.

Many manufacturing processes started in batch mode (project) and changed eventually into continuous production mode (evolution). In most cases this change yielded over an order of magnitude improvement in productivity. Outstanding examples are the assembly line and continuous extrusion. Both were industry shattering breakthroughs.

While these don't offer a perfect analogy to a solution for better software production, they do point towards "continuous production" as the preferred concept. Since this would involve continuously changing existing software, we call the answer space continuous change.

This must mean that the most important feature of the Better Way is both your software architecture and process treats continuous change as the norm. And not only is it the norm, it is the central element. Software development must be redesigned so that lots of successive, tiny, little bitty changes continuously extrude new software somehow.

This requires a standard software architecture that does not have to torn apart, discarded, or rebuilt for each perceived product or project. And it requires a new process that does not assume everything is project based, or even requires such slowdowns as design/run or developer/customer dichotomies, and is undistracted by mundane elements like code and bugs.

Somewhere out past the Horsehead Nebula you can park your '57 Trekker and find a seldom visited, little known group of developers who are light years ahead of us. Their species is 1.2 billion years old, and has evolved to the point where merely light and entertainment is all that is desired for existence. These fellows amuse themselves by programming Stellar Quebics, which are used to nudge planets around. They mildy compete to see who can do the smallest amount of programming to make a Quebic do its work.

Fortunately this year's competition was picked up by ham radio and recorded. Here it is, unretouched:

"Good grief, the Nurino Nines team has done it again. Last year they entered nine sentences and nudged the Third Orbital Ky to eclipse precisely at Noon. The plantary hoards and hackers were spellbound. This year, well... I can't believe it. Let me rerun the Tapestry... sputter... glonk! .... whirl...

"There it is again! Look at that, hoards and hackers! The Nurino Nines have completely redefined the game. Their Stellar Quebic is no longer just sitting there with the Nines hunkered around entering Quick Thoughts. Look at that! Did you hear that! ...sputter... blip...

"This time the Quebic is asking the Nines what they would like to do. So they said 'Nudge a planet.' Astounding! It's a dialog driven programming session. The Quebic instantly responded 'What's a planet?' and so they twisted the Que's monocular swivelex to point to the Third Orbital Ky and said 'That's a planet.'

"Then, and this really did happen, hoards and hackers, believe me, that little old two foot wide Quebic said 'Okay, so that's a planet. Now what do you mean by nudge? Please be specific.' There! Do you see what this means? The Stellar Quebic is in control of all input to program it. It rejects bad input and asks for what it doesn't know, and has a perfect memory. The Quebic must be based on some kind of ultra advanced framework that allows building behavior by controlled injection in response to what appears to be intelligent dialog... fibtxx... gronk!... zzzzzzz... snap......... z... z... z..."

Unfortunately the ham radio operator was recording on the cheap, using old 2 minute answering machine tapes, and the tape either ran out or broke. But don't worry. Next year he has promised to use a larger crystal for clarity and dig around for some 10 minute tapes he thinks he has somewhere.... He says one was still in the wrapper...

I find it useful to fantasize about perfect solutions and then work backwards from there to what will work with today's technology. Thanks to the above digression we now have several identified concepts to try.

Guiding Concepts

Strategy - Anything intricate requires a high level plan to solve it:

New Technical Process - We now get bold and reckless and map out a winner: