Value Driven Software Evolution

Created 4/23/99 - Last Updated 4/23/99 - Jack Harich - Go Back

Biological life forms have done rather well using evolution and "Survival of the Fittest". Can we apply these abstractions to software, using more rapid evolution and "Perceived Solution Value"? This document explores how to do this by examining the mechanisms involved.

The Key Abstractions of Evolution

No complex entity can be completely designed and built right the first time. Instead we rely on evolution to continuously improve entities one step at a time. Evolutionary mechanims vary depending on the entity type.

An entity can be conceptual, such as the laws of physics, or physical, such as planets, software, hardware, people, countries, plants and animals. The two can combine, such as in the princples governing people's behavior. The two form a continuum. The more conceptual an entity is, the less obvious its evolutionary mechanisms are, and thus the harder it is to evolve. This explains why hardware is evolving so much faster than software, and hence why software is still in its very young evolutionary stages. How to use the abstractions of evolution to drastically speed up the improvement of software is the intent of this document.

Evolutionary mechansisms fall into two groups: those affecting how change in new entities occurs, and those governing whether an entity contributes to the next generation. This is an important distinction, allowing us to focus on these separately.

1. How change in new entities occurs - This produces new entities with new features. It is the total product of the features that determine an entity's behavior. It is the behavior that determines an entity's "value".

2. Whether an entity contributes to the next generation - This is the system feedback loop. Somehow something decides if an entity will contribute to the next generation. How that decision is made is the prime driving force in that type of entiy's evolution.

To summarize, we see that evolution is How Change Occurs and Feedback Loop Decision. These are the key abstractions of evolution, apply to entities of any type, and can be used to better evolve software entities.

Evolution is How Change Occurs and Feedback Loop Decision.

Biological Evolution Mechanisms

Looking at the evolution of plants and animals, we can see it's gone rather well. This is due to the mechanisms used for reproduction and the mechanism used to determine who reproduces. These mechanisms are:

How Change Occurs - Reproduction

  1. Continuous improvement over a long period of time, giving many cycles of parent/child
  2. Incremental changes via sexual reproduction resulting in genetic combination of the two parents.
  3. Drastic changes via sexual or asexual reproduction and random mutation.

Feedback Loop Decision - Who (or what) contributes to reproduction

  1. "Survival of the fittest" determines who lives long enough to reproduce. This includes how parents and communities teach and help their offspring.
  2. "Contribution from the fittest" determines who contributes to the offspring. This is often done by competition among suitors for the favor of the "best" female, as in many animals, or attraction by the "best", as in honeybees and flowers.

Software Evolution Mechanisms

Feedback Loop Decision has the main drivers, while How Change Occurs has supporting drivers. Systems Theory tells us changes to main drivers affects a system the most. Thus the more mechanisms we can put in or improve in Feedback Loop Decision, the better the system.

We start by anwering the probing question, "How can we best determine what contributes to reproduction?" We feel the best answer, at the most appropriate level of abstraction, is "Perceived Solution Value". (PSV)

All software is a problem solution. All software users perceive the value a system provides them. They may differ on their perceptions, but each is correct from their viewpoint. An important distinction is to use "perceived" value. This puts the emphasis on what a system really gives the user, acknowledging the user is the true expert.

Probably our top strategy should be how to speed up the Evolution Cycle. The second top strategy should probably be all users have different needs.

We now spare the reader a long tedious boring discussion, and jump right into listing mechanims that seem to have good potential. Since a part can contain other parts and can be an entire system, we use the word "part" to mean system or part.

How Change Occurs - Reproduction

  1. Let users customize or create their own parts.
  2. Give users choices of other parts they can use.
  3. Let editors suggest the "best" way to customize parts for the context involved.
  4. Let editors suggest the "best" parts to use for the context involved.

Feedback Loop Decision - Who (or what) contributes to reproduction

  1. Monitor user behavior in How Change Occurs mechansims and save the data.
  2. Let users rate a part or feature on its PSV. This can be relative to other parts or absolute using a scale. Save the data.
  3. Use the data from 1 and 2 for new part design and which parts to disseminate. This makes software evolution "Value Driven". User customization, creation and part choices are an accurate and immediate measure of Perceived Solution Value, but fall short if the user cannot or does not find something better. User ratings are only an approximate measure, but are necessary for when decision monitoring falls short.
  4. Make user customized or created parts available to all users.

How To Proceed With Any Part Centric Industry

(To be continued after we have stabilized the above abstractions and strategic mechansims. We will avoid being BA specific to encourage thinking at a high level.)

Software is already evolutionay, it's just way to slow, indirect, and vendor controlled....

Talk about how this is all really just better process....