Unpredictability

December 8, 1998 - Jack Harich - Go Back

Suppose your group suffers from unpredictability. What is the minimal process necessary to achieve predictability?

The Problem

By far the most common problem I encounter is unpredictablity. This is usually the inability to reliabiliy meet milestones such as product release or completing a feature. It can also be the inability to predict how much of a person's time will go to what area, usually due to unscheduled requests to do something that competes with their planned time on projects. It can also be the unpredictable level of bugs in a release. Indeed, an organization suffering from unpredictability finds it hard to predict anything important reliablity.

Associated with this are the reasons, aka excuses, behind each disappoitment. Examples are:

The Solution

This problem has been addressed before. We will present only the most essential elements of a solution for small groups, ie 3 to 10 developers. For simplification we will ignore issues such as politics and personalities. For reuse we will use some ideas from the Software Engineering Institute's (SEI) Capability Maturity Model (CMM).

The CMM would diagonose this as being at Level One in the five level maturity scale. This is called the Initial level and is characterized by guess what, unpredictability. Experience shows that the best way to improve is to try to get to Level Two, characterized by predictability on types of work you have done before. Level Two is called Repeatable and relies on basic management controls such as project schedules, configuration control and defect control. Anything more advanced has been found to be counterproductive until a repeatable process has been firmly reached.

Part of the problem is the group usually has these basic controls, and is still wildly unpredictable. What should they do? In fact, many groups have exactly this problem. Thinking back, I've seen this very often. Thinking back 20 years, one manager said "It's not hard to find a good system. The hard part is using it." I believe that's often the root of the problem.

So we start with the simplest possible solution, basic project management. This will provide a core process that will naturally identify further improvements. Other promising areas exist, but we must start with one thing first.

Basic Project Management

This is not rocket science. The basic idea is plan your work, then work your plan. If unplanned work appears, put it on the plan. If things are running late or changes occur, modify the plan. Keep the plan complete and up to date. Otherwise you have a plan, but it's the wrong plan, and the project is out of control.

A project achieves a goal with a plan. A plan is a collection of measurable objectives in a scheduled sequence. Resources are used to achieve each objective, also called a task. A project manager is responsible for a project. A program manager is responsible for the conceptual integrity of the software. Developers, including the managers, are responsible for tasks.

Here are the minimum steps:

Basic Project Management Steps

1. Define the project objective.

2. Evaluate the project enough to size and staff it.

3. Staff the project with the resouces necessary to do it well.

4. Develop a task schedule based on developer's estimates.

5. As each task is done, mark it done and record the variance.

6. As changes to the plan occur, reflect it in the schedule real time.

Project Elements

Project Manager - Responsible for the successful completion of the project. They must maintain the task schedule themselves. Delegating this guarantees trouble. They must also take immediate action when the schedule or quality is threatened. They need the authority to select, direct and remove resources, or a direct line to someone with this authority who will react swiftly and rationally. They need the ablity to evaluate market, technical and people issues. They also need the ability to make swift decisions with a minimum of data and know where to get that data. They do not need to be a past or current developer, only be able to understand developers and related issues. However they are usually a much better manager if they are still doing some significant technical work.

Many organizations try alternatives, such as one person responsible for the schedule maintenance, another for people management, another for selecting and changing key milestones. This is nuts.

Program Manager - Responsible for the conceptual integrity of the software. This frees up the Project Manager and gives the project a specialist on what matters most, the software architecture and "goodness". Experience has shown that separating the roles of Project and Program Manager encourages much better results. It's a left and right brain kinda thing.

"Conceptual integrity" is a term popularized in Fred Brook's "The Mythical Man Month", the best selliing software book of all time. It describes the need for complex software to have a solid, scalable, extensible, portable, reusable, intuitive, effective high level design that is implemented in the correct spirit all the way down to each program. Without this guiding wisdom, a software product cannot rise to greatness. We are talking about more than architecture here.

My rule of thumb is to have separate Project and Program Mangers for 4 or more developers. Some organizations have a Lead Developer instead or a Program Manager, but that role does not suffice. See the book "Dynamics of Software Development" by Jim McCarthy for a convincing description of the Project and Program Manager roles.

Task - Each task on the schedule needs a name, description and a paragraph of how the estimate was calculated. When the task is completed another paragraph is added explaining the variance. A task must have a physical deliverable which is the task objective. Tasks should vary from about 4 hours to 3 days. Any smaller is over management. Any larger indicates the developer has not thought through the work in enough detail, probably has a wild estimate, and makes it difficult to measure progress. Simple or standard tasks need no description.

Do not track percent done on tasks. Instead consider a task as done or not. This is known as "Pebble Inch Milestones", one of the ten best practices on the Department of Defense's list. Percent done is a diversionary illusion, since too often 90% of the work is getting the last 10% done. The reason we see percent done so often is task duration is way too long. Instead take the time to chunk out the project in smaller tasks.

If you are not having trouble with missing deadlines, you don't need this much info about each task Perhaps your problems are in quality or efficiency instead.

If you are having trouble meeting task dates, consider task descriptions, time estimate and variance explanations mandatory. Whithout them it's all in people's heads, and you spend gobs of precious time continually extracting, discussing and updating unrecorded information. Lack of this info makes it impossible to easily determine why a project is in trouble. Remember the purpose of a project schedule is managing time.

Task Definition - This should be a joint process, with the project manager playing the lead role. The developers can be given general objectives, and come back with subschedules to achieve them. The group can then combine all this into a single master plan, which may have subschedules. Usually a previous similar project or template can be used as a starting point.

An approach that helps is doing the high level Logical Component Model early, and using this to help identify subsequent tasks. For example if you had a four level Plugpoint Framework with LoginView on level 1, LoginTask on level 2, UserSecurity on leve 3 and Database on level 4, these would fit nicely into a group of tasks, with a summary task called Login and subtasks for the components.

A problem is planning tasks that are far away. Here one can plan small tasks for the next month or two, and larger ones after that. As the larger tasks approach, the developers responsible can analyze them and break them down into normal size small tasks. Some large tasks will be projects themselves. This is typical top down decomposition. If you want more accurate long term project schedules, break those distant large tasks down into small ones, which is a bit of work.

Task Estimates - This is hard, but no harder than design or coding. The important thing is to take a responsibility driven, systematic approach and get better.

It is best if the persons doing the task make the estimates themselves. As this happens they also think through the task and end up with a better understanding of how they will do it. They may also discover they need additional information or preparatory work, or that the task needs redefinition such as splitting up or resequencing. They also acquire psychological ownership of the task. They also learn how to estimate, such as by perfecting a formula using past data and ratings.

The goal here is to make your engineers self-managing. Otherwise the managers have too much to do. Managers should merely provide the inspiration, educational mentoring and services developers need to get the job done as easily as possible.

Feedback Loops - The process must have formal feedback loops. The most important involves time estimates. The steps for time variance analysis are:

  1. Set the task goal
  2. Analyze the task
  3. Set the time estimate
  4. Do the work
  5. Record the actual time
  6. Calculate the variance (estimated minus actual)
  7. Determine why the variance occurred
  8. If negative variance, take action to do better next time
  9. If positive variance, spread and reuse what you have learned

Probably the second most important feedback loop is status awareness. This is information from developers describing how well they're doing, what they're doing, problems encountered, things learned, etc. If this info hits the manager (or other developers) and they can take approproate action early, problems can be reduced or avoided entirely, plus opportunites can be seized early on.

Taking advantage of technology, this is easily done with a daily End Of Day email report to the project manager and affected developers. It gives the project manager a clear summary of where the project has moved in one day. It also gives the developer a record of what they've done, which can help on all sorts of things. It also tells the other developers what's happening, helping team dynamics. The report should be very small. It should include what tasks were completed, what work was done, problems encountered and are tasks on schedule. It can optionally include useful stuff learned. For example:

Jack's EOD Report for December 4, 1998
- Completed Data Validation Subsystem high level design task. The original estimate of 1 day grew to 2 days because I was diverted into other work. (the below reviews, the BA meeting and such adding up to a whole day) I should have realized that my role was changing from developer to adviser/developer, and that half my time would be advisory.
- Schedule is looking okay.
- Reviewed and approved JCON changes by Cindy and Joshua
- Discovered that Internationalization will require large server changes
- Fixed 2 bugs in Data Framework

Note the Time Variance Analysis step 7 after the task completion notification, with the root cause analysis of the variance.

An interesting benefit of EOD reports is they keep developers from "going dark". This phenomenon occurs when a developer starts a long task and no one else has any idea how well it's going. Smaller tasks can also reduce this problem.

As one wag put it, avoid status meetings, not status reports.

Risks - Projects are risk driven, software is architecture driven. The first thing to do once the objective and basic functionality of a project is understood is to plunge right into Risk Management. This will give you the biggest bang for the buck the fastest.

As the project rolls along, resolve the high risks by scheduling them first somehow. Then do the medium risks next, etc. As the project nears completion, it should be running smoother and smoother. A project in trouble has always muffed up on Risk Management. For example, interruptions, staff availability and bad time estimates are risks that must be managed directly. See further info.


Where To Start

If you are having trouble with unpredictibility but are already using project schedules, try these key things:

You will quickly discover that the developers will find what and who is causing problems.

You may quickly discover that much project time goes to other work, such as support, defects, explanations, reports, training, interviews, meetings, personal things, sickness, vacations, rework, phone calls, email, travel, lunch, various standing responsibilities, etc. Congratulations. You have discovered that no one spends 100% of their time on a project. The real issue is how to change things to allow a predictable percentage of people's time on a project. If this is a problem, start by measuring where people spend their time. There are time tracking tools on the market to do this.

You may discover that many people are spending less than 50% of their time on a project, and that these tend to be the projects in trouble. This is because a person's effectivenss is "dissipated" if they are doing too many different things at once. A golden rule is "No one can do more than one thing at a time well", with about a 2% exception. Reorganize responsibilities so that people average 80% of their time on the front half of projects. The second half is usually easier in a well run project, so concentration can usually fall without large problems.

You may discover that people are complaining about too many interruptions or distractions This is a big problem, because it prevents one's mind from performing at full potential. Studies have shown that people perform best when in a "State of Flow", which takes 15 to 20 minutes for the average person to reach. An interruption causes the mind to have to start over to achieve a State of Flow again. So if someone is interrupted (or changes to different work) once an hour all day, they have lost 2 hours and 40 minutes. If their "rereach time" is 30 minutes (like mine) they have lost half the day. This is why so many people work best alone in the early morning, evening and weekends, or at home. If the type of work is changed and the type of work is mentally difficult, mentally shifting gears may take hours or days.

Take action if you have an interruption problem. Encourage people to schedule large chunks of time where they tolerate no iterruptions and work on a single task. Encourage people not to interrupt others, but to instead use email, short scheduled meetings, and lists of questions and issues.

Consider changing to rooms with doors instead of cubes(1). After all, a developer costs about $100,000 a year. A 10% productivity increase far outweighs the cost of rooms. Microsoft has given every developer a room with a door from day one, and considers this a strategic advantage.

If quality variance is a problem, begin Defect Management, a separate issue. But really a task is not done until the results have reached a certain quality bar, so a well defined task covers quality. If defects are discovered later, track them back to the task that created them and examine why they were not caught there. Often this means concurrent testing or Use Case validation was not employed.

One must remember not to slip into schedule madness. There are other things far more important, such as architecture, reusables, quality, product design, and the people involved.

For further ideas, such as a resuable approach to projects, see Development Process Overview.

(1) Quoting from an authoritive source:

"Laurie's our new engineer. Show her the ropes, Dilbert."

"This is your anti-productivity pod."

"It's equipped with a little device that rings anytime you try to concentrate."

"The top is open so none of the background noise is inadvertently mufled."

" And you're on the main aisle, so you'll be haunted every minute by footsteps behind you... step... step... step..."

In 'Seven Years of Highly Defective People', Scott Adams on page 55 added a foot note to this strip, saying "I believe cubicles were invented by someone who didn't like people."

Actually I'd like to find some studies comparing the effectiveness of cubes to rooms. It may be that those who need to mesh with others a lot are better off in cubes, and those who need to think a lot do better in rooms. It could also turn out that rooms are becoming better than cubes because when people want to collaborate they can do it with email, news groups and web sites.