To be or not to be Predictable?

This article was originally published by Serious Scrum on Jan 5. 2022 Written by Kurt Nielsen.

“It will happen when it happens!” Does it really matter?

Recently the issue of predictability has come up in discussions a lot, perhaps triggered by almost two years of uncertainty due to the Coronavirus. It seems that increasingly, we have to regroup and repurpose our resources all the time due to constant changes in the surrounding conditions.

For us who work with Agile and Scrum in particular, the idea of constantly adjusting the course of action — also known as “inspect and adapt” — is no stranger to us.

That is what we as Scrum Trainers try to give people the tools to do.

But mostly predictability is not spoken about in Agile circles. There has been a tendency to just look at the “here and now” in the next couple of sprints close to the tactical execution. That is of course important, indeed necessary; but it has often been silently assumed that if you zoom out sufficiently and only look at the big picture, then goals can be set, plans be made and follow up done the traditional way. The tactical realm then has to absorb the uncertainty and “inspect and adapt”. It can be argued that a framework like SAFe at its root rests on exactly that paradigm.

But a certain uneasiness and suspicion has crept under the doors like a fog. Maybe this is not a workable model anymore, now that the world apparently has moved deeper into complex territory.

Now, I am not an Agile or Scrum theologian who analyzes deep semantics of The Scrum Guide or the Agile Manifesto; but it came to my attention that the Scrum Guide (since 2010 ) has stressed the importance of predictability with statements like

“Scrum employs an iterative, incremental approach to optimize predictability and to control risk.”

and

“Sprints enable predictability by ensuring inspection and adaptation of progress toward a Product Goal at least every calendar month”

At least the last statement indicates the long term “Product Goal” is the focus of our efforts to achieve predictability. And yet it is rarely spoken of in the Agile community.

Although Agile and Scrum are generally accepted as a good way of working on complex challenges, there are still areas of usefulness to be explored, predictability is one of them.

Why the interest in Predictability?

Anybody trying to lead an organization knows there are future obligations and commitments to be met. There are contracts, payroll, rent and other commitments to be fulfilled; there are owners, banks, government and employees that have expectations to be honored. Wouldn’t it be nice to be convinced that we can deliver those things and sleep well at night? That requires that it is well known what the organization can be relied on to deliver. The rapid change in our environments is putting pressure on leaders everywhere to tighten up their commitments reaching into the future.

One company we work with is under pressure from its large clients to reduce delivery times, being more predictable and at the same time innovative. These are really conflicting demands.

In the ever-changing and highly competitive world, those who have a better handle on their organization’s predictability clearly have an edge.

What is predictability in the complex domain?

When invoking the term complexity, we normally use Dave Snowden’s Cynefin model as a definition of terminology:

• Obvious/Clear — Where there is widespread, solid and stable knowledge about cause and effects, any reasonable person can see what needs to be done.
• Complicated — Knowledge is there but harder to get, work of analysis or research is required before passing judgment on the preferred solution, indeed several solutions can be fine depending on the expertise at hand.
• Complex — Only fragmented knowledge exists, but we have to act anyway. An explorative approach with experiments and verification of hypotheses is called for.
• Chaos — No understanding of cause and effect exists. We have to act based on (hopefully) informed intuition to try to stabilize the situation.
• Disorder — Where people have not understood the problem yet in its context. It’s often the result of a failure to see past their own biases, habits, and entrained patterns of solving things.

We are familiar with work in the Obvious/Clear and Complicated domains, in fact practically all management instruments for achieving predictability rely on our challenges being in these two domains. In Chaos and Disorder there is no basis for predictability. Complexity is the domain where we need to sharpen up our act, and where a novel approach is required.

The most challenging aspect of trying to predict anything in the complex domain is that we have to operate with probabilities. We cannot make a fixed plan to reach a desired end state, we have to express it as a range of possible roads each with its own probability. In fact the end state itself also needs to be expressed as an area of possibilities, not a specific point.

The goal is to reduce the variability of the result of our efforts, in terms of time, cost and outcome. We do that by deploying methods and tools that reduce the number and severity of unhelpful decisions.

In the picture above, the two travelers in this complex territory have a goal, they want to reach the high plateau on the horizon (specifically not a particular mountain top), anywhere on this range will be good. From their present perspective, there is a path leading almost in the right direction, so the first leg of the journey is not that complex. However, they have precious little knowledge about what is hiding behind this first row of cliffs. That is their life condition.

Of course, they should seek all the information that they can, as long as it is not cost-prohibitive, neither in terms of time or money. For example, it would be of immense value to the travelers if they had access to Google Earth of the area.

 

There they could then get an idea of the terrain, some measure of distance, What is hiding beyond the first ridge etc. even though the camera is at 16 km height. So what experiment can they do to get a cell connection and access to the service and the information? It is still not possible to make a detailed plan and prediction, but it is better.

The illustration would have been better if there were five-seven travelers instead of just two because their combined observations and experiences create a better chance of making good decisions and predictions. As they travel together, they build up experience of the terrain, their capabilities (how far can they go in an hour in this territory? When will they arrive?) and possible endpoints on the value plateau.

Although this is just a crude illustration, it shows some of the elements of prediction and forecasting in the complex domain:

• Have a negotiable goal or target, an area of acceptable value generated, not a single fixed point.
• Estimate parameters in ranges, be specific about risk. Do this repetitively to iterate towards an acceptable target.
• Work in teams and iterations to constantly seek more information and better understanding of the fragmented knowledge available.
• Building up expertise and experience to gradually improve the quality of forecasting and decisions.

Scrum is the perfect platform for Predictability

So finally let us look at why Scrum is such an ideal platform for also achieving the value of greater predictability.

Negotiable Goal

In Scrum there is a Product Backlog with Backlog Items (potential deliverables), each of these has a certain estimated value, cost and risk. Iteration after iteration the Product Owner facilitates prioritizations about setting the goals for the short term — the iteration.

Some of the Backlog Items can be “must-haves”, others “exciters” and others can be quantifiably compared for value (the stakeholders must be involved in this evaluation). To a degree predictability is in the hands of the stakeholders, an unwelcome fact for some! Not everything can be a “must-have’’ and not everything can have fixed deadlines, there must be some maneuverability left for the Team that does the job.

Estimate in Ranges with Risk

We always try to persuade Teams to estimate the effort (and the monetary cost) as a range. We ask them to assess their “Best estimate”, what they believe most in, where they say “It is equally possible that it is smaller or bigger”, that is a 50% estimate. We then ask them what they think the Worst Case is? This is not a precise mathematical definition, but most people tend to think something like “With 90% certainty, it is not worse than this”. There we have our uncertainty, that is very useful to know when choosing Backlog Items for the next Iteration.

Do the same with value estimation. Get the Stakeholders to think in terms of probability and uncertainty.

Teams that estimate in ranges are more likely to get full team participation and open and objective responses. And, by adapting the Backlogs to reflect the 50% and 90% ranges, expectations are framed in a more realistic and predictable way.

In Scrum, the Backlog Refinement is a perfect place to assess and reassess estimates.

Work in Sprints and Teams

The Sprint in Scrum represents a perfect learning cycle, big enough to get something done and small enough to remember during Review and Retrospective what actually happened. This also goes for improving the estimation and therefore the predictability.

The timebox of the Sprint forces the Team to decompose Backlog Items into manageable chunks that can be completed in a Sprint. Together with Scrum’s radical focus on “Definition of Done”, uncertainty is seriously reduced. Working on large pieces of work is inherently much more risky, as everybody has a hard time fully grasping complete understanding; furthermore what is done is done and has now zero uncertainty.

In each Sprint a batch of Backlog Items is taken across the border between Complex and Complicated in Cynefin parlance, so a sufficiently precise goal for each of these can be set. Next Sprint a new batch and so forth with the necessary hypotheses and experimentation. This also reduces risk and improves predictability to address experiments constantly in smaller quantities.

As alluded to above, the fact that a Team works on understanding, estimating and forecasting together improves the quality of this work enormously. For example, the practice of doing Planning Poker is not just to get a number, but to constantly increase the shared mental model of the solution. We get a strong focus on working in Teams from Scrum as well.

Building up experience

Being able to predict and estimate with any sort of precision comes with experience of the domain we are in, the tools we use and how the journey has been tackled so far. We also learn about our capability; how much can we accomplish in a Sprint? Some Teams measure their velocity in terms of how many Backlog Items or Story Points (a relative measure of size) can typically be accomplished in a Sprint, this is used for making better predictions. There is no substitute for learning and experience, again the Scrum Framework is an ideal starting point.

• Teams are stable and get to know each other and intuitively understand each other’s reactions.
• Daily Scrums help everybody focus on the near-term Sprint Goal. One goal met is one less to worry about, the uncertainty of “Done” things is zero.
• Retrospectives are there to provide a disciplined reflection and learning event.
• Frequent Sprint Reviews of small batches of solutions reduce the risk of getting seriously off track in relation to stakeholder expectations. In fact, this is one of the major risks often not talked about enough: the risk of delivering something other than what the customer thought he/she asked for.
• A Scrum Master is responsible for making sure everybody sticks to what they committed and agreed to, including improving predictability.

Other Contributions to Predictability

There are other ways in which better predictability can be achieved, here is a couple of less understood ones:

Focusing on the uncertainty of a batch of Backlog Items in a Scrum Sprint instead of the single Items increases predictability because of simple statistical math. The uncertainty of a collection of individual and independent items follows the square root of the sums of squares of variance, which is less than linear. Intuitively it makes sense you win some you lose some, uncertainty evens out to an extent. This is a much more constructive experience for the Scrum Team.

Decomposing into smaller Backlog Items reduces uncertainty. If an item has an estimate of 5 and worst case 13, that will contribute to the variance with (13–5)² = 64. If it is decomposed into two items with estimates 2 and 3 and worst case 5 and 8, the variance is 34. So there is clearly a benefit to breaking it down into smaller Items as Scrum teaches us.

Announce intent. Let people around you know what you are about to do. That gives them the opportunity to suggest an even better course of action, adjust their own way forward or perhaps just have the psychological comfort of knowing a bit more about what is going on.

Waiting, not rushing to conclusions can help manage our error-prone intuition, estimating more than once is a good idea. Making predictions at the “latest responsible moment” to use a Toyota buzzword is great because it helps us counter some of the cognitive bias traps we tend to fall in. Don’t get into details before you have to, another Scrum lesson.

Conclusion

This is not intended to be an exhaustive treatment of the subject of predictability, but only a couple of appetizers and tools to become a bit better at it; using Scrum enhances predictability. However it is still something the Team has to choose to engage in, the tools bring very little value if they are just drilled exercises.

Therefore the Scrum Teams must understand the value and benefits of predictability both for themselves as active and committed contributors and to the customers they serve.

For further discussion feel free to contact me through my company or private mail or LinkedIn.