One of the controversies that people currently delight in getting upset about is the question of whether teams working the Agile and Lean way should estimate or not. The controversy is especially vibrant in the software community.
Without diving into the midst of this whirlpool that is a topic for another rainy day, I will try to take another angle on estimation, claiming that when done properly estimation is a great way for a team to build knowledge collaboratively and iteratively.
For the sake of the argument, we will assume that there are situations where the estimates of effort, value, risk and complexity matter. That there are situations where Product Owners have to choose between working on different things based on how much value is generated from a limited budget for example. Let us also assume that teams collectively have some idea of cost based on experience and that stakeholders are able to express views on experienced value of different things.
In those situations Product Owners have to prioritize based on these expectations – estimates. They do not have anything else; the future is not here yet. In those situations some sort of predictability is of the essence, and the more precise the better. It is therefore a worthwhile effort to learn how to do it.
Tools for Estimation
Traditional project management often operates with four tools to estimate costs in project management:
- Analogous Estimating
- Parametric Estimating
- Three Point Estimating
- Bottom up Estimating
These four estimation methods are said to represent a hierarchical structure where analogous estimating represents the least accurate method, and bottom-up estimating represents the most accurate method. Readers can explore those techniques at their leisure. For now it suffices to say that what we are about to present is in fact a way to combine all four in a fairly useful yet inexpensive approach.
Before starting for real, I would like to make the comment that the claim of bottom up estimating being the ultimate and the most accurate, rests on the assumption that the work estimated is in the ordered domain (complicated or obvious) where sufficient knowledge does exist. In many situations that is however not the case, and such an effort becomes a waste of time.
When the work that has to be done has a significance of complexity, in situations where there is only fragmented knowledge available, then estimation requires collaboration and trawling together for common understanding and building up knowledge.
The methods we are to use are consequently not just about getting a number, but are in at least equal measure about facilitating collaboration, triggering people to come up with information they have and use the collective experience.
When estimating a certain property of a proposed deliverable, the purpose is to develop the best possible common understanding among those who are involved and make decisions within the time and budget constraints, so that they can make wise decisions.
A couple of things to note: Firstly it has to be done fast; experience shows that spending more time after a certain initial investment tends to degrade the quality of estimates. Secondly with increasing complexity typically there is increasing uncertainty in numbers produced; it is wise to always work with these number as ranges.
We have recently introduced a new version of our agile cards with cards for different estimation scenarios. Let us step through the different methods.
Estimation Poker
Estimation Poker (EP) or Planning Poker® is a workgroup method popularized by Mike Cohn of Mountain Goat Software. You can read more about it here…, there is even an online version here… We have chosen to use a series of cards that is an extension of the sequence promoted by Mike Cohn. EP can be used by a group of people to estimate any numerical value; in Scrum it is used to estimate effort, cost and business value.
The Process
For the sake of simplicity we shall use the Scrum terminology here and assume the objective is estimation of effort in delivering a deliverable. Here is how it goes:
- Each team member has a deck of cards as shown above.
- During the Product Backlog Refinement, the Product Owner (PO) has brought a number of stories (Product Backlog Items, specifications) to the table. Ideally the Team has had the opportunity to familiarize themselves with the stories.
- The PO explains the stories, the Team ask questions, there is dialogue. Everybody tries to understand what this is about and avoids passing judgement about size as it would produce anchoring of other people’s judgement.
- When the discussion quiets down, the Team is asked to make their individual best estimate. Each Team member shows his card, they all do it at the same time, again to avoid anchoring. The estimate is based on comparison with some reference stories that the Team is familiar with. The Team may have references for pieces of work of size 2, 5 and 13 (small, medium and large). So what unit is this size measured in? It may be absolute time (ideal man days for example) or it may be “Story Points”, a relative measure, so the “5” is about 2½ times bigger than the “2”. For now we will assume the use of Story Points (SP), see the detailed discussion later.
- Based on the estimates shown, people now give their reasons for the estimates. It is common practice to let those with the extreme values start. So if the maximum is “8”, the person who had that estimate starts explaining. The one with the lowest estimate, say “2” follows, then others may chime in with their views.
- When this discussion quiets down, the cards are picked up again and another estimate is done. If the Team can agree on a number, then the game is over for this story, otherwise another round of discussion is taken. Eventually the Team can choose to go with the majority
- It may happen that the Team cannot reach a conclusion, then the story may have to be put aside for further research, analysis or experimentation before trying to estimate again.
Adding uncertainty
A very good extension to basic estimation poker is to add assessment of uncertainty.
- When asking people about giving the “best estimate”, it is normally understood as a “50% estimate” in statistical terms, that is the estimators regard it equally probable that the end result is smaller as the end result is larger.
- So a simple trick for adding uncertainty is to ask the Team right after their first estimate to think of “worst case”. It is commonly accepted that when people think about worst case, they normally don’t think of tsunamis, earthquakes and other outlier events, but often think of missing clarification, uncertain technology etc. It is normally accepted that asking for “worst case” results in around a 90% estimate, meaning with 90% certainty do we believe that it cannot be worse than this.
- So typically, the Team does the card game described above for the 90% number as well. If the result of the card game for the 50% estimate was that about half the Team members had “5” and the others had “8”, it is perfectly OK for the Team to say “Can we agree 5 for the 50% and 8 for the 90%?”. It is all about the Team expressing their honest opinion.
- In our experience, using this extension actually helps the Team reach a statement; they don’t have to agree on one number, but can express a range.
- It helps the PO to use uncertainty proactively in his prioritization. He may choose to take high risk items to the top of the backlog, to cut out uncertainty early in the project. He may decide to push such items further out in time, because there is an important deadline to meet, or because he expects that new knowledge may be harvested and reduce uncertainty later. Finally the discovery of high uncertainty on a particular Story may trigger important discussions between the Team, PO and stakeholders.
- When a set of stories have been estimated this way, the uncertainties for the set can be calculated using standard statistical techniques. The 50% estimate for the set is the sum of the individual estimates. To achieve the 90% estimate for a set of independent stories a 90% buffer is added to the summed 50% estimate and the buffer calculated as the square root of the sum of squares of individual differences between 50 and 90 percent estimate
E90 = E50 + SQRT( (A90-A50)^2 + (B90-B50)^2 … (Z90-Z50)^2 )
The different cards
The set of cards for the Estimation Poker is an extended set with cards for different purposes:
- 1, 2, 3, 5, 8, 13: Traditionally used for Stories that have a size appropriate to implement in a Sprint. The numbers are the Fibonacci numbers, and provide some rather large gaps in the number series to speed up the action.
- 0, ½: For smaller stories, that might benefit from being combined with slightly larger ones.
- 20, 40, 100: Really large Stories, often referred to as Epics. Such an estimate is an indication of a need to decompose such a story. It should also be taken as an indication of a very imprecise estimate. Human beings can apparently only estimate with reason within one order of magnitude.
- ?: For people who have no clue about what is being discussed. It should not be used just because someone thinks another person is likely to know more.
- Coffee Cup: Time out, a break, when someone can’t do it anymore.
The Concept of Story Points
People often struggle with the concept of story points or relative estimation against reference stories. SInce the real world need time and money, why do it any different then? Here some ideas
- By using a relative measure and not an absolute time, we accomplish to separate judgement of size from commitment, it gives somewhat more objective estimates.
- By focusing people’s attention on reference stories we are eliminating some negative consequences of cognitive biases relating to underestimating one’s own work.
- When estimating in story points, it makes sense to track the Team’s velocity, i.e. how many story points can the Team deliver per Sprint. That can be useful for forecasting in relatively stable environments, it is also useful for judging if changes the Team introduced to their way of working have positive effects.
- If the team’s velocity changes, the estimates in story points are still valid, and do not have to be redone.
- When estimating using story points, the Team has got to have some references to compare the stories being estimated with. There are several ways of getting those references. They could be jobs that the Team already have done together and know the size of, it could be some of the stories that the Team chooses to do to establish a baseline and it could be some piece of work that the Team has a common understanding of. As the Team delivers stories, they will think about and select good references for future use.
Extra recommendations
Some people prefer to use averages when a Team passes different estimates. The practice is discouraged because it adds an impression of precision that is typically not warranted in stable knowledge. It can therefore mislead stakeholders to trust too much in the estimates.
Some people allow Team members to use two cards for estimation, but it is not recommended. It defeats the purpose of showing stakeholders that these numbers are rough estimates; adding more detail usually obscures this fact, like the use of averages, mentioned above.
Other uses of Estimation Poker
Estimation Poker can also be used for other things. The most obvious use, is to use it for estimating Business Value, in that case of course it is different people that does it, the stakeholders or users, who have opinions about value. If estimating value, references of value are needed, and worst case estimates of course are smaller than best estimates.
Why this method is useful
This idea of estimating like this in a Team using a limited set of outcome possibilities has a lot of useful effects:
- Estimating in a Team immediately triggers a focus and sense of accountability, that make people take the effort more seriously.
- Estimating in a Team also tends to remove the worst misunderstandings and hence the really big estimation mistakes, because more people with more perspectives are involved.
- Estimating in a Team give some protection against the most common cognitive biases. Daniel Kahneman have explored many of these, and we can learn from him that involving several people in judgement can block some of the most negative consequences of the most prevalent cognitive biases.
- By having only a limited set of outcome options, the process is accelerated and there is less risk of spending too much time and degrading the result by over focusing on one particular aspect.
- The whole team is involved, this improves the likelihood of the Team taking psychological ownership of the result together, it is not just the expert or the one that normally does this kind of job that expresses his opinion
The whole method is a way involving everybody, to create the mental triggers for people’s knowledge to come to the surface. We cannot just say what we know, “we only know what we know, when we need to know” (Dave Snowden). It is of course useful to get a couple of numbers to wok with, but it is at least as useful to have such a work group method that enable people to reach a common understanding. Estimation seen as a knowledge building effort is NOT waste, it is also a way of working through the process of reaching a fairly robust common understanding.
This article will be followed up by another one, where we describe our new Extended Agile Cards, that contain Estimation Poker, T-Shirt Estimation, Cynefin Estimation, Kano Estimation and Confidence Estimation, look out this!
