Agile Lab Blog

I’ve been thinking about this for a while. Imagine there was this tool that everybody said was fantastic – let’s say it was a wood chisel, let’s call it the Wonder Chisel. And all of the people from the Wonder company, who make the Wonder chisel claim that if you use the Wonder Chisel, your carvings will be totally awesome. And indeed, many skilled craftsmen who already understand how to use tools and know a lot about wood carving, achieve great things with the wonder chisel. But there’s a problem. Every amateur, not skilled in the art of wood carving, and I mean absolutely every one of them when they first pick it up, stabs themselves in the eye with this chisel. That would be a problem wouldn’t it? You would probably say that the wonder chisel has some design flaws. You would probably want to take it off the market until you’d fixed them. You might even want to brainstorm some other ideas and then try them out with amateurs before you release a literally blinding product like this on the market again.

This is how I feel about the Agile concept velocity. It is a massively powerful concept because in a very short space of time it can give you an answer to the question that people seem (we’ll come back to that later) the most desperate to answer in any project – when am I going to get my stuff? The concept is really simple. How do you manage a project?

1. Make a list of all the things that you’re going to do on that project – this is called your product backlog

2. Estimate how complicated those things are – give each of the items on the backlog a number indicating it’s complexity – if you’re going to get fancy, use the Fibonacci series. How big is your backlog at the end of this? Let’s say you add up all the complexity points on the different items and it comes to 500.

3. With your team, plan to do a small number of these in a short space of time – let’s say, two weeks. We call this an iteration. Each of these things has a complexity score – so, let’s say you plan 50 complexity points.

4. At the end of the two weeks, count up the total complexity points of all the things that you’ve completed: THIS IS YOUR VELOCITY. And, it’s normally less than you expect it to be, for this example, let’s say it’s 30 points.

5. Here comes the clever part. Take this number, your velocity, 30 and divide it by the number of points remaining in your backlog.

 

= iterations = 30 weeks

And Ta Da! You have your answer to supposedly the most important question in project management. You’re very pleased with this and you show it to senior stakeholders, customers, programme managers. Unfortunately this is the bloody, stabbing in the eye blinding part.

Your senior stakeholders, customers and programme managers say one of two things:

Eye stab 1. Oh, the velocity is 30 is it? Well then next time I want it to be 60.

Eye stab 2. Really? The velocity of this other team over here is only 15, I think we should fire them.

Actually, there’s also a “slasher movie version” of statement one, where both eyes get stabbed and blood spurts everywhere.

Eye stab 3. “It can’t possibly take 30 weeks, I have a deadline in 10, I need you and the team to work 19x7 (yes, that’s 19 hours, 7 days a week and yes, this really happens) and “re-plan” so you can do 90 points per iteration.

Why are these things so bad? Well, actually the eye-blinding Wonder Chisel is perhaps a better analogy than it might appear. All of these ways of using the velocity tool blind the project. Let’s take them one by one.

ES1. If the team is “set the challenge” of doubling its velocity, there are really only two options:

·         Do a really bad job on twice as much stuff

·         Inflate the complexity scores on the stuff they do so that it looks like they’re doing more.

Actually, there is a 3^rd more “healthy” option.

·         Pay no attention and carry on as before.

But the first two really are blinding. If the team doubles its output, but lowers quality, then there is going to be a disaster in the future, and it’s been totally hidden from view. If the team starts inflating complexity figures, it means that you literally don’t know where you are with progress in the project.

Something similar happens with “ES2,” word gets out that a team’s being evaluated on the size of the velocity! And lo and behold everybody’s velocity is stupendously high and everybody is blind to how the teams are actually performing, how the projects are doing and when they will actually finish.

Actually ES3 is even worse. ES3 is like turning the lights out and squeezing the trigger on a machine gun.

What’s the answer! Sadly I don’t have one. This one of those situations where I’ve shown you that I know a lot about the problem – but I really don’t, yet, know much about the solution. There is at least one conclusion to draw from this:

If a tool is this dangerous for amateurs and they always do the wrong thing with it, you probably shouldn’t let them anywhere near it. Velocity is one of those tools.

I do have some suggestions about directions that it might be worth exploring. Coming back to a book that I’ve been talking about and thinking about a lot I get two ideas from Daniel Kahneman’s book “Thinking Fast a Slow”. One is the over-arching rule that Kahneman calls WYSIATI – What You See Is All There Is. Basically this means that people reason from what they can see and imagine – not from what they can infer logically and certainly not from what might be statistically most likely. From the disastrous performance of velocity in field trials I think we have to conclude that velocity makes visible and imaginable the wrong things.

Related to this idea is a finding that comes out repeat-ably in Kahneman’s book – whether people do the rational thing or the dumb thing in response to a given proposition – e.g. a set of bets with certain odds that pay off certain amounts – depends hugely on how the proposition is presented.

Finally, I’d like to suggest some ideas from a field that I used to be involved in – usability testing. One method used for interface evaluation was something called “cognitive walkthrough”. Idea of cognitive walkthrough is that a team of experts can evaluate an interface quickly by asking a series of questions about the interface at each stage (I’m paraphrase slightly because the original questions are a bit strangely worded).

1. Will the user try to achieve what the task they need to do at this stage of the process?

2. Will the user notice that the correct action is available?

3. Will the user understand that the task they want to achieve can be achieved by the correct action?

4. Does the user get feedback on whether the action is completed?

What I think is fascinating about this is that “velocity” as an interface fails at the first step. Velocity is a useless interface, or certainly the way we report it is no help because it will not make the user – i.e. senior manager, stakeholder, customer, programme manager want do the “right” thing. And what’s maybe more interesting is whether the only way to get the user to want the right thing as an answer to question 1, is to present the right “correct” options to explore as answers to question 2. Food for thought, and maybe another blog post.