Agile Lab Blog

This is just a short note about something that should be very exciting  for anybody who’s involved in project management (I know, I know, there’s something oxymoronic about that sentence).  I’ve just been reading a book called “Thinking Fast and Slow” by Daniel Kahneman.  It was recommended to me by my friend, writer and professor of history, Richard Toye.  Sadly, it took me about a year to get around to reading it.  And then it took me three or four weeks to get around to chapter 23, entitled “The Outside View.”  Seriously, if you have anything to do with project management, especially IT project management, you should be ordering this book right now instead of reading this blog post.  And of course, with the magic of kindle, you could be reading it within a few minutes.

Chapter 23, “The Outside View” outlines one of those concepts that is so forehead-slapingly obvious that people who don’t experience it every day as a problem will say “so what.”  This is the momentous thing that this article says:

The project that you’re working on will probably take about as long, cost about as much and experience the same kind of problems as similar projects.

That’s it.  Earth-shattering huh? No?  You’re not convinced?  Well this wouldn’t be earth-shattering if it weren’t for one thing: this is almost NEVER the approach which is taken to estimating projects, especially IT projects.  IT projects are almost all estimated in this way:

1. Write down all the things that you think you want to get done in this project

2. Estimate how long each of those things is going to take.

3. Add up all those estimates (possibly divide by the number of people in the project team) – this gives you a number, X.

4. X is how long this project will take (depending on how negative you’re feeling, you might add in a small, or a huge extra figure of “contingency).

This is what Daniel Kahneman calls “The Inside View.” There a few problems with this method, even though it is used all the time.

•  The estimates of how long things are going to take is usually an optimistic estimate, very close to the minimum time that a task might take.
•  You’ve forgotten some of the tasks, or many of the tasks, or nearly all of the tasks that are actually involved
•  These estimates do not include, what the great philosopher Donald Rumsfeld referred to as “unknown unknowns” (interestingly, the next chapter in Kahneman’s book is about why it’s a really dumb idea to start wars but people do it anyway).

When you think about these problems, you start to see the power of what Bent Flyvbjerg in this article (http://flyvbjerg.plan.aau.dk/Publications2006/Nobel-PMJ2006.pdf) calls “reference case estimation.”  This is what Kahneman calls “The Outside View.” Looking at other projects that are similar to yours: how long they’ve taken and how much they’ve cost, deals with all three of these difficulties.

•  Optimistic estimates, when looking at similar projects get replaced by how long things actually took.
•  Looking at other projects gives you an idea of how many tasks had been forgotten, how much extra work was discovered.
•  Although the “unknown unknowns” on another project may be different, you at least get an idea of the scale of the surprises that might affect your project (actually, from my experience, a lot the things that come as a “total surprise” that derail projects, tend to not to be that surprising – they’ve derailed a ton of projects before and everybody knows this!).

 One of the things that I find really interesting about “The Outside View” and “Reference Case Estimation” is that Agile/Scrum methodologies already kind of do a version of it.  I don’t know of an Agile method that specifically says that you should go and look at other similar projects and see how they fared as a strategy for estimation – now that Kahneman’s book is out there probably should be one, very soon. But within an Agile project the velocity for each iteration does give you an idea, very quickly, of how long a project is actually going to take.  You have an effort-estimated backlog.  You have the velocity from the previous sprint, or an average from a number of sprints.  Possibly, you have a graph showing how “discovered work” is getting added to the backlog as you progress with development. This gives you a very good idea of how long it’s going to finish the project and my guess would be it’s a lot longer than the “ideal figure” provided by an “Inside view”/traditional estimation.

OK, so once we get to development, Agile methods are doing something right.  Good.  But that shouldn’t stop anybody who’s involved with trying to estimate the size and cost of IT projects up front from becoming inordinately interested in the size and cost of other IT projects which are similar to theirs, within their organisation.  As a project manager, charged by stakeholders with coming up with an estimate, and probably being tempted by everybody around you to take the “inside view.” You should be doing everything that you can to “take the project outside.”

Don’t believe me, please read Kahneman’s book, all of it is good. But if you've got anything to do with project management, you definitely need to read Chapter 23 (his story about he got fooled by the inside view, and discovered the outside view while writing a school text book, is pretty compelling).