How to forecast project progress without estimating… or not?

Here is a summary of how to forecast project progress without estimating:

  • Do not estimate stories with story points. Instead, just count them. Use some kind of slicing heuristic when taking stories into development. Do not estimate them.
  • Constantly collect throughput data (# completed stories per time period).
  • You know how many stories you have in your backlog.
  • Using this information, you can steer / make decisions about your project without estimating. You can forecast using the team’s throughput and project it to the work remaining. You constantly “steer the ship” by using data, not estimates.

As far as I’ve understood it, that’s the approach Vasco Duarte is explaining here in Agile Riga Day 2012 keynote speech (starts around 30min mark). I like that approach a lot. That’s pretty much what I’m using in my current job. I think it’s a lot better approach than the good old story points + planning poker approach, where we try to estimate the duration of each task.

But.

burndown

Here comes the catch. “You know how many stories you have in your backlog”. Actually, you don’t.   You have two options if you want an approximate number:

  1. You do up front requirements gathering, and groom the stories to the backlog.
  2. You estimate how many dev sized stories does the remaining work consist of.

The approach #1 is, IMO, problematic because it really smells like big up front design. And there is lots of potential waste grooming stories that never actually get implemented. Aren’t we trying to avoid that? The approach #2 is, IMO, a lot better, but it kind of ruins the promise that we had, that we would forecast the project progress without estimating.

Still, I think that estimating the number of stories is easier than estimating duration, because then you don’t have to estimate wait times. Which often are really hard, maybe even impossible, to estimate.

There is actually a third option. It would be to start working and learning. But in any case, even in that approach, you somehow need to approximate how many stories you have left IF you want to do forecasting.

6 thoughts on “How to forecast project progress without estimating… or not?

  1. I think you explained quite well a valid approach that I have presented in that video you mention.
    However, the question you pose is easy to solve.

    First the assumptions:
    We need to know the “number of stories in the backlog” only if
    – you have to make an irreversible “go/no go” decision
    – you need to know the cost of the whole set of functionality and are not willing to change that functionality as you know more

    I propose that we can work in a way that invalidates the assumptions above. Specifically if we design the work to be incrementally valuable (each increment can potentially be shipped), we don’t need to k of how mich work is in the backlog, rather we can focus on defining the investment we are willing to make for an expected value for that project.

    If we can work in small increments that have value on their own, then we don’t need to know all the work that may be useful in the future. Rather we can project based on what we already know is needed and adjust the projection when we learn that a new feature is needed.

    As the Backlog size changes, so does the release date – unless we can remove some work from the Backlog.

    1. Yeah, that just made me wonder, because in that demonstration that you gave, one of the key takeaways seemed to be that you could do a go/no go decision based on the forecast.

      And also, what is the purpose of forecasting, if we would like to avoid prematurely groom stories into backlog?

  2. ■ How did those “needed” throughput of stories get on the graph?
    ■ Who put together the planned reduction in their number?
    ■ Did all the backlog get created on day one?
    ■ Was the backlog created as the project went along? How did the number of stories get discovered as the project went along, and how did they get sized?
    ■ Are they are the same size, and does the team work at a constant rate, resulting in a straight line reduction of the backlog?
    ■ Was the “burndown” rate constant for each story?
    ■ Was there any variance in this rate, and was that variance used to project when the project would be ending?

    To answer these and many more questions about the project’s variables – all of which are actually random variables – what do we have to do?

    WE HAVE TO ESTIMATE THEM FOR THEIR FUTURE BEHAVIOUR

    And when Vasco says he can make a “go/no go” decision based on a forecast, he’s “estimating” the value of a random variable.

    Ignoring for the moment the violations of the principles of Microeconomics for decision making, the random number principles of all project variables, the untested assumption that all stories can be same-sized for a non-trivial project, and the productivity of the work for those stories is always the same.

    We have yet to hear how we can made a decision about a future outcome – date, cost, probability of all the feature showing up when needed – without making an estimate of those values. Unless of course you just pull it out of your ass.

  3. Hi Glen. Most of the Qs that you pose are kind of why I wrote the post, and my point too. But I can offer some answers here.

    ■ How did those “needed” throughput of stories get on the graph?

    In the simplest case it’s just simple linear function from now to our DL. If we have 160 stories “now”, and need to be done in 8 months, we must produce on average 20 stories per month.

    ■ Who put together the planned reduction in their number?

    It’s just a linear function here.

    ■ Did all the backlog get created on day one?

    That’s one of the things I’m questioning and thinking that shouldn’t be done.

    ■ Was the backlog created as the project went along? How did the number of stories get discovered as the project went along, and how did they get sized?

    In my approach, stories get “sized” JIT, just before they get into development.

    ■ Are they are the same size, and does the team work at a constant rate, resulting in a straight line reduction of the backlog?

    They don’t need to be same size. They need to be in same size range so that meaningful statistical forecast can be made. As a stats guy you probably get this better than I do.

    And yes, in this we assume team would be working at a constant rate. Questionable assumption, yes.

    ■ Was the “burndown” rate constant for each story?

    Even if it weren’t, outliers don’t matter if # of stories is large enough.

    ■ Was there any variance in this rate, and was that variance used to project when the project would be ending?

    We could do for example Monte Carlo simulation to project the end date using our mean cycle times and standard deviation.

    One assumption here is that we don’t here foresee any technical big surprises in the future. That, too, is quite an assumption.

  4. This is pretty much the same conclusion we have arrived in my project last year. As you make stories smaller (I would prefer about one person work day) a team of 3-6 developers completes multiple stories every day, every week. This creates enough data for reasonable forecast and hides the random fluctuations. This is surprisingly accurate way of forecasting the future 1-4 weeks ahead. Still I prefer Kanban flow over Scrum and strict “committing” to a scope.

    What still remains is the question of estimating the longer term development. Customer (internal or external) has the money and needs to know how much money and time will be needed for some desired functionality. They usually are not interested in the next week but rather months or years.

    I have no clear solution, but some ideas. Like mr. Sulanto I’m convinced that big upfront planning is not the way to do it. There are a number of tricks one can use, like timeboxing and doing research spikes to reduce uncertainty. Or estimating the number of stories. But in the end I’m still left with relying on the experience on development work in general and experience about the context. No magic tricks.

    In the beginning there might be zero experience on the context (domain, customer, external stakeholders) but as the project advances the team gains this experience. In the beginning there is usually some idea of what set of functionality forms a MVP. You can go ahead and aim at the MVP without thinking too much about the estimates. By definition the MVP is a valuable increment. Later you should be able to predict cost and time based on experience from that project. It is important to understand that even the exact same team will have very different velocity depending on the project conditions and making a guess based on their previous project is just a guess. In the beginning you only have the previous project. And usually the team is not exactly the same.

    1. Thank you for your reply. Your thoughts are very similar that I’ve had.

      One thing I’ve kind of “realized” is that an approach without estimates absolutely requires that you deliver value often. And actually the Q “how much? when?” becomes less important and the Qs are “is this enough? what next?”. That, of course, requires a leap of faith from customers. A huge leap sometimes.

      Regardless, even if you estimate or not, it’s valuable to gather actual development metrics. If not for forecasting but also to notice long queue times etc.

Leave a Reply

Your email address will not be published. Required fields are marked *