Category Archives: Agile

What agile mindset means to me

Imagine a perfect project in an utopistic world. For you, personally,  would it be more like:

A: We can can prepare for everything beforehand. For project success we create the perfect plan and then execute the plan.

..or perhaps more like:

B: We don’t need to know anything beforehand. Our system can adapt to any changes and surprises that comes along the way. For project success we set the goal and then just start doing.

It goes without saying that these are extreme examples.  We cannot know everything beforehand, and we cannot ever build a system that can adapt to everything.  You need both type A and type B kind of thinking in real projects. But still, are your management efforts more towards type A, or type B? Which one are you pursuing more?

In my perfect world, projects are more like type B. And this is what “being agile” means to me.

From schoolbook Scrum to Kanban without estimates

This is a story about our team’s journey from schoolbook Scrum to Kanban development flow. Everything written here has actually happened. One could say that this is our #NoEstimates approach, but I tend to shy away from that term. I like to think this is more like a story of continuous improvement that has been influenced by many discussions under the #NoEstimates hashtag. For those conversations I’m very grateful.

Background

  • This is an internal multi-year product development program. We are building a product that is installed to customer premises containing both hardware and software. The life cycle of the system will be decades.
  • Team funded for nn months. Unfortunately I don’t know how the team size and the length of the funding was decided. I haven’t seen any detailed work breakdown structures or bottom-up estimates, though.
  • Distributed development. Half of the team and the PO in another country, time difference 7 hours.

Kickoff week

We started the development with a co-located kickoff week. One of the days was “process day”. Managers left the room with message “you decide how you want to work”. This is what we decided after the initial confusion:

  • Scrum was chosen as process framework
  • Decided to go with one team approach instead of having two co-located teams
  • Two Scrum Masters, one in each location
  • Common daily using video conferencing
  • Two week sprints
  • One hour weekly grooming session
  • Retrospective after each sprint
  • Planning split into two 2h meetings due to time difference. Between plannings story sub-tasking in pairs.
  • Definition of Done defined

Sprint 0

After the kickoff week we had one week preliminary “sprint 0”. That week was used to set up version control, getting bean bag chairs, video conferencing TV etc. We also had three planning sessions. In one of those we chose a baseline story for estimation. The heuristic was “this feels like medium sized story”, and it was given 8 story points.

Early sprints

After sprint 0 we started working in the way we agreed in the kickoff week. Process improvements were discussed and decided in retrospectives. Here are some of the things we changed during the first sprints:

  • We noticed we didn’t have agreed way to add more work to sprint if we run out of work. –> We decided that we would have a mini-grooming session with minimum of 2 persons to add more work.
  • Due to time difference, other location would have to wait until afternoon to find out what other location did the previous day. –> Decided to use email dailies instead
  • Grooming sessions were painful. Decided to try sub-team groomings. Noticed that things would anyway need to be discussed with the whole team, so changed back to whole-team approach.
  • Planning 2 would only be held if needed. No need to review the sub-tasks with the whole team.

Our early estimating approach

We used planning poker to estimate our stories. Usually the estimation process went like this: lengthy discussion -> “are we ready to estimate?” -> playing poker using online tool. Here’s what happened after the poker:

  • 5-5-5-5-8-8-8-8 -> pick higher
  • 3-5-5-5-3-8-8-8 -> small discussion, pick higher or settle with average
  • 1-3-2-8-13-1-2 -> Whoa! More disucssion and then new game.

Moving away from planning poker

Soon we started to notice our stories were almost every time 3 or 5 points, sometimes 8. An idea came that “what if we just stop playing the poker and agree that it’s 5 points”. Or just count the number. That’s what we did, and amazingly enough that didn’t change anything. Instead of committing to 50 story points we just committed to 10 stories.

From Scrum to Kanban

Working with distributed team with longish time difference made the Scrum start-stop cadence quite large part of the 2 week sprint: demo on Thursday, retro on Friday, planning part 1 on Monday and part 2 on Tuesday. From 10 day sprint that’s actually 40% of the days!

What happened was that we unofficially already worked on items even though they weren’t “accepted” and “committed” to a sprint. We kind of anticipated what was coming next and already started working on it. So,  we decided to make it official and move to Kanban. Here’s what we did:

  • Dropped the phased planning meetings
  • Kept the weekly planning/grooming meeting. Main idea was that stories would be groomed to ready to dev in these meetings.
  • Kept recurring retrospectives
  • Setup the kanban board to Jira, columns were setup in the same way that we already worked in Scrum (ready to dev – in dev – in review – testing – ready to demo – done)
  • Write up kanban policies for the team and rules how tickets move in the flow

Not estimating stories anymore

First column in our kanban board is ready to dev. This means a ticket has been discussed with a team and any developer can grab the ticket and start working on it.

Our entry criteria for story to be accepted to ready to dev is that minimum two team member accepts it. Our heuristic is “can we make it simpler?”. If answer to that question is “yes”, then we try  to peel the story simpler. We also try to prefer vertical stories over horizontal tasks.

Currently, using this approach, our median cycle time is little over 6 days, and mean time 8 days. Cycle time for us is the time that starts when developer starts working on a ticket and stops when it’s ready to be demonstrated.

Changes along the way

Along the way we’ve fine-tuned the process bit by bit. Here are some of the improvements we’ve tried along the way:

  • When we switched from Scrum to  Kanban, first we tried triggering demonstrations on demand (i.e. when ready to demo column reaches certain amount of tickets).  Pretty soon we switched back to regular demo cadence, it seemed to fit stakeholders better.
  • Adding queue columns to the flow, like “waiting for testing”.
  • Adjusting our WIP limits in the flow
  • Adjusting where to test and when are branches merged to master
  • Decided not to talk about implementation details in demos

As I’m writing this I feel we’ve reached a certain plateau. We have a nice flow and even though we make small adjustments to it, we haven’t changed the way we work drastically in the past few months. In a way that worries me.

But wait, we still estimate… 

Our team’s high level goal is to get 1.0 out of the oven by the end of the year. Our PO has defined a desired set of features that he would like to have in the 1.0. We have fixed time and fixed cost, so the question has been “are we able to produce enough scope that would make a meaningful marketable 1.0?”

To answer that question, I’ve used our throughput data to produce a forecast. One problem has been — because we groom our stories just in time — we don’t really know how many dev-sized stories does the rest of the backlog contain. To address that, I have simply estimated how many tickets do each of the higher level features in backlog consist of.  I have phrased my answer like “based on our current speed, it seems likely that we will reach that scope by XX”.

I have also had an idea to gather throughput metrics for the higher level features, and then use that data to forecast. I haven’t done that (yet) though. The problem with that approach is that the sample size is really small.

Trying to tackle personal biases

This is a true story, but on the other hand it’s my true story. I’m fully aware that my brain is adjusting this story. I remember what I want to remember and I’m putting pieces together to make a more coherent story. I don’t do it consciously, it just happens in my brain. To compensate for that bias, I try to list some of the thing that haven’t gone like they do in books usually:

  • We’ve had a “8 story points” ticket that took 6 months to get done.
  • When forecasting future performance, I’ve ignored dependencies and technical impacts. I’ve just assumed future work is just a big mass of tickets waiting to be chewed.
  • Grooming sessions are sometimes very painful and confusing.
  • We don’t explicitly ask “can we make this simpler?” out loud that often anymore. But I guess we’ve grown into it; cycle times haven’t been growing.
  • We have defined WIP limits to each kanban column, but they’re more like soft limits.
  • We have quite a few horizontal technical tasks and the ideal of just implementing vertical slices is quite far away
  • Our standard deviation for ticket cycle times is quite high. I don’t know if it’s too high to make any kind of forecasts about the future performance.
  • I use kanban and Kanban interchangeably, don’t really remember what was the difference and too lazy to check it up. =P

—-

So that’s my story. What’s yours? Would you like to hear something more? I’m pretty sure I’m forgetting to mention something.

How to forecast project progress without estimating… or not?

Here is a summary of how to forecast project progress without estimating:

  • Do not estimate stories with story points. Instead, just count them. Use some kind of slicing heuristic when taking stories into development. Do not estimate them.
  • Constantly collect throughput data (# completed stories per time period).
  • You know how many stories you have in your backlog.
  • Using this information, you can steer / make decisions about your project without estimating. You can forecast using the team’s throughput and project it to the work remaining. You constantly “steer the ship” by using data, not estimates.

As far as I’ve understood it, that’s the approach Vasco Duarte is explaining here in Agile Riga Day 2012 keynote speech (starts around 30min mark). I like that approach a lot. That’s pretty much what I’m using in my current job. I think it’s a lot better approach than the good old story points + planning poker approach, where we try to estimate the duration of each task.

But.

burndown

Here comes the catch. “You know how many stories you have in your backlog”. Actually, you don’t.   You have two options if you want an approximate number:

  1. You do up front requirements gathering, and groom the stories to the backlog.
  2. You estimate how many dev sized stories does the remaining work consist of.

The approach #1 is, IMO, problematic because it really smells like big up front design. And there is lots of potential waste grooming stories that never actually get implemented. Aren’t we trying to avoid that? The approach #2 is, IMO, a lot better, but it kind of ruins the promise that we had, that we would forecast the project progress without estimating.

Still, I think that estimating the number of stories is easier than estimating duration, because then you don’t have to estimate wait times. Which often are really hard, maybe even impossible, to estimate.

There is actually a third option. It would be to start working and learning. But in any case, even in that approach, you somehow need to approximate how many stories you have left IF you want to do forecasting.

#NoEstimates confusion

So you’re always late. Your friends ask when are you coming, and you always give way too optimistic answer. Or something else happens. In any case —  you’re always late. Simply put, there are two possible ways to handle the issue:

  1. Find a solution: You try to be better in estimating your arrival. You take into consideration that traffic affects your travel time. You remember that putting your clothes on takes longer in winter. You add marginals to your “arrival estimates”. This way you are better estimating your time consumption, and as a consequence, you’re more often on time.
  2. Change the problem: You change the setting so that either you don’t have to give an “arrival estimate” or it doesn’t matter that much if you’re late. For example, instead of meeting outside of restaurant before going in, you agree to meet in a bar nearby. So if you’re late, your friends can just have one more aperitif, no big deal. Maybe your dinner party starts with a flexible cocktail party period instead of punctual time. So we change the problem from “How to be on time?” to “How to change things so that my friends don’t need to rely on estimates so much?”

The way I see it , currently #NoEstimates is about option #2 — changing the business so that you lessen the need to estimate.  If you stay within #1, the solution space, I agree that #NoEstimates makes no sense. If someone asks “how long is it going to take?”, you need to estimate to answer, there is simply no way around it!

The key questions here are “How do I change my way of doing business so that I don’t need to ask ‘how much? when?’ so often, or if I do, it is for shorter period of time? How do I deliver value sooner? How do I validate my concept ASAP?”

One small addition: in real life, #1 and #2 are not mutually exclusive. You can, and should, usually do both.

 

Up-front planning and Agile

This post was triggered by this conversation I recently had in Twitter with Henrik Ebbeskog. Reading that before continuing on this one may help you understand better.

So let’s assume a scenario. A client wants to set up a web shop(*), and would like to hear your rough estimate how much effort would it require. The client just wants a ball-park, so we’re not signing contracts here. Here’s what you should  take into account before giving your answer:

  • High availability requirements, how many users do you expect? Simultaneous users?
  • Different user personas
  • SLA requirements
  • DB backup requirements
  • User support requirements, do you perhaps need a live help line?
  • External interfaces, do you need to interface with a bank payment interface perhaps?
  • What markets are you targeting, maybe countries that write from right to left?
  • Etc etc

You probably can come up with more, and adjust these to your own domain. And the exact questions here are not the point — the point is that if you plan for these, and take these into account, you can base your estimate onto these. “It’s roughly 2-4 weeks IF just a single server, no user support, no redundancy in servers, we offer 9-17 support weekdays only, and it contains more or less these features”

And here comes the catch: this doesn’t mean you’re setting these in stone. This isn’t Big Design Up Front. This isn’t waterfall and forgetting about Agile values like responding to change. This is planning, being prepared, listing your assumptions explicitly and asking your customer questions she didn’t think of.

Agile doesn’t equate to #NoPlanning.

*) Sheez, always a web shop as an example. I need to expand my imagination…

How much will it cost and when will I get it?

Whether we like it or not, eventually someone will ask the dreaded questions: how much? when? Or the variant: what can I get with this money? This post is my view on the topic; how would I start answering those questions.

Disclaimers:

  • I don’t claim that this post covers all the things needed to manage a SW project. It’s probably just a scratch on that topic.
  • There are principles that are common to SW industry, but on the other hand there are so many different setups and domains that one perfectly good approach could be just the wrong one in different domain. So I’ll just say it once so that I don’t need to repeat it constantly: it all depends.  Use your own head.
  • I stole the title for this post from this great post by Neil Killick. Read that, too. And his other posts.

Our example case

A customer approaches you. He wants to build an application. It doesn’t matter what the application does, but let’s assume a non-trivial application like health care system for a municipality or order management system for a retailer.

Before you even begin

Before I would even begin to think about duration and price I would try to figure out the following:

  • Does this project have to be this big? Could we divide it into smaller projects, pick a most important one, and start with that?
  • How much money is the customer willing to spend?
  • Have we worked with this customer before?
  • Have we worked on this kind of software before? Has anyone worked on this kind of software before?
  • Is the team that is supposed to work on the project worked together before?
  • What is the risk involved? What happens if this project fails? What is the profit that we’re expecting to get from this project?

There are of course many possible outcomes when you answer these questions, but here I’m roughly classifying projects into these three categories:

  1. Big project, new customer, new kind of software for you and failure isn’t an option. Example: pension payment system that has a specific launch date. Banking software.
  2. New kind of software, expected gains are big, but you expect to fail often. Non-SW example: new drug development. Mobile game development. Composing a world class hit.
  3. Something between the extremes. You may have done similar project before or are familiar with the problem domain. If you fail, someone loses medium to large amount of money. Example: online ticket reservation system.

Which of these categories does your project fall into? Is it more like…

  1. In this case I’d strongly suggest to have a smaller “test project” to have some kind of the feasibility of the project. Otherwise I would be reluctant to even start. Or you might just want to politely reject that customer’s request. Not your piece of cake.
  2. In this case I would not spend too much time trying create a detailed work plan. Just set a budget and/or schedule and start doing. You don’t even know exactly what you are trying to achieve, so why would you try to break that work into smaller pieces? In this case you need a experienced team.
  3. Rest of the post assumes that your project falls into this category.

Create a solution blueprint / concept

So someone needs to know “how much?” I would address that need by gathering roughly the following information:

  • Goal / “mission statement” for the product
  • Business case for the product. Someone is always interested in $$.
  • High level capabilities/functionalities for the product. What does it do? This breaks down the general topic of your product into smaller pieces of functionality.
  • Main user roles
  • Main concepts of the product and their relationships as a concept map
  • Interfaces to external systems
  • Initial idea of the system architecture

Usually you do this with your own money. If you can do this with your customer’s money as a first phase of your project, good.

If needed and requested, I would use this information to create a top-down estimate about the project’s expenses. I would use any information I have available, for example data from similar past projects and expert opinions. It’s been a while since I last participated in top-down estimating, but there are plenty of resources out there, like this list and this gentleman.

Based on the estimate I would come up with some kind of minimum promise for the customer: this is what you get, minimum. If we fail to deliver, we deliver the rest on our own expense. I would probably come up with “this is what we are likely to deliver” and “this is what we deliver if things go really well” lists, too, if applicable.

During the project

In general, I would start with the most important capability and seek to deliver working software from day 1. If not day 1, then as soon as possible.

As you take the capabilities into development, break them down into smaller pieces. We call them epics, you might call them features, or something else. You might want to consider classifying these epics into S, M and L size lots. Do this as late as possible.

Epics still aren’t the raw food of the development team(s). Let the team split the work into user stories in co-operation with PO. Prefer “can we make this story smaller?” -heuristic over story point or other estimation techniques. If your team wants to split the story into tasks, fine, but don’t ever estimate those tasks separately. In any case, if your stories are small enough, you probably don’t need sub-tasks. Do the story splitting as late as possible, too.

Let the team decide a process / workflow on how they are going to deliver software that meets the constraints that your organization and your customer’s needs impose. Visualize that workflow, for example with kanban board. Gather metrics about the cycle times and queues in your system. Do you know how long does it take, in average, that your story has to wait until it gets to testing phase?

If it’s a new team that has never worked together, start with a school book process framework. Scrum is a good option. Make continual improvements to your process, see what works for your team. Henri Karhatsu has written about his team’s progress. It’s a nice read.

Measure epic throughput and use this data to fine-tune the original top-down estimate you did in the beginning. You might consider using story throughput instead, depending on the size of your project. Use this constantly updated information to answer the questions how much? when?

Conclusions / my view on #NoEstimates

I claim that during development, you don’t need to explicitly estimate development efforts. You don’t need to estimate stories using story points and estimate how many hours does it take to complete a task. I don’t see a need for bottom up estimates.

You can, and maybe even should, however, use the actual development velocity metrics to forecast the future. You know how many stories does your epics contain in average, and you know how much is your story throughput. Use this data to predict the future. Some call this estimating, and I’m fine with it.

Some say “you cannot predict the future”. Sure we can, we do it all the time as human beings. We cannot be 100% sure, ever, but we can make the percentage higher.

One final note on #NoEstimates. One light bulb moment for me was that #NoEstimates doesn’t mean #NeverEstimate. I would be comfortable if someone called the above described approach as #NoEstimates approach even though it possibly contains macro level estimating.