All posts by Matthias Jouan

About Matthias Jouan

project manager and software developer

Ravings on #NoEstimates

In this probably too long post I will try to expose my thoughts on #NoEstimates as I see it. I will also try to give some pieces of answer to the most recurring questions and critics exposed by #NoEstimates opposition.

Twitter

#NoEstimates is by essence a twitter thing… and this is a plague, really. I’ve been reading many, and participating in some, #NoEstimates twitter conversations, and what really hit me is how much it is difficult to have a built conversation within 140 characters. It especially is with #NoEstimates because of an obvious vocabulary issue in the very definition of what an estimate is, and how much it differs from a forecast for example. I am convinced that many arguments about estimates would not have occurred if people would have first worked on a kind of “mutual understanding”.

So, for the record, and in order to make future conversations with me easier, I consider estimating as very different from forecasting in the sense that forecasting uses data to predict the future, usually with statistics and probabilities, while estimation is more based on experience and is more “vague”… but that’s only my definition and you may very well disagree. We just need to know what we are talking about before we start fighting.

Not doing estimates existed way before #NoEstimates

The first story I heard (read) of was the one of Dragos Dumitriu, as described by David J. Anderson in “Kanban : Successful Evolutionary Change for Your Technology Business”. This story dates back to 2004. There are of course many other “old” stories like this one, including from people in the anti-#NoEstimates side (I’m thinking of Trevor K. Nelson here).

What is important is that although the hashtag is rather new, the question is pretty old. This also means that Woody Zuill’s point of view (he initiated #NoEstimates as a hashtag for those who don’t know) has nothing to do with a dogma or anything of this kind. I find it’s a shame to read sometimes things like “zealotry” or “evangelical” when talking about #NoEstimates. People who wants to talk about not/less using estimates in software development can use this hashtag. Period. Woody Zuill has his own view, others have other views, no “absolute” here. #NoEstimates is neither a movement nor a religion.

Common sense

I am paraphrasing Peter Kretzman here, but I genuinely think that trying to get rid of estimates is pure common sense and I will try to explain this as much as possible.

muda

The first thing that comes to mind is that estimates do not produce concrete value. That makes them a waste in the sense of Lean. It seems logical, if you follow this paradigm, to try to get rid of them when possible.

only a tool

“[Estimates are] a tool, not an outcome” (Peter Kretzman, The case against #NoEstimates, part 3: NoEstimates arguments and their weaknesses). I agree with that. One of the main critics anti- have against #NoEstimates is “Don’t blame the tool because you don’t know how to use it”, and this argument is a bit flawed in my opinion : let’s abstract away “estimates” one moment and talk about a random tool instead. So let’s say we have a tool. This tool have proved useful many times but it appears that many people fail to use it correctly. This tool is intrinsically dangerous when not correctly used. Often it is so much misused that it can jeopardize the whole project, or the whole company. Facing with this situation in his company one can either

  • Make the tool easier to use, which means change the tool, so that people can use it correctly
  • Train people in the tool, so that they eventually manage to use it correctly. The “don’t blame the tool” critic live here.
  • Find another tool that would do the same, or better, and would be easier to use
  • See if we could completely get rid of the tool, maybe by changing the very problem the tool is designed to tackle

#NoEstimates is about the last two points, or a combination of both :

  • some of the #NoEstimates advocates seek to find ways to remove estimates as much as possible and replace them when needed by forecasting based on actual data and statistics. A very common practice in the Lean/Kanban world.
  • others – or the same actually – believe that by changing the business relationship we could completely remove the need of providing any kind of future prediction. Hence no estimates at all. Some relationships are more difficult to change than others though, and of course internal software development with fixed time and budget (“own money”) would be easier to do with no estimates than a fixed- scoped public-procurement based projects…

obvious in certain situations

There are domains where estimations are often not used at all, in nowadays’ software business. Unsorted : very urgent operations, fixed-budget fixed-time research and development, fixed-time options testing (UI design testing with prototypes for examples). You can figure some more situations I am sure.

We all agree !

Maybe the main thought I have about #NoEstimates, and it will be my conclusion, is that most of the #NoEstimates folk agree with most anti-#NoEstimates people. Unfortunately some barriers exist and it is our duty to try and break them so that the discussion go a step further. That is the goal of my Open letter to anti-#NoEstimates people.

 

Open letter to anti-#NoEstimates people

Dear Peter, Glen, Trevor…and friends,

As some of you might already know, I’ve been hanging around in the #NoEstimates hashtag for quite some time. Even before that, before I even noticed that there was a hashtag, I and my team had been practicing a less-no-better-fewer-leaner-estimates kind of software development.

To sum it up, my story is basically that, in a context of co-called “Agile” software development, I was not comfortable with how estimates were done (the whole poker planning thing). Instead of trying to improve our estimation process, to make it more accurate and reliable, which is a path I respect, I, in a Lean-minded attempt, tried to remove estimates as much as possible and use historical data instead. I know that there is a recurring discussion out there about the very definition of what an “estimate” is and the difference there is between “estimate” and “forecast”. Honestly, although this discussion drives – consumes – a big part of the whole #NoEstimates thing, I don’t care much.

Ever since I began to read things about #NoEstimates, on twitter or elsewhere, I keep asking myself the same question. I might already have tried here and there to talk about it. Maybe I sometimes was a bit too vehement (euphemism), which may or may not explain why I still haven’t found The Answer. That is why I’m writing this open letter.

So my question to you is very simple :

Why do you even care about #NoEstimates ?

#NoEstimates is a hashtag where people who want to try to do software development business without estimates gather and talk together. To be precise I should write “do their software development business”. What I am meaning here is “I am a software editor, and I run the company without estimates because I do blah blah” : good for you! On the other hand “Estimates are a very useful tool for me and I can’t possibly imagine why I should not use them in the context of this US DoD public procurement I’m interested in.” : good for you too! That’s OK if you use estimates successfully. I once asked if anybody had ever heard of a no-estimate-like story in the context of public tendering. The answer was “no“. That’s not a big deal either.

So why do you all fight #NoEstimates like it was some kind of a threat ? Does anyone of you think that he is going lose his job if #NoEstimates proves usable in some domain ? Would it somehow jeopardize your business ? Please ! Please tell me why this permanent, impassioned rant about #NoEstimates !

P.S. : I know, because I’ve seen it already, that some of you will accuse me to try to avoid having a discussion : not at all! On the contrary I am really, genuinely eager to understand the very Why of all this.

P.P.S. : This open letter can be read as a follow-up to #NoOthers and #NoEstimates

Definitions are harmful

Yesterday I attended an amazing talk by Stéphanie Walter at DevFest Nantes. In this talk she presented a series of very “lean” tips and tricks about web design and how to organize the whole process from wireframing to development “the hard way”.  I don’t usually write about webdesign…and this is not going to happen today neither, but something in her talk caught my attention.

At a moment near the end of the talk, Stéphanie described an iterative, incremental process where quick feedback loops allow quick reactions, avoid big re-works, induce a better collaboration and a better communication between everybody and allow creating better design, better user experience, … in opposition to the usual process implemented at her company. After the talk, someone asked “Would Agile help you?” and she answered something like “We tried once but it failed. We designed and developed each part of the site one after the other, but the developer missed some opportunities to make more common CSS and had to rework many things after the fact, etc.” I must say I was very surprised because

  1. what Stéphanie described sounded totally Agile to me. She talked about incrementally creating the design, about very Lean-like ways to avoid wasting time, about better collaboration, etc. So the question was very bizarre to me because she was definitely there already.
  2. She answered no!

Actually this is not the first time I hear or read something like “Agile won’t work for us : what we do is [some kind of definition of Agile]” and I really don’t know what to do about it. I’m not even sure that we should do anything about, apart from being aware of the phenomenon. This is just a funny fact : applying our “Agile” definition failed, let’s drop it completely and go back to our Agile practice.

This whole thing reminds me of a question a colleague asked me some time ago. The team I work in used to practice Scrum, but we improved a couple of things, making us get away from theoretical Scrum and move to a more “kanbanish” process. Since I am supposed to be the Kanban guy, my teammate asked me if we were “doing Kanban now“? I said “Why do you care? Do you really need a new religion?!” Maybe that was a bit a condescending answer, but my point was that we did not need a definition of what our way of working had become. I really wanted to avoid inhibiting improvement initiatives by saying “we are doing method FooBar now”

I think that as long as people feel free to make improvements, as long as they think they are inventing something, then they will keep on improving. But if they happen to reach a point where they recognize something that has already been described, something that has a definition, a concept, they will stick to it and stop inventing. People tend to stop “thinking”, they tend to stop trying things when they “hit” a known, comfortable word.

That is why definitions are harmful : they are very static, very…defined. On the contrary improvement, changing things, is not defined, it’s moving.

 

#NoOthers interactions

The classical interaction between people supposed to work together on a product (marketing, product management, software development management, developers, designers, etc.) is driven by requirements. The usual (implicit) process is then

  1. I identify a problem I need to solve for doing my job (better)
  2. I identify things I would need to solve this problem
  3. I identify people around me who could provide me the things I need
  4. I require them – namely the others –  to give me those things. Depending on my hierarchical position this part could be more or less easy…
  5. I collect others’interpretation of what I required
  6. I try to solve my problem with those things, and blame the others for not giving me what I required

Typical things I might require here are precise specifications, prioritized user stories, overall goal, estimates, cost, deadline, etc.

The main concern with this approach is inherent in the concept of others :

  • although I might think I know what I need, my understanding is limited to the things I know, to my own domain, and thus I might not be aware of what others know.
  • others are not very likely to understand what exactly I need, leading to a very incomplete solution.

My suggestion is to turn this interaction process upside-down :

  1. I identify a problem I need to solve for doing my job (better)
  2. I publicly expose my problem to the team : no others here
  3. Some of my teammates propose me things they can give me that would be relevant to solve my problem, according to their own knowledge and their own domain
  4. I collect what my teammates gave me
  5. I try to use those things to solve my problem, and improve collectively to a solution.

Some benefits of using this approach :

  • My teammates understand my job better
  • Non-obvious solutions might emerge
  • Nobody feels like they are working for nothing, or losing their time

A very naive example now, to make things clearer : My job consists in hammering nails. For each kind of claw there is a kind of hammer that I get from my blacksmith. One day I am facing big difficulties with a new kind of claw.

  • Classical interaction : I go to my blacksmith’s and ask him to make a bigger hammer.
  • No-others interaction : I go to my blacksmith’s and show him the new kind of claw. He suggests me to use a screwdriver!

Of course this is all very theoretical, so any comments would be appreciated.

#NoOthers and #NoEstimates

Yesterday I promised to give some concrete examples related to what I call #NoOthers. The first one will be #NoEstimates.

What #NoEstimates should/could be

Example #1

Product Manager : “We have plenty of things to do on the product. There are many new features to add and there are also a couple of really embarrassing defects that we should address too. I need to know how long each of those things is going to take so that I can balance defects with new features in the schedule.”
Dev#1 : “Well…It ain’t easy to estimate bug fixing time before we get into it. As far as I can tell right now no bug solving ever took more than 3 days, but we can do some stats if you really need to make accurate forecasting.”
Dev#2 : “The thing is that we are not confident at all with the estimates we give you. For the smaller things we can have some interesting enough statistics based on what we already did in the past…but for user stories like foo or bar the scope is way too big.”
Product Manager : “I understand that you can’t give me accurate estimates…but I really need to know  when we are going to start working on baz. At least sales people need to know.”
Dev#2 : “We can try to split foo and bar into smaller user stories so that they can fit into the stats we have, but maybe we could just consider working on baz right now if it’s that important?”

Example #2

Customer : “I need to know how much it will cost me to have this product, and when it will be done.”
Project Manager : “Of course. But allow me to suggest that maybe your real need is not the one you think.”
Customer : “What do you mean?”
Project Manager : “Well, as far as I understand the product you want to build, you definitely need to get to the market as soon as possible.”
Customer :  “Who needs not?”
Project Manager : “Yes. So let’s first try to focus on a core set of features that will allow you to get to the market quickly. We really want to extract the features that compose the very minimum product here. Then we could improve the product by adding the other features as described in the specs…or other features if needed. Does it make sense to you? ”
Customer : “It totally makes sense…but I still don’t see how much it will cost us in the end.”
Project Manager : “It will cost you the amount of money you want to put into it! Of course for the minimum product we will provide you some kind of cost estimate after breaking the selected features into small pieces of work, but for the rest you will be able to stop when you want to.”

What #NoEstimates sometimes is / How it is perceived

Product Manager : “We have plenty of things to do on the product. There are many new features to add and there are also a couple of really embarrassing defects that we should address too. I need to know how long each of those things is going to take so that I can balance defects with new features in the schedule.”
Dev#1 : (thinking) He wants estimates, let’s give him estimates! Maybe he wants some random numbers to fill in a lottery ticket after all “No more than two days for each of these bugs IMO. foo is pretty big : I think of 7 or 8 days of work. bar looks easier so I think it won’t take long.”
Dev#2 : “Not sure about bar : nobody has touched this part of the product  for a while. I don’t think we can estimate that upfront… Hmm… What is the real purpose of this story again?”
Product Manager(thinking) Here we go again! NoEstimate bullshit! She doesn’t want to commit herself with an estimate and will argue about the purpose of every single piece of user story “Come on! You guys are incredible…”

No Others

I must admit those (almost) fictional pieces of dialog are a bit cliché, but they outline a key difference between what #NoEstimates is and how it is often perceived :

  • #NoEstimates is a hashtag that gathers people who want to try and find an alternative to using estimates as a tool. It really is a #NoOthers concept in the sense that we need to understand everybody’s needs to find the best suited tool / way of working together (and we think that estimates might not be that tool).
  • #NoEstimates is perceived as a rebellious community of developers – others – who do not want, or who simply can’t, understand what business people – us – need.
  • #NoEstimates can be used as a banner to wave in front of product/project/whatever managers – others – who do not, or simply can’t,  understand what dev people – us - need.

 

More will come on this subject…

No Others

L’enfer, c’est les autres (Hell is other people)
Huis clos, Jean-Paul Sartre

Silos. Much have been written about them for years, DevOps has made its way, and I’m sure you are familiar with the whole break-down-the-silos thing as we have been hearing about during the last decade.

Yet silos still exist. Why?

Because there are others.

Others. They don’t know us. They don’t know our job. They don’t know what we need to do our job well. They don’t quite seem to even bother trying to understand us. They are selfish and self-centered. They sometimes want to force us do our job in a certain way that is unnatural to us. They don’t want to help us. They basically don’t understand us at all…I’m sure you are thinking of some particular others right now.

The very notion of others implies a conflict of some sort between them and us. But ask yourself : Are others the cause of the conflict or does the concept of others arise as a consequence of this conflict ?

My opinion is that there should be no others.

Others only exist because we choose to solve problems by accusing each other instead of working together…but this attitude does not solve any problem in the end of the day, and only breaks the team into silos :

  • You are a developer and feel frustrated because you don’t really understand the problems your code is supposed to solve : don’t blame the product owner for not understanding your job. Instead try to work together as a team a figure out a better way to communicate
  • You are a product owner and feel frustrated because you have to deal with new features that have been sold to clients but do not fit well into the product : don’t blame the sales people for not understanding the product. Instead try to work together as a team a figure out a better way to work together
  • You are an IT security engineer and feel frustrated because…
  • You are a project manager and feel frustrated because…

The message I am trying to convey here is basically we are others’ others. When I complain about others not letting me do my job well, or not understanding me, do I even try to understand them? Do I even bother asking what they need to do their job well? Now think about the people you designated as others a couple of paragraphs above…

This little rant is only a beginning. More will come with more concrete examples in the next couple of posts on the subject, but as always feel free to leave a comment below or share your opinions on this subject on twitter.

Losing product vision

Developers, and I am no exception, usually complain that product management people lose product vision. They are often tempted to fill the product with a bunch of “useless” features which dilute the real purpose of the product, the main job of it. And since one of the most unpleasant thing of a developer is not understanding the reason why a feature should be added, this often causes endless arguments and create a real barrier between us and them.

Now let me tell you a story that happened yesterday.

A teammate and I were were talking about making coffee. As both him and I own a single serving coffee maker at home, we always experience huge difficulties when we have to make coffee with the regular coffeemaker that we share at work. When I first had to use the machine three weeks ago, I had not touched a regular coffeemaker in the past four years. I didn’t know the proportions of coffee and water to put in the machine and the result was…spectacular.

During our conversation we came to think about a product. We had a clear problem to solve : measuring the volume of coffee to put in the coffeemaker. Instead of using a traditional measuring spoon, which would only partially solve our problem (we would still have to know how many spoonfuls to use), we quickly designed a kind of 10-cups measuring bowl :

10-cups measuring bowl

We had a product vision, a problem to solve, a job-to-be-done, or whatever you call it…and then the conversation continued. Within about 2 minutes our simple, easy-to-use, problem-solving, user-focused product was looking like this :

coffeemeasuringrobot

The new product was much more complete and full of really cool features : voice command interface, measures both water and coffee, can deal with more of less strong coffee, can make coffee for any number of cups, changes used coffee filters, washes the coffee pot, etc. But what was the problem we were trying to solve yet? Clearly we had lost our product vision.

Quickly designing a product is easy. Making it evolve in a consistent and coherent way is much harder, and it’s everyone’s job in the team to try and stay focused on the main problems the product is meant to solve. Because losing this focus is also very easy, we must work together on the product, not against each other like we often do.

Indian Planning Poker

When I first heard of estimating work items – let’s call them stories – with Story Points I was like “WOW!! That’s amazing!! No more question about how long it will take! Just estimating the relative effort, it really is a good idea!

Then I discovered Planning Poker, and again was amazed by how it can solve Asshole Driven Development issues (me being the asshole most of the time). I immediately decided to apply it with the team I was in charge of. We did ONE session, and everybody agreed that we were wasting our time. Yet we learnt a couple of things then:

  • We could easily treat small stories, the ones worth 1 or 2 points
  • We could not easily decide about big stories, and almost always decided to break them into smaller stories

All around the Web you’ll see that people recommend not to use the bigger cards of your planning poker deck, and to only keep the 1/2, 1, 2, 3 ones or so.

So we decided to give a try to T-shirt-size estimates, which was basically like keeping only 3 cards in our deck : S, M and L. We did ONE session, and everybody agreed that we were wasting out time. Yet we learnt a couple of things then:

  • We could easily treat small stories, the S-sized ones
  • We could not easily decide about big stories, and almost always decided to break them into smaller stories

Now here is the thing : we had a product to build and we wanted to now how long it would take to build some of the pieces of it (we are not discussing why here). We realized that we could not give any (accurate) estimate to answer that very question unless the pieces were small enough.

So we decided to always work on small-enough pieces of work. Obvious.

Now if for some reason you believe that you need estimates, what you can try with your team, during your next Sprint Planning Meeting, is a session of Indian Planning Poker. You actually only need one card in your Planning Poker hand: the thumbs-up card. Stick the thumbs-up card on your forehead if the story is small enough, or as small as usual, and not if not…and tell me about the results.

Post-Agile : Software Development As A Service

Maybe a new way of working for post-agile organisations

I wrote most of this article long ago but decided to keep it as a draft until recently. I actually thought that was more of an utopia than of a real, possible thing…but this changed after I watched Neil Killick’s talk on #NoEstimates.

I’ve been practicing software development in a Kanban team for several years now, around a single product. A first observation I want to make here is that current “Agile” way works fine with long-term projects, at least if :

  • you build a stable team
  • you slice the work into small SMALL pieces of work
  • you try to keep the work items of the same size, asking simple questions like “Is this work item as small as usual?”
  • you limit work in progress at every level of your process
  • you get rid of “expert” effort estimates and use historical data, statistics and probabilities instead (possible because you built a stable system thanks to the first 4 points)

When I speak to people working in web agencies or other kinds of software development shops, they often tell me it’s not going to work with small projects, when teams must work on several projects at a time, for several customers. They also tell me about contracting problems : how to be Agile – and what are the benefits of it – when dealing with fixed-scope kind of contracts?

That’s why I’d like to describe my hopes for a post-agile world, hopefully solving the issues above, by describing the kind of practice I’d like to see emerge in the years to come. I call it Software Development As A Service. It goes like this:

  • The team implements some kind of Agile/Lean principles so as to get a stable process, like what I succinctly described above.
  • We consider the team as a whole, a system converting customer money into customer business value at a (statistically) predictable rate.
  • The customer and the shop agree on some sort of Service Level Agreement based on lead time distribution for the given team. Transparency is compulsory here.
  • The contract becomes a fixed-cost contract

So I guess that post-agile really is about customers becoming agile too, and embracing the fact that software development is partly made of uncertainty.

If you have any thought on that, feel free to leave a message here or on twitter.