On Responsive Development

Describing the difference, and similarities, of the two words iteration and increment have been very hard for most of us. Using paintings have never really “clicked” with me…

But now, there is a nice and clearifying description by Eivind Nordby. With the help of some well known guys and maybe someone not so well known, Eivind takes a step forward in understanding the concepts and explains that both can be applied in the process and the product dimensions, but might still mean “adding” (incremental) and “reworking” (iteratively).

And I suppose that it’s the fact that both addition and re-work can be applied in both the process and the product dimension, that makes it so hard to pinpoint and describe.

In a “true” agile sense we really like re-work in the process dimension (repeating the activities, so that we can get good at them). But we dislike re-work in the product dimension (could be considered “unknown amount of work left to do”) becase we want the functionality to be Done Done. In real life though, mostly we aren’t really, really Done Done. Sometimes because of misunderstandings, time constrains and what not, but also not seldom because of the “systems implementation uncertainty principle”, the fact that implementing a system changes both the perception of that system and the needs it should fulfill.

So I guess that we should continue to strive for pure incrementatlity in the product dimension, but sometimes accept a “failure” and then iterate a bit. Particularly to get the early feedback that is so essential for delivering the functionality and properties that are really needed, and not the percieved needs.

When I talk to project managers, or managers in general, one of their main concerns is the precision, or lack there of, in their planning. It is still common that development projects overrun their deadlines, resulting in frustration, loss of money and trust, and cause a lot of extra work in re-planning dependent activities. So many managers look to Agile for a solution to this problem.

But very few seem to realize the inherent problem in planning development work. It is not uncommon for managers of large projects to think of planning as a simple process of converting required functionality to manhours and then allocate enough people to do the hours. It seems to be working when planning other types of projects, so why shouldn’t it work for development?

Well, first, does it really work for other types of projects? Software people have always been blaimed for beeing the worst when it comes to planning; road work, house building, are always on time. Well, no, their not. At least not most of them.

What is development, really? Some people view software development as a production process: from the requirements manufacture the software. Sometimes it can be, but usually we then quickly create a tool that can do that repetetive work for us. So what’s left? Only the parts that are not repeatable. The ones that require engineering and design. That means that development work is a creative process. Or rather, it is problem solving. Constantly solving new problems is what development is. At its core it’s like solving a continuous flow of crossword puzzles. And like crossword puzzles, some things are easy, sometimes as hard as we thought. And some things, much, much harder. Did you ever give up on a crossword puzzle?

So can you tell me how long it’s going to take you to solve the Sunday crossword puzzle? Of course you can’t. You don’t know what the questions will be, what problems you will have to solve. So development work is inherently unplannable. Period.

How can we then promise anything at all about when something will be ready? Agile planning uses statistical methods to get planning to work. And statistical methods needs multiple values to work, you can’t use a single value as a basis for statistics. And you don’t do statistics by guessing, you measure. And the more data points you have, the better your statistics will be.

Doing the Sunday crosswords for a year will give you 52 datapoints, resulting in a reasonable probability in your guess for the next one. It still won’t give you any guarantees for how long the next one will take, but on average you will know. If you wanted to know the total time for the next 52, you’d have a pretty good guess.

If you do various sizes and types of crossword puzzles you could probably find some statistical correlation between the number of squares or questions in a puzzle and the time it took. This adds to you statistical samples, maybe up to a few hundred squares over a year, increasing the statistical probabillity of your future projections.

If you want to have good statistically based projections you need many actual samples, and many planned samples. And how do Agile planning help us with that?

By

• breaking down functionality into small parts
• always include everything required to keep quality
• measuring average development speed

Because development work is problem solving we need statistical support for our planning, and because we need statistical support for our planning we need many samples. The agile techniques to do that are small stories, done criteria, velocity and story points. And as many of them as we can get.

I’m doing a short “tour” together with two other speakers, Janne Lundberg from Atlas Copco Tools and Joakim Pilborg from KnowIT Technology Management. The theme of the tour is of course lean and agile and I’m talking about “Agile in the Large”, one of my favourite topics. Here are the slides that I show.

My talks from Agila Sverige 2009 (“Agile Sweden”) is now up on Vimeo, in fact all the lightning talks are. There are some good ones there. I’m embedding my two talks here. (Swedish only, sorry!)

Vem har ansvar för hastigheten? – Thomas Nilsson from agilasverige on Vimeo.

Reduktionismen, agilitetens fiende #1 – Thomas Nilsson from agilasverige on Vimeo.

For each new school of thinking models of that thinking are built, believed, proposed and adopted but also interpreted literaly. So on the note of my last post I’d like to quote George Box, “All models are wrong, some can be useful.” I try to live by this quote when I teach and coach. Believing in models as a true depiction of the real world are not only fundamentally wrong but also leads to fundamentalism or versusitis.

I found a wonderful definition of properties of a model which you might consider benchmarking your favorit model against, be it Scrum, XP, CMMI, Lean, Kanban, RUP or the process models of your own company.

I just found an excellent description, and name, for a “disease” that I hope I’m not afflicted by. You can see the effects of it in many places, particularly in development and engineering environments. I think many instances of Agile initiatives gone wrong can be traced back to someone with the disease. This article by Scott W. Ambler describes it in detail.

There has been a very active discussion on the LinkedIn Agile group on the topic of “Surely waterfall methodlogy is no more but who is secretly still using it? Come on, admit to it!!!“.

Interestingly enough, there were a number of people “admitting” to using waterfall 😉

My first comment in that discussion went on the lines of Agile and Waterfall not being two well-defined methods, but instead are strong simplifications based in two very different viewpoints. Waterfall being based in the thinking that we should think the whole thing through before starting, drawing and deciding on a detailed plan, write complete, reviewed & approved specifications before doing any implementation, and so on.

Agile on the other hand is based in the adaptive thinking. The best way to get to the goal is to start the journey. And ensuring that we get feedback along the way.

I would presume that all development organisations are doing something in between, at least I have not seen one doing “true” waterfall or “true” agile (in this sense, at least).

In the discussion, however, there where multiple comments along the lines that “once we get our first release out we can go agile, until then however we must think things through, so we’re sticking with waterfall”, or “in our environment demands on <insert requirement here> is so high that we cannot use agile”.

To me these arguments express a misconception of agile which I meet now and again. The misconception is that Waterfall is the “standard model” and if there are some particular circumstances you are “allowed to become” or “might benefit from” Agile.

I think the reasoning is backwards. There is no way that thinking everything through before starting can be better than starting out to get quick feedback. Unless your cost of adopting and adjusting based on that feedback is very costly. Agile, and Lean, techniques are very much about reducing this cost. It is the avoidance of this cost by reducing it that is Agile, and by avoiding it by avoiding the adjustment that is Waterfall.

So the only way to reason is to start with an Agile viewpoint, embrace the feedback and the adjustment that we can make based on it. That will bring us the best product to the market with the best possible cost/functionality/quality ration. If there are reasons why the cost of this adjustment is prohibitively high, look for ways to lower it, e.g. by adopting some of the Agile or Lean techniques. If that is impossible, move the problem towards a more “early decision”. But make sure that only influences the things which are prohibitively costly to act on the feedback.

I strongly believe that the viewpoint that Agile have is going to be, will have to be, the way all developing organisations will have to take in the future. The “Agile viewpoint” brings so much business benefits that companies not adopting it will not exist. There is no longer a price per hour competition, its about maximizing business value per invested dollar or euro.

Yesterday I held a 1 hour+ presentation/discussion for the Management Team of a large organisation. They met up at Vidbynäs Castle to discuss and find inspiration for their next steps into the future. I was invited to talk about Agile, of course.

There was a lot of good questions and I liked the audience very much. I hope I made some points valuable enough for them to bring back to their own part of the organisation. It is difficult to sift out the important points from the million things you want to say, but these are the slides I showed.

On a side note it was not summer as in the photograph, it was the first snow “storm” this winter with 15 cm of snow. The 200 km drive (one way) was, hmm, interesting…

I stumbled across Objectmentors website and noticed that they did help their customers with “Legacy Systems Design”. My customers have no need for help in that area, they are doing this just fine by themselves 😉