Why measure?

Something I have heard quite often of late are discussions around Agile Metrics. Generally these take the form of “How to measure Agile success” or “Is Agile working for you?” style discussions.

These often state that is only a matter of time before someone will ask for hard evidence that all of that hard work to become agile is paying off!

So what should we measure? To answer that, we should look at some of the popularly suggested measures and also see what the Agile Manifesto principles say.

Suggested measures

Time to market

“Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.”1

This on it’s own should show some improvement over previous processes. If a company has to wait for six months or more for a product to get to market and start saving money / making money / spreading joy in a waterfall framework, surely if their software is delivered in two weeks (or even two months) when using an agile framework, all (or even some) of the business value is realised far sooner.

I can think of a project that I worked on in the past, where sitting down with the stakeholders led to a project that we felt would take around six months. After fully documenting the entire solution, getting it agreed and signed off, and finally built and live – the landscape had changed. Only half of the solution was required, and even parts of that were being used for a purpose for which they were not designed. In addition to this, parts of the system that could have yielded significant business value could have been delivered far earlier.

Cost

The problem in measuring this is that unless a project is conducted twice using two different methodologies, it is impossible to answer objectively.

The agile manifesto does not actually promise reduced costs. In some cases it may be more expensive. It is a balance between doing things right the first time, and doing only what is necessary at the time. Doing things right the first time may delay getting something to market, and not reaping the financial rewards early. Doing only what is necessary at the time may mean repeated rework of particular parts of the system leading to increased cost.

Quality

“Continuous attention to technical excellence and good design enhances agility.”1

This is an agile principle, but it is down to the team to really ensure this happens. Certain practices can be suggested, such as test driven development and pair programming, but these could just as easily exist in a waterfall framework, or even be absent in an agile one.

Developers should be given the time to ensure that anything that they produce is quality. It just so happens that this is more easily achieved in an agile environment where software is released regularly, keeping customers satisfied. When the pressure is on to deliver, corners can be cut.

Consider a six month project. As soon as the project appears to be falling behind, or even worse the deadline has already passed, quality is often the first thing to suffer. “Let’s just get it done!” is the cry I have heard from Project Managers and Developers alike. This in turn leads to increased support and technical debt, and technical debt will slow future projects, and turn increase cost.

When software is delivered in increments, quality is built in from the beginning, and usually not abandoned if a single task is not going to finish within an iteration.

Customer satisfaction

“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”1

What really satisfies the Customer? Well it certainly isn’t waiting six months to a year for software! If the software is quality and timely, the customer will be satisfied.

“Business people and developers must work together daily throughout the project.”1

Keep the customer in the loop. All of the time. When the customer has a constant understanding of what is happening, and has early and constant input, the customer will be satisfied.
This must be an improvement on showing nothing for extended periods of time.

Flexibility

“Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.”1

I don’t remember seeing anything like that in Waterfall World! What I do remember were painful conversations and tons of change request documents. In agile, because so little design is done up front, and business people are close by, the development is far more collaborative. When change does happen, it is welcomed.

Team Happiness

I find it hard to believe that anyone ever sold an agile transformation by saying “The team will be happier”. It’s a lovely idea, but to be honest, I can’t see many companies agreeing to the pain and cost of an agile transformation based on this benefit. I absolutely agree that under agile, development teams are happier, and that it is desirable. I don’t think that this on it’s own is really reason to ‘go agile’, and to use this as a proof of success feels to me like a little bit of a cop out. Anyway….

“Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.”1

“The best architectures, requirements, and designs emerge from self-organizing teams.”1

I could honestly talk about this all day (and probably will in a future post). As a software engineer that transitioned from waterfall to agile, painful as it was, I was much happier. I would never work as a developer in a waterfall environment again.

“Working software is the primary measure of progress.”1

This is not generally a suggested measure, which is surprising, considering that it is the only one suggested by the agile manifesto. The fact it says it is the primary measure does suggest that there could be more, but it is the most important one, and I rarely see it mentioned in agile metric discussions!

So why measure…?

For me, the benefits should be *so clear* that there shouldn’t be any need to measure, at least not in an attempt to produce hard evidence that an agile methodology has provided benefits over a waterfall methodology.

I wonder sometimes, that if metrics are resorted to in an attempt to justify a process that if working correctly, should be producing benefits that are plain to see. However, I do believe that certain metrics for use only within the team, are useful to help a team grow and track their own progress.

Personally, I would feel really uncomfortable trying to justify an agile transformation by saying “well… the team are happier…”