Measuring software development performance – Part IV : Debt minimization

In our previous articles about software development team performance, we talked about alignment productivity, and quality.

Today, let us talk about debt minimization.

Debt minimization capability

IT debt is a notion that is getting more and more popular at the analyst level. Gartner estimates IT debt to grow as large as 1 trillion $ by 2015.

Up until recently, IT departments have presented the cost of developing applications without really explaining or measuring the induced cost for the future.

But as soon as you develop and deploy an application, you generate recurring maintenance costs that will last as long as the application is used. This cost will disappear only when this application is replaced.

Experience shows that applications always last longer than anticipated for several reasons. Even when you think of an application as a temporary solution, there will be business changes or budget restrictions that may impact the timing of the next version; there might be issues, as with all software projects, causing late delivery; or even, possibly, there may be technical disruptions that result in the failure of subsequent application launch.

So delivering an application that will run with the minimal maintenance effort is actually one of the most important elements to consider when measuring performance of a software development team.

About five years ago, in a software vendor meeting, Microsoft had shared this slide. It was the evolution of developer teams for the Windows group split by roles. The blue bar indicates developers dedicated to maintenance, the orange bar indicates the ones dedicated to compatibility and the yellow indicates the one focused on innovation. This illustrates the evolution challenge cost (especially for an ISV) when a piece of software is used and successful. A lot of your development bandwidth is necessary to maintain the legacy.

The Windows slide also illustrates a very important point. Application debt is usually tricky to measure because most of the associated hidden costs are not included the pure corrective maintenance cost. Of course, when an application is really buggy, people notice they have a big issue and usually take some actions.

But in most cases, a significant part of the real debt is hidden in a bad design that translates into over-sized evolution maintenance costs, the issue becoming bigger and bigger over time. The slide above does not really indicate if the innovation part produces as much value as it did in the past. However, one can get some insights thinking that the effort was mostly part of some pre-Vista work.

Issues are difficult to avoid, because people usually only realize there is a problem after it is too late. It is a bit like the “leaning tower of Pisa”. And the real solution in these cases is to rebuild, which organizations think they cannot afford.

So they will spend much more money in maintaining a badly-designed system but – because it is usually less visible than the tower of Pisa in software – this fact will be hidden in operating costs and associated with the cost of “unreasonable evolutions” asked by those “over-demanding business users”.

If your evolution costs look like the non-industrialized curve below, it is time to think about a real modernization of your application:

In our view, there are two different kinds of systems:

1. The ones which are very stable functionally. With these systems, even if an individual evolution might be too costly, it is probably good to maintain it for long as possible, even when the technology is very old. People often launch project when the technology is no longer supported by vendors. In our view, this is not a so important argument, when your system has run for decades. There is little risk that it stops with such a history behind.
2. The ones which are still alive because the business requires relatively quick adjustments to the application. With these systems, if you have run into a “heavy debt” situation, it is always better for the long term to re-design and solve the issue. The tricky thing is that you probably need to do this “by pieces”, to secure the success of the project; “big-bang” projects often lead to major failures. Throwing away old pieces of code, reducing the size of the code base and aligning it to more recent technology will save a lot of money rather quickly. But you need to do it the right way, with the proper approach, the right people and efficient tooling.

The leverage effect of application debt

As your application grows, the legacy you have developed weighs on maintenance, and over-complexity may severely impact the cost of evolution.

Let us take a typical scenario for an application that has a 10 year lifecycle. The following table describes a “nominal scenario”, with an initial development of 100 and evolutions over the following years. Maintenance is calculated as 15% of the cumulated past workload.

Nominal scenario

Cost line	Year 1	Year 2	Year 3	Year 4	Year 5	Year 6	Year 7	Year 8	Year 9	Year 10	Total
Initial Development	100										100
Evolution		30	15	50	25	10	40	25	16	5	216
Maintenance		15	20	22	29	33	35	41	44	47	284
Yearly cost	100	45	35	72	54	43	75	66	60	52	600

This scenario gives an overall cost of 6 times the initial cost, which is consistent with many projects that live normally and have certain levels of evolution over time to align with the business.

Now, let us imagine a team that is not fully optimal but adds a bit of extra complexity with a classical factor of 20% a year, which is not outrageous. Still, we estimate that the complexity multiplies over the year and increases the cost of evolutions by this cumulative approach. We still calculate maintenance by a 15% of the past workload.

Yearly over-complexity deviation factor: 20%

Cost line	Year 1	Year 2	Year 3	Year 4	Year 5	Year 6	Year 7	Year 8	Year 9	Year 10	Total
Complexity deviation	1,2	1,2	1,2	1,2	1,2	1,2	1,2	1,2	1,2	1,2
Initial Development	120	0	0	0	0	0	0	0	0	0	120
Evolution	0	43	26	104	62	30	143	107	83	31	629
Maintenance cost	0	18	24	28	44	53	58	79	95	108	508
	120	61	50	132	106	83	201	187	178	139	1257

The cumulated cost of the project is doubled in the end, and interestingly, the cost of the evolution in year 10 is 6 times what it would be without the burden of this overly complex legacy!

Another observation that might be of interest is to understand what happens when projects start the wrong way. With a starting complexity deviation factor of 30% at the beginning, even if you react and behave perfectly over the following years by progressively lowering the factor from 1,3 to 1, you will never decrease your yearly maintenance cost enough to align with the previous team.

Yearly over-complexity deviation factor: starting 30% but going down to zero

Cost line	Year 1	Year 2	Year 3	Year 4	Year 5	Year 6	Year 7	Year 8	Year 9	Year 10	Total
Complexity deviation	1,3	1,3	1,3	1,2	1,2	1,2	1,1	1,1	1	1
Initial Development	130	0	0	0	0	0	0	0	0	0	130
Evolution	0	51	33	132	79	38	167	115	73	23	711
Maintenance cost	0	20	27	32	52	64	69	94	112	123	592
	130	70	60	164	131	102	236	209	185	146	1433

Projects that start bad never end well, this is also a field observation that one can make. When it is really bad, it may be better to just start a new project. Here is a graphical version of the application cumulated costs of the three above scenarios.

Although a bit simplistic, we believe this model is consistent with some of our observations in the field where complexity translates into cost increases over time.

As we explained in the article about productivity, it is also very important to be able to estimate the real size and complexity of your project. A good estimation is also a way to ensure developing only the essential elements, because every single line of code that is not strictly necessary will be very costly over the whole application life cycle.

From our observations in the field, we are always surprised to find people are not familiar with simple metrics such as:

• Number of business entities or tables,
• Number of screens / pages,
• Number of reports,
• Number of lines of code.

For sure, one needs these factual elements (and even better, other details such as business rules, data properties, user interface fields, etc…) to be able to evaluate costs, both for producing software but also for maintaining it and implementing changes.

If the only criterion used to measure the impact of a feature request is workload, as defined by your developers, you may be in trouble. Firstly, you have no way of challenging this estimate, and second, there is a high degree of variability among developer skills.

As always, feel free to comment and share your own experiences.

Daniel COHEN-ZARDI
SoftFluent CEO