Measuring software development performance – Part II: Productivity

In our last article about software development team performance, we developed the alignment pillar.

Today, let us talk about productivity.

Productivity

Productivity is obviously a critical performance indicator in every business. But as software developers are costly and scarce resources, it is all the more important in that area.

Still, and although developing software is not really a new discipline, there is no universally recognized methodology to measure software development team productivity. The first thing to ask is: "Why is it so?"

First, by definition, each piece of software built is unique. You don’t need to build two times the exact same piece of software as you can reproduce it indefinitely by copying it. This makes it really impossible to make a formal and rigorous comparison between two pieces of software.

Second, the up-to-date technology is something that changes at a quite rapid pace. So each time a methodology relative to a certain wave of technology is reliable enough, it is generally already obsolete.

Third, there is a huge area for creativity in finding the different solutions to a unique problem. So measuring the result in terms of "lines of code", for example, is measuring the size of the solution, not the problem. And among the solutions to a problem, the lightest solution is actually the most valuable one, not only because it might cost less to produce (sometimes not by the way, it can take longer to write a synthetic text than a long one), but mostly because it costs less to maintain.

This said, should we give up with the idea of measuring software development productivity?

This is not our opinion at SoftFluent.

Let us start by looking at different industry approaches to address the topic.

Agile methodologies

Undoubtedly, this is the major methodology trend at the moment in software development. Still, the focus of agile methodologies is mostly about securing an efficient collaborative process between product management and developers. It addresses pretty well the alignment challenge we talked about in our previous post.

Talking about productivity, the standard practice is to evaluate development task using complexity levels and "points". Depending on the level of complexity, a non-linear formula gives the development time.

In many agile methodologies, the evaluation uses Fibonacci series to find development time depending on complexity. Fibonacci series is built in a way each number of the series is the sum of the two previous numbers, which gives us this kind of table.

 

Complexity level	Estimated development time ratio
1	1
2	2
3	3
4	5
5	8
6	13
7	21
8	34

When managing the backlog of features, these are evaluated in complexity which translates into points. The classical measure used in agile methodologies then focuses on velocity which is the number of points developed per iteration, as per the schema below.

This works pretty well to evaluate the relative efficiency of a given team over time. But the main weakness is that it only gives a relative measure. Points are defined by the development team and cannot be compared to anyone else on the market. So there is no way to benchmark your development team using this approach only.

Function points analysis

Function points analysis is an evaluation method that addresses this challenge by defining in an "absolute" way what needs to be produced in terms of complexity.

Using categories such as:

• External Inquiries
• External Interfaces
• Internal Logical Files
• External Outputs
• External Inputs

One needs to describe in a detailed way the different functionality of the software in each area.

There are then complexity matrixes where one can position the different functionality and calculate the adjusted function points of their project.

For people who want to know more, we recommend this document from Software Metrics, a company specialized in that area.

Working with traditional waterfall methodologies with a significant design phase can work pretty well and should be comparable from project to project.

The main drawbacks of this method are the following:

• it requires a significant skill investment to really master the approach and get the expected results and measure,
• it requires a quite rigorous and complete process in describing the software, an effort that we see less and less on the field as people have shifted to agile methodologies.

Our own approach for business software

In our view, we think that getting some absolute measure of the software that is produced by your team for a given budget is really important.

And as we understand the challenges mentioned above, we adjust our own simplified methodology to measure the size of a software development project, specializing in the area of "business applications".

When we talk of business applications, we consider all kind of applications that usually manipulate structured data, internal applications focused on a specific business or vertical, customer-relationship management software, complex e-commerce or Extranet web sites, and etc.

In our experience, the size and complexity of a project is a function of a quite limited number of parameters, that compares from a project to another, whatever the sector or domain area.

Each application usually has a bit of specificity in some way (workflow complexity, external connection challenges, dynamic user interfaces, customized advanced business calculations or multi-tenancy for SaaS applications), but we observed that the impact it has in terms of development cost is relatively constant from a project to another.

If you combine several complexity factors, you may also find ratios that help you deal with this complexity and you should be able to evaluate a project based on the following numbers:

• number of entities
• number of simple business rules
• number of advanced business rules (cross-entity for example)
• number of user interface elements (screens, web pages)
• number of reports
• number of external interfaces
• number of batch calculations or processes
• number of technology variations for components (rich-client, web page, mobile, database providers, cloud systems)

In projects using CodeFluent Entities, we often even found out that a project (including all project management dimensions) should never cost more than:

• USD 5,000 per entity if it is an enterprise application,
• USD 8,000 per entity if this is a packaged product that needs to be deployed at various customers (as it is generally more complex than a single-deployment application).

This gives a very simple way to detect issues when auditing real-world projects with a bit of history and legacy. We also compare this information with the total number of lines of code for the application. Our experience gives us relatively constant ratios correlating the number of entities to the numbers of lines of code in languages such as C# or VB.NET.

We also found out that CodeFluent Entities reduces the number of manually written lines of code by a factor of 2, with a direct link to the future maintenance effort.

And of course, the good news when using our tool, is that you get at least metrics about all the parameters mentioned above, whereas we still see many "blind" customers on the field, that are unable to know these key metrics.

What about your own experience?

We are eager to hear from your experiences, as this is a very hot topic.

Daniel COHEN-ZARDI
SoftFluent CEO