Riccardo Viviani - 25 July, 2019 - 8 ’ read

TDD: why I decided that debugging wasn’t worth my time.

Every tech company should write tests when they build software or apps. Yes, it takes time, and yes, it's hard at first, but well-written tests may be your one and only parachute when you're falling into the tangle of bugs. This is my goal today: give you a useful example to help you spend all the time you wasted on debugging in a different (and probably much funnier) way.

How much time do you spend debugging your code?

No need to tell you, dev friends.

Debugging – i.e the process of detecting and removing of existing and potential errors in a software code that can cause it to behave unexpectedly or crash – is the most time consuming and expensive activity a software developer has to do, and, often, the most frustrating.

Even though debug tools are becoming increasingly efficient, we must absolutely pay attention to avoiding bad habits related to this practice (especially when you’re dealing with your own code). Also, debugging always involves a risk of addiction and may cause an actual loss of money and opportunities for your company.

A 2013 research from Cambridge University reported that developers, generally, spend more than 50% of their time debugging, in order to have the code working and fix defects, trying to avoid catastrophic scenarios that may lead to delayed releases and, consequently, to shorter product life.

Ultimately, the research states that debugging costs can represent up to 75% of a project.

Of course, there are cases – very few, to be honest – when you really need to do some debugging, and, if you want your software to work, you can’t avoid checking, detecting and correcting errors or bugs.

For example, if you need to analyze a crash dump of the production systems to understand what’s gone wrong, or if you’re facing obscure paths in legacy code trying it in a spike, well: that’s when you can’t avoid debugging. But you should never perform this activity during the development phase; that, indeed, is the time for safer and cost-effective practices like Test Driven Development (TDD).

TDD: real-life usage example.

Just to be clear, TDD is a discipline, and as a discipline, it has specific rules that you need to follow consistently. As we mention in a previous post on Agile software development, they can be combined into a Red-Green-Refactor cycle:

Red phase – create a test and make it fail.
Green phase – make the test pass by proper implementation.
Refactor phase – remove duplication and improve design and readability of your code. It takes just a second to run a suite of tests after every change you make.

And now let’s see some down-to-earth experience.

Speaking about how much time programmers spend debugging during the development phase, some years ago I had the opportunity to compare the performance of three different teams working at the same project and developing similar plugins.

The plugins were designed for a system monitoring tool, to collect performance data from different sensors and sources. The three plugins were also similar in term of estimated effort.

Each team consisted of three people: the first two teams worked according to the classic approach code-test-debug, while the third team applied a Test-Driven Development approach; all teams included expert programmers.

The development environment used by the teams was Visual Studio, and the plugins were developed in C#; to get a correct idea of the development time versus debug time, I used a Visual Studio extension able to measure the percentage of time the IDE (Integrated development environment) was in “debug mode” compared to the time it was in “development mode”.

The results were quite impressive: the teams working with code-test-debug approach spent 56% of time debugging, while the group which adopted Test Driven Development practice spent less than 8% of its time in debug activities, and this happened especially during the refactoring phase, in order to fix unexpected failing tests (pointing out the necessity to add new tests for these scenarios).

…Yes, I know they shouldn’t have needed to debug at all, but sometimes it’s hard to get rid of a bad habit.

All teams completed their work and released the three plugins within the end of the sprint: plugin A was developed in 175 hours, plugin B in 180 hours, while plugin-TDD took almost 200 hours.

TDD: why I decided that debugging wasn't worth my time.

At first glance, it looks like TDD is more expensive than the classic approach code-test-debug. But let us consider some additional metrics, in particular, line of code (LOC), cyclomatic complexity (also known as McCabe metric) and max block depth.

According to Wikipedia, “Cyclomatic complexity is a software metric used to indicate the complexity of a method or function in a program. It is a quantitative measure of the number of linearly independent paths through a program’s source code”, while Maximum Block Depth is the maximum nested block depth level found in a method or function.

Now, looking at the code quality metrics, Test Driven Development overperformed the classic method on all the measurements.

In particular, the number of code lines written applying TDD varies from less 20% to less 30% than the code written using code-test-debug approach, while the complexity is 70%-75% lower.

LOC and Cyclomatic complexity are quite simple to measure, and they are crucial metrics to take into consideration during the development process. In fact, there is a close correlation between the number of code lines and the number of the defects: the more code you write, the higher are the chances to put a bug in.

Besides, even though complexity is not necessarily related to defects, it is strongly linked to the amount of time required to modify the code: the more complex the code is, the more time you need to add some new feature or change it.

But don’t lower your lens yet. Let’s continue to investigate these results.

Some weeks after the delivery of the three plugins, it turned out that a new feature was required: all plugins had to support a new event-driven protocol and generate alarms whenever the value of the measured quantities exceeded the threshold set.

In the following table, you can see the time required to add the new feature to the three plugins:

Yeah, I can hear all your “Wow” and “Ohhh”.

These new measurements are quite impressive: adding a new feature to a code developed with TDD practice required about 75% less time (and cost) than adding the same feature to a code developed with the traditional approach.

Also, the time saved at the moment of the first software release was lost as soon as the first change of requirements was needed.

Now, looking at the total cost of each plugin after the first feature update, we have the following data:

Okay, you still have one-hour-time less for plugin B compared to TDD plugin (and plugin A it’s barely two hours longer than the TDD plugin), but consider that this is only the first change in requirements: things will get much worse as new modifications arise, as reported in the following chart that describes the cost of implementations with the two different developing approaches.

The cost of a software cannot be evaluated only based on its development, but also on its maintenance over time.

In fact, most of the time, what seems to be a cost-effective methodology turns out to be the one that accumulates the highest technical debt – and higher technical debt will lead to a higher maintenance cost over time and, consequently, a slowdown in time to market.

Time spent on writing tests, then, is not wasted time. It’s the time you invest now to save your time later. Basically, in the long run, it could cost you a fortune if you don’t do TDD.

Last but not least, the TDD plugin has a huge advantage over its competitors: it has tests!

Right, guys, TDD has more than two hundred unit tests that ensure almost a complete code coverage (greater than 95% in this case).

It’d seem that TDD developers spend half of their time writing and running tests according to TDD practice, and it might also seem like TDD requires twice as much work as the traditional approach (since you have to write the tests AND the features), but at the end of the day, the time spent writing tests results in a set of repeatable and automatic test suites that can be run at your convenience, preserving you from potential regressions and allowing you to refactor your code safely. On the contrary, all the time spent on debugging would simply be wasted (as someone once said: “All those moments will be lost in time, like tears in rain”)

Your debug sessions, in fact, are arduous to repeat with the same conditions. You may have changed the environment where your code is running or, most likely, you had to change the code itself, fixing some wrong behavior (and breaking something else), especially if you are addicted to the bad practice of Debug Driven Development.

When code is complicated, it gets much harder to get anything done, and, at some point, no one wants to touch it, because even a minor code change could lead to an unpredictable effect.

On the contrary, by following TDD, developers are able to painlessly update their software to address new requirements or other unforeseen variables while being quite confident that no part of the code will be broken.

As the developers saying goes, “time saved is equal to money earned”.

Wrapping up.

We all give in to temptations, sometimes. And if you are a developer, the little F5 square (or whatever you have on your keyboard) to start debugging can be a great temptation.

But you can be smarter than this. Let the tests guide you through the development so that you can react to changes faster, managing to adapt your code more easily.

Use the debugger like a technical spike. You get in, find out what is going on, and write tests on what you learned. This allows you to avoid wasting your time by stepping over each line of your code in the debugger to make sure it’s doing what you expect it to do.

Today (besides the past experience of the experiment), working with Test Driven Development for more than 5 years here at Imagicle and having performed over 60,000 tests, I can say with no doubt that automated tests actually allows developers to release projects much faster, and with a higher quality.

In particular, we can summarize TDD 3 main benefits as follows:

Increase of the internal quality (perceived by the development team) and of the external quality (perceived by users) of the software;
Increase of the programmer’s productivity and cut of development costs;
Flexible software architecture, easily adaptable to the customer needs.

All right, then. I hope I have done my small part to help you discover a method that will simplify your daily work life (and smooth your relationships with colleagues by reducing the “WTF!” rate per minute!).

For us at Imagicle, an accurate software development technique is one of the things that allow us to offer our partners, resellers, and customers reliable, clean, extendable products, constantly implementing our UCX Suite with 3 seasonal releases, to provide you with an increasingly simple, fast and efficient experience.

Now, if I have convinced you that this is the best way not to get stuck in the bugs quicksand, it is time for you to find out how to apply TDD in the most effective way.

If you want to learn more on TDD, share your thoughts below.