What is your estimate of the Return on Investment for Test-Driven Development?

Question

At least on the surface, TDD seems to require additional work, because you need to write tests, etc.. Presumably this work is justified because it saves work later on.

If someone told you the amount of work required to write tests wasn't justfified, as many newbies do, what would you tell them? Hopefully you wouldn't say to write lots of tests because everyone else does it, but rather because you have some evidence.

Can you cite any studies showing work savings?

Realistically, there are probably some cases where the amount of work is not justified, and others where it is completely justified. Obviously, the cost of any errors would have a strong impact on this, since some errors (imagine code controlling an airplane) are very expensive while others may have little cost.

Further, TDD becomes more useful as team size grows. That may or may not be enough to justify its use.

Besides citing any studies, what are the determining factors for making TDD worthwhile?

Thanks Vegan -

I agree that TDD provides some benefit, but I prefer to make evidence-based decisions. The best way to make decisions is ones that are based on data. There are plenty of practices that seem like a good idea that, when analyzed, turn out to be less than effective.

Frequently our ideas about how efficacious a practice is will be influenced by things like popularity or how 'hot' it is at the moment.

The best way to make effective use of your time is to spend it on the action that will provide the highest return on investment - that might be TDD, or it might be something else.

Answer 1

I'm too lazy to research studies, but I can share my anecdotes.

TDD is a lot like making backups. It seems like a waste of time until something goes wrong. Then you're pretty darn glad that you've done it.

I once did rm -rf . /* instead of rm -rf ./* (that space made a huge difference. Fortunately I had backups.)

Similar scenario with tdd. There was a patch in the programming language I was using to fix a security problem. I ran my tests and found some of them failed. I was able to adjust my code to incorporate the changes quite quickly. And the error was insidious, it would have silently corrupted data, and would have been very difficult to identify.

Sometimes when you make a small change in one module of your code, you will see problems occur elsewhere. TDD really saves a lot of time and effort in debugging.

I'd say creating the tests doubles or triples the time to develop, but breaks even by the time you reach deployment. It really pays off once you get into the code maintenance phase, you can react to customer requests quickly and not worry as much about breaking the entire application.

I develop in Ruby on Rails, and testing is built right into the framework so it's very easy to get started. You may have a different experience when dealing with the issues of setting up your testing framework in .NET or JAVA.