One of the things I (foolishly perhaps) promised myself when I started up a new project at work was that I was going to use Test Driven Development (or TDD for short). I advocated using a web application framework that supports good testing practices and discussed how it would be beneficial when trying to track down and solve bugs to have a set of tests. I ran through the impact of my horrible decision to use a framework without a good culture of testing in a previous project: fixing things became tedious as I could not run automated tests to verify that things were working properly.
But a funny thing has happened while building the new application. Sadly, there are no automated tests, and the question I’ve been asking myself is “other than being lazy as hell, why did you not write tests?”. The answer I seem to have come up with (or justified is probably the better word) is “I don’t see anything complicated in what I’m doing that I need to write a test for.” Before you dismiss my thoughts on this, consider the architecture of what I’ve built:
We have a backend consisting of MySQL with two distinct databases on it: one containing raw stats for sporting events, the other summarized records about fantasy points accumulated by players. My thought was to make the database do all the work of crunching the numbers for me as a daily cron job and then the web front end only needs to do reads. With proper indexing of the tables, performance is at an acceptable level with a full season’s set of test data. Sounds like a pretty decent strategy to me.
The scripts that collect the raw data and pre-calculate fantasy point totals are written in Python. The web site that displays the fantasy points info was done with Zend Framework. So what sort of tests should I have written for this. I’ve come around to the idea that you should write tests for things that are non-trivial. Then you get into the problem of deciding what is non-trivial:
- Verify that point totals are being calculated correctly
- Verify that when players are assigned to teams in our admin panel they actually end up, you know, being on the proper team
Every else seems to me to be, well, not worth testing. Make sure teams are sorted in order of total fantasy points? Database does that for me. Make sure list of players for a specific team contain expected players for a team? Um, database does that for me(?). I can verify that players are properly changing teams in the admin panel by hand. Of course, I shouldn’t be testing things by hand, but I’m wondering if I’m either missing the point or have stumbled upon something here.
Am I missing something here? Am I doing it wrong? I do believe I need some tests, but only for the things that are not super-obvious. I’d appreciate people’s thoughts on this. Maybe I just need someone else to look at it the app as a whole and say “we should be testing for the following things”. The curse of the lone programmer maybe?
Tags: PHP, Python, TDD, tests, Zend Framework
>>crunched by the database
If the database is doing all the crunching, then you should write some tests to verify that it’s crunching correctly.
>>I don’t see anything complicated in what I’m doing that I need to write a test for.
Sure, but it’s probably not you who’ll be fixing whatever problem crops up.
>>you should write tests for things that are non-trivial
as well as for any bugs you fix.
>>I do believe I need some tests for the things that are not super-obvious
Test the things that provide the user with the most value
Test everything that directly or indirectly provides your company with revenue (i.e. checkout cart, signup, login, password reset)
the reality then is that errors are not kicking you hard enough or code-base is not shared , functionality is not lost on version upgrades or no backward compliance is needed
so take a simple web interface that is configurable with 9 dropdowns or checkboxes , test out all the combinations as they all affect some result on the app frontend. Good luck with testing out 24310 combinations by hand and after testing add another dropdown for configuration and test again only this time there would be 92378 combinations and so on. That’s where tests come in handy.
Of course you are right no tests are needed for small scale stuff that can be supported with ease by one developer
You realize what tests are needed, when you catch the first bug because you have a test or you miss one because you don’t.
I also find a value in really simple tests, even if they seem unnecessary at the time. They can be the starting point for the more important ones. And sometimes even those can save you.
But I don’t write enough tests either. Nobody does.
I was recently of the same mind as you, but I’m coming around on my last project, which was a very simple but critical enterprise project – I am glad we have tests to run before deploying, even if coverage is still only about 75%. I don’t have much to say that TDD advocates haven’t said ad nauseum elsewhere. But I will point out that the trivial functionality you mention should be even more trivial to test – one line. So it’s probably worth doing on the chance that some unknown future upgrade causes a problem (maybe a new DB server version will have a sorting bug…)
Also you might take a step back from the unit tests and consider client-side testing with Selenium, which can really rock and helps me sleep at night too.
Writing tests for all functionalities allows you to be confident for every change you commit to your code. It’s simple to brake something far far away from the code you are working on. With automated test you know every time that all works, you don’t need to test anything manually. Even the simple things.
>>Make sure teams are sorted in order of total fantasy points? Database does that for me.
I think if you write a testing case for checking the correct sorting of the records from the database you are not really testing the proper working of the database engine, you are making sure that no one else change your functionality and, at the same time, you are documenting your software (assuming that you are not the only person working in your code).
Remember to write tests for incorrect inputs. Remember to write tests for malformed inputs. (SQL injection attacks).
[...] This post was mentioned on Twitter by Chris Hartjes. Chris Hartjes said: New blog post about me struggling with TDD, comments appreciated -> http://bit.ly/cSUxFT [...]