A New Way Of Judging Frameworks: Where are the tests?

As a project for work gets ready for an alpha release, I’ve managed to eliminate all the serious bugs and now have some time for what should’ve been part of the project from the beginning: writing tests. As a lapsed tester, I always feel guilty about not writing tests when things start to get complicated with an application. Whenever you make a change, you have to point-and-click your way through the application to make sure that nothing has broken. I got tired of pointing-and-clicking, so I decided it was time to shut up and write some tests. Since I’m using Code Igniter instead of CakePHP for this project (did I mention that I inherited the project and couldn’t switch?) I started looking into the culture of testing surrounding Code Igniter. It’s weaker than a newborn baby.

The built-in testing system is some weird mishmash of unit testing and “you have to create your own test results template”. WTF? You can’t be serious. So I quickly ignored that. Besides, the first set of tests were going to be using PHPUnit combined with Selenium so I could make the “acceptance testing” automated. As an aside to this, I remember speaking with Sebastian Bergmann (the driving force behind PHPUnit) at a conference maybe 3 years ago and him telling me that he knew someone was working on getting Selenium and PHPUnit to play nicely together. Anyway, it’s working just fine.

So, next step was to see if I could use something else than the incredibly lame testing tools that Code Igniter came with. I found something called fooStack, that contained CIUnit. It lets you use PHPUnit with Code Igniter. Now we’re talking! Lickety split I had some unit tests for my models written. Well, after spending quite a lot of time screwing around with the tests and finding out the difference in relative paths between the application running via Apache and trying to test the models via the command line. Thanks to all those on Twitter who yelled “use php_sapi_name to figure out what environment you are in, you idiot!” at me.

But all this got me to thinking: how do some other PHP frameworks stack up in terms of having tests?

  • Code Igniter has ZERO tests for the framework itself. Perhaps I will mention this to my boss next time I’m asked to create a project from scracth. Hrm, guess I’ll have to trust that their code actually works the way it’s supposed to.
  • Zend Framework (full disclosure: I was one of the original authors of the Zend_Service_Audioscrobbler component) has huge amounts of tests for the framework itself. In fact, any new contributions MUST have working tests submitted with it.
  • CakePHP has lots of tests for the core, and I believe that I saw something in the changelog for CakePHP 1.2RC3 that indicated they were at 80% code coverage in terms of testing for the CakePHP core. Bug submissions for CakePHP are supposed to come with tests to prove that there is a repeatable problem.
  • Symfony recently rewrote it’s core tests and created their own testing framework for it.

So now when you start comparing frameworks to each other, I think it’s important you also consider how much effort has gone into creating tests for the core functionality of that framework. A well tested framework should mean far less surprises when using it. Given how many tools are out there to assist you in testing, I don’t think there are any excuses other than being totally lazy to not create tests for the core functionality of any application. Pay now, or pay later, but you will PAY to fix bugs in your application. Paying now is simply cheaper.

Would you stake your project on complex code written by someone else that is simply not testable by you? Perhaps I am not looking at from the proper distance because I am inside the framework-creator culture, but I can’t see why you wouldn’t want to provide tests to prove your stuff actually works the way you claim it does. Tests prove that the code is *behaving* the way you expect, and code that is testable is also usually easier to modify as time goes on. There is some sort of sick joy in making a change to code, running the unit tests, and seeing that you didn’t break anything.

Article Tags >> ||

CakePHP 1.2RC3 Released

Thanks to the power of Twitter I found out that The Show for CakePHP was resuscitated and brought back to life last night. I listened to the (surprisingly short) show, the main thrust of which was that CakePHP 1.2RC3 was released last night. Why should you care?

Well, maybe not so much on the last item but it’s ALWAYS good to see any project make a significant release. 1.2 has been in the works for more than 18 months, which seems to many to be way too long. I’m not so sure I share that sentiment. If the goal is to try and produce quality releases, then “it’s done when it’s done” is a reasonable answer. I’m sure this is not a new argument, so I won’t get into it here.

My own contributions to RC3 have been, what’s the word…non-existent. Non-computer hobbies + side work = no time to contribute to Cake. I found out that commit access for people who haven’t contributed anything in more than 60 days has been revoked, but that’s really just a minor annoyance. I’m sure I could get it back by asking nicely and actually having something to contribute.

But I think I’ve gotten back into Cake developing at the right time as I caved in and decided that my long-rumoured plan-a-road-trip site shall be built in CakePHP after all. Yes, I’ve been playing around with Django but I’m violating one of my favourite rules: Just Build It, Damnit! I’ve been fooling around with Balsamiq Mockups to do the pages and are now digging around for helpers to do the mapping mashup I need. If you want to mock me over how little progress I’ve made, feel free to visit the site. If you’d like to get involved at all (I could use some help on the “make a pretty UI for it” front) let me know.

Article Tags >> ||

Komodo and CakePHP

For those who remember how I was an editor waffler will remember that I play around with Komodo from time to time, and decided to get back into using it again. There are alpha 5.0 versions available so I downloaded one and tried it out. Seems to be less resource hungry than previous versions (as in it doesn’t cause my fan in my MacBook to start up every time I open Komodo). Now, since I know a few people at Activestate I emailed Jeff Griffiths to ask him a few questions. He responded by hacking together something very quickly for adding code intelligence for CakePHP to your views. If you’re a Komodo user (or are interested in trying out Komodo), try it out and let Jeff know how it’s working out.

Article Tags >> ||

What’s In Chris’ Brain: September 2008 Edition

It’s time again for a round-up of the ideas that have been bouncing around in my head this past month, because I don’t have enough material for a big blog post. Not yet, any way.

  • Oh look, another CakePHP book coming out that I can review if the author pulls some strings. Yes, I’m talking to YOU AD7Six
  • Although the title of the presentation is “Why I Hate Django”, Cal Henderson from Flickr shows why you simply cannot avoid frameworks when building applications. Deny this fact at your own peril.
  • If you’re not already reading their blog, check out Felix and Tim (and sometimes Nate) over at the Debuggable Ltd. blog where you can find some of the best CakePHP tips around. Felix is closing on on 30 straight days of blogging, though not always CakePHP related.
  • I’m going to be in Mountain View, CA next Wednesday (the 24th) for a super-secret meeting and have some time to kill before my red-eye flight back home. Maybe we can hook up so you can hurl abuse at me and so I can get enough alcohol in me to sleep on the red-eye flight back to Toronto?
Article Tags >>

Book Review: CakePHP Application Development

A few weeks ago I was approached by Pakt Publishing and asked if I was interested in reviewing the recent CakePHP book ‘CakePHP Application Development’ by Ahsanul Bari and Anupom Syam. In the interest of full disclosure, I have communicated with Ahsanul a few times via the CakePHP mailing list and on Twitter. We’re not total strangers, more like casual acquaintances who know each other through the CakePHP community.

One of the biggest criticisms in the past of CakePHP was the lack of published documentation on how to use the framework. As shocking as it was to me, there were people who preferred a paper copy of the Cookbook so they could have access to it when they might not be online (what, working on your computer and not online?!?) or to read offline. Naturally, several books about the framework have been published in the past few months, and I can’t think of a better book to recommend to people interested in learning the framework than this one. And I’m not just saying that because I got a copy for free. While the book is good, there are a few places where it could be even better.

The Good: Examples, Example, Examples

The target audience for this book is the right one that CakePHP is attracting: intermediate PHP developers who may have not used a framework before and are trying to figure out how to use one to their advantage. Without clear and relevant examples of how to use it, CakePHP would be just another framework struggling to gain acceptance with the PHP deveoper community. If you want examples on how to do most basic things with CakePHP, this book has them. Want to know how to configure CakePHP to work for your web server? Check! How do you get results from your database? Check! How do you use the ‘bake’ utility from the Cake console? Check! Validating form fields? Check!

I feel that the examples in this book are it’s strengths. The code is well written and every code example is followed by an explanation of what they are doing. Many times the example is not enough, you have to understand WHY it’s written that way.

The Good: Sample Applications Always Help

Almost half the book is devoted to creating an application from scratch, including configuring the often-misunderstood Auth component and showing how the built-in Javascript helper allows you to add Ajax-powered functionality via the Prototype and Scriptaculous Javascript libraries. Seeing an application being built from start to working prototype is an invaluable lesson for the “learn from example” crowd.

The Bad: Less Info On Doing Things The Cake Way

I feel that some more time could’ve been spent explaining CakePHP’s “convention over configuration” preferences and how it changes the way you approach building an application using CakePHP. Perhaps that wasn’t a topic for this book’s intended audience, since it’s only the deep thinkers and the rant-and-ravers (which I have been accused of belonging to) who like to discuss the philosophy behind CakePHP and the question of just what a framework is good for.

The Bad: Thin on Server Configuration Information

While there is a chapter devoted to getting CakePHP up-and-running using Apache and MySQL (specifically using WAMP), it would’ve been very helpful to see an example of the type of configuration recommended for production environments, where the APP directory is web-accessable while the CakePHP core is located somewhere outside the webroot, linked in via an option in the configuration files.

The Bad: No mention of the differences in debug levels

When I do my development work on a CakePHP application, I set the debug level to 1 so that I see all errors and the SQL statements being generated by my models. In production I have the debug level set to 0 as I don’t want error messages on the screen that reveal any information, and I certainly don’t want people seeing the SQL statements being executed. Sure, it’s security by obscurity but it’s also debugging information that’s cluttering up the screen.

The Bad: No mention of using the built-in testing

While I’m a lapsed tester, the fact that CakePHP will create the shells for writing unit tests for the models and controllers you create should not be overlooked. Although testing gets a bad rap (”It’s too hard, and it wastes my time!”) it’s a programming practice that should be encouraged.

The Ugly: CakePHP is still a moving target

While this in no way diminishes the efforts put in by the authors to create this excellent introduction to CakePHP, at the time it went to press CakePHP 1.2 was still not feature-final. Some tihngs have changed, and I expect some more things will change before CakePHP 1.2 is finalized and work on CakePHP 2.0 starts. This is the danger in writing about any programming language: the information could become obsolete at any time. But don’t get all upset about it, as it is likely only very minor details that have changed. The CakePHP core team does try to not break things *that* much.

So, don’t let the number of “bad” issues above dissuade you from checking out this book. The items I highlighted are probably bettered suited for a “Advanced CakePHP Application Development for Grumpy Developers” book. I would not hesitate to recommend this book to any semi-experienced PHP developer who is looking for a resource guide to get themselves started.

Article Tags >> ||
Want to advertise on this blog? Send email to chartjes@littlehart.net
GTcars Canadian Car Audio TurboDodge Audi Forum
Mustang Forum Dodge Intrepid Miata Turbo
GTscene Pontiac Bonneville