For a project at work, I am using PHP (specifically Zend Framework) for the front-end but are using Python scripts run as a cron-job (and also on-demand when statistical corrections occur) to collect raw stats for a variety of sports, and then generate fantasy point totals for the games we run. I'm already using PHPUnit for tests of the front end, and I decided to to use py.test to test my Python scripts.

Setting up tests in Python was pretty simple. Here's one of my test scripts:

Very similar to tests with PHPUnit, right? So now that I had both PHPUnit tests and py.tests tests (hrm, is there are better way to say that?) to run, I had to figure out how to automatically run them. More specifically, how to get our installation of Hudson to run them.

Getting PHPUnit to play nice with Hudson was relatively easy. I installed the NUnit plugin for Hudson, made sure I installed phpunit, and then I added it's use to my build scripts. However, the strength of Hudson is that with the use of another plugin I could read reports of all those tests. So when things failed, I would not have to look at the console output to figure things out. There's a place in the Hudson config where you can configure this:


Now, I figured that the same thing could be done with py.test. It had an option so that at run-time you could tell it where to put JUnit-compatible test result files. After a little tinkering, I got it to work. First step was adding execution of it to my build script. Here is the latest-and-greatest version of that script:

mkdir /var/www/games-hudson/${BUILD_ID}
cd ${WORKSPACE}/games
/usr/local/zend/bin/php doctrine-cli migrate
cd ${WORKSPACE}/games/tests
/usr/local/zend/bin/phpunit --log-junit=${WORKSPACE}/build/logs/phpunit-results.xml
cd ${WORKSPACE}/games/scripts
/usr/bin/py.test --junitxml=${WORKSPACE}/build/logs/pytest-xmlrunner.xml
cp -R /var/lib/hudson/jobs/${JOB_NAME}/workspace/games/* /var/www/games-hudson/${BUILD_ID}
chmod 777 /var/www/games-hudson/${BUILD_ID}/tmp
rm -rf /var/www/games-hudson/current
ln -sf /var/www/games-hudson/${BUILD_ID} /var/www/games-hudson/current

Next, I then told Huson where it could find the JUnit-compatible files generated by py.test:

So there you have it. Now, when I do a commit and trigger a Hudson build, both my PHPUnit and Python tests get run. And there is output to check, so I don't have to dig through console output to figure things out.

PHPUnit

Since Kevin and I are travelling together through the world of snake handling, we have been looking at the available options for building web applications in that environment. Our conclusions have been surprising to ourselves, but I think they can be very instructive to others. These two lessons are applicable to other programming languages as well, so feel free to substitute your favourite language where applicable.

Lesson 1: Full-stack gets you in the door, lightweight lets you find your niche

If you want to learn Python for the web, the first place you go is Django. It is a full-stack solution for building things in Python. Awesome documentation, and with some help from Python's online documentation, you can build an app in a reasonable time frame without knowing a ton of Python. Awesome tutorials and code samples FTW! I have built a Django app and briefly made it public. It was, like so many other things, a piece of crap missing functionality so I am trying to rectify that in my ever-dwindling spare time. I did not find any serious obstacles to building this app, except for figuring out how to do some dynamic form-field-generating code. It took me a while to find the right info (amazing how having the correct keywords in your search helps) but I did.

As an old warrior of frameworks (10 and counting across 3 languages) I found that because I already understood the concepts behind MVC (or MVT in the case of Django) I could concentrate on learning Python first, and then the unique features of the framework after that.

In many ways, my early experiments with Rails (back in 2004-2005) helped me understand frameworks a lot better. All frameworks have their own magic methods for doing things, with Rails making heavy use of Ruby's ability to create Domain Specific Languages to make Rails the incredibly useful framework that it is today. But eventually, your skill with the language and the framework gets to the point where you see the limitations of the conventions and magic methods the framework uses.

Now that I've gotten my hooks into building stuff with Django, I now recognize the same seductive promises that I found in things like CakePHP: the full-stack takes you away from actually figuring out how to code things, and instead you find yourself just using all the built-in methods. Don't get me wrong, the built-in stuff for Django is awesome. But if I want to push my skills forward so I can say "I can build stuff for the web using Python" instead of "I can build stuff for the web using Django" I need to use other tools. What if I want to run my stuff on Google App Engine? Sure, there are these hacks available for Django that make it "usable" on GAE, but that's not what I want. And what about the use of WSGI instead of relying on mod_python? Forgive me if this stuff comes across as technobabble. It was technobabble to me at one time too.

So now the next step is (after finishing the current Django app) is to build an application using a much more lightweight framework. Why? Because it's time to actually learn how to do something instead of relying on magic methods. Because once I learn to write code to duplicate those magic methods, I'll be able to use Django *and* Flask. And that, my friends, is how you make yourself useful. That and the ability to run the app both as a standalone WSGI app or on Google App Engine seems like a bonus to me.

So now that you've gone from full-stack to lightweight, it's time to look at the next lesson...

Lesson 2: Don't use anything made by One Guy

This will undoubtably be a touchy subject for some. Let me try and explain it.

In the quest for the lightweight solution, I ran across web2py. Nice and lightweight, awesome documentation, but it's all driven by ONE GUY. Experience has taught me that projects driven by ONE GUY end up looking like how the ONE GUY would do it. Sometimes the ONE GUY is brilliant and it all works out. That is rare, in my experience. Open source projects are littered with the wreckage caused by egos and control freaks who were unwilling to accept outside help to solve problems.

So look for projects where it appears there are solid contributors beyond ONE GUY. web2py sure has an impressive list of people who contribute, but is Massimo the ONE GUY when it comes to actually making non-trivial code changes? I sure hope not.

There will be no test afterwards

While I do not expect other people's experiences to be the same as mine, I'm pretty sure I am not the only person who has learned these lessons. So the next time you are evaluating using a web application framework (in a new language or an existing one) think back on these lessons.

A programming language is only as good as the community that supports. By supporting it, I mean willing to get together several times a year for conferences and bust their asses to have people share interesting talks with everyone else. Look at Python and Ruby. Awesome languages to work with. But even more awesome are the conferences that serve as anchors for the community.

There seems to be lots of little regional Ruby conferences, which give the language itself a unique flavour in return. Of course, it helps that Ruby can be used quite easily outside the traditional web application building environment. I don't see so many regional Python conferences, just PyCon and DjangoCon both in the US and over in Europe. It could be that I'm not looking hard enough.

As for PHP, to be fair, it shows up at pretty much any open source web-related conference. OS Bridge. OSCON. Codemash. I could go on, but I am not in the conference-promotion business. I mean, why is there no Great Canadian PHP Conference? I know there was phpworks in Toronto back in the mid 2000's. It was my first conference I ever went to. But I'll tell you why there is no Great Canadian PHP Conference: there is no PHP community in Toronto.

Oh sure, there are tons of PHP jobs to be had in Toronto. Lots of code monkey positions to be filled. But the Toronto PHP User's group has been dormant since last year. I sent an email to the organizer telling him I wanted to help out at least 3 days ago. Haven't heard a damn thing back. Perhaps it is time to go around such a person and just do it myself.

See, without the PHP community I do not have a career. Once I took the plunge, submitted a talk, and got accepted (I gave a talk about what the PHP community could learn from Ruby on Rails) I found I had suddenly joined the PHP *community*. Instantly I had people asking me questions AND people willing to answer my questions. Like I was their long-lost buddy. That was an awesome feeling, let me tell you. But even more important were the people I met.

As a result of the community, I have found myself in the awesome position that every time I needed a job, it took less than a week to get one. It was a combination of my blogging and my work in making those connections in the community that put me in this position, but it is sure a nice feeling to be able to convince your nervous wife that we were not about to face a rough patch while I struggled to find work. So it's a no-brainer for me to continue to submit talks to conferences and try to attend even when I don't speak. It's about giving something back to the community that has helped me grow my career.

So don't just *use* your language of choice. Try and become part of that community. Doesn't matter if it's Python, Ruby, or good-old-solved-the-web-problem-first PHP. You'd be surprised what doors are opened up by your participation.

But a funny thing has happened while building the new application. Sadly, there are no automated tests, and the question I've been asking myself is "other than being lazy as hell, why did you not write tests?". The answer I seem to have come up with (or justified is probably the better word) is "I don't see anything complicated in what I'm doing that I need to write a test for." Before you dismiss my thoughts on this, consider the architecture of what I've built:

We have a backend consisting of MySQL with two distinct databases on it: one containing raw stats for sporting events, the other summarized records about fantasy points accumulated by players. My thought was to make the database do all the work of crunching the numbers for me as a daily cron job and then the web front end only needs to do reads. With proper indexing of the tables, performance is at an acceptable level with a full season's set of test data. Sounds like a pretty decent strategy to me.

The scripts that collect the raw data and pre-calculate fantasy point totals are written in Python. The web site that displays the fantasy points info was done with Zend Framework. So what sort of tests should I have written for this. I've come around to the idea that you should write tests for things that are non-trivial. Then you get into the problem of deciding what is non-trivial:

• Verify that point totals are being calculated correctly
• Verify that when players are assigned to teams in our admin panel they actually end up, you know, being on the proper team

Every else seems to me to be, well, not worth testing. Make sure teams are sorted in order of total fantasy points? Database does that for me. Make sure list of players for a specific team contain expected players for a team? Um, database does that for me(?). I can verify that players are properly changing teams in the admin panel by hand. Of course, I shouldn't be testing things by hand, but I'm wondering if I'm either missing the point or have stumbled upon something here.

Am I missing something here? Am I doing it wrong? I do believe I need some tests, but only for the things that are not super-obvious. I'd appreciate people's thoughts on this. Maybe I just need someone else to look at it the app as a whole and say "we should be testing for the following things". The curse of the lone programmer maybe?

What it really became was a tool for me to learn how to write a solid web-app using Python and Django. I'm happy to say I finally have a version that I feel comfortable sharing with the rest of you. So feel free to check it out. Pound on it, try and break it and I will also read your feedback and comments. I'm sure you can figure out my email address...

Also, I've found it has been easy to pick up not only Python and Django itself. I'm ready to tackle some other applications using Django, and I have one just in mind.

Yes, I am aware at how spartan it looks right now. I'm more focussed on functionality than how it looks right now. I've already gotten some interesting ideas on expanding the app to include adding in images to go with locations, so we'll see how that goes.

For those who are curious, I built it using the following:

• Python 2.6
• Django 1.2 beta 1
• CouchDB to store the trip plans
• couchdb-python
• GeoPy
• django-registration
• Google Maps services
• Django Debug Toolbar
• deployed using Nginx + Gunicorn, configured with the help of Eric Florenzano's awesome Django Advent article about deploying Django using FastCGI because it mentioned how you could use Gunicorn

Some of this is due to my desire to always push myself to learn new things, so sometimes the joy of discovery tramples the ability to get things done as quickly as I would like. So I've been looking around at how other people have been solving problems in an attempt to discover blind spots I've developed. The first person I found that made me say out loud "boy, does this guy make me feel stupid" during this process is Geoffrey Grosenbach.

About 5 years ago I was looking to get into the Ruby on Rails world. I had bought the Pick Axe Book and the 1st edition of the Rails book. Through my online digging I discovered the Ruby on Rails podcast, run at the time by the previously-mentioned Geoffrey Grosenbach. It was all cool and interesting, and he seemed to have a knack for asking the right kind of questions. Either that or he was really good at the editing.

Alas, things didn't work out. I built a Rails site, but it had some problems due to my n00bishness with the framework and the language. Let's just say I wasn't using Ruby best practices. The final nail in the coffin for my Ruby aspirations was when I turned down a Rails gig to instead take a telecommuting PHP gig (life choices dictated that one) and I drifted off into other things. But I kept an eye on what Geoffrey was doing anyway and was happy to see him find success with Peepcode along the way.

Then I came across an awesome blog post from him where he talked about how he builds the Peepcode Blog. I read it and was totally blown away, and totally humbled by the simplicity of what he has done. Rather than simply do some link bait and show you everything he used, you can go and look at it

He explained that his goal was to produce an "art blog" for Peepcode, where every blog post would have it's own unique look to it. In order to do this, he had to think about what goes into building something like this:

• A CSS grid layout system, because it's really too hard to always come up with your own unique layouts. A grid makes it easier to make interesting content I think.
• A very simple application framework for spitting out static images and HTML. (This was the lightbulb-went-on thing for me, as a blog that does not have comments can be totally static)
• A modified version of an existing lightweight blogging engine to fit his needs.
• Storing blog content in flatfiles on the system. Again, without dynamic content why bother with the overhead of storing posts in a database (says the guy using WordPress, one of the favourite targets of criticism by PHP developers who like their code clean and modular)
• The use of a markup language to reduce the actual HTML he had to write
• The use of a CSS framework to again reduce the need to write actual CSS
• A sprinkling of Javascript to add unique features to each page

I've read that blog post at least a dozen times now, and still marvel at how frickin' obvious it was to do things the way he did. It's also obvious that he put a lot of thought into what he needed and the best way to couple things together to make it happen.

Compare his methods to my current set of decisions as to the various tools to build a web front end for a corporate draft

• A glue framework with lots of components that can be legitimately accused of starting to become bloated
• Use of an ORM to spare me from writing SQL to pull in dynamic sports data
• Table-driven HTML layouts from the previous generation of the application

When you start to factor in the time to actually learn not only the framework itself (and also heavily relying on a particular module of it to give me the MVC features I wanted) but the ORM itself, I wonder if it was the right decision.

Because if I were to break it down in a similar fashion to how the Peepcode blog was built, I'd say I need the following

• lightweight scripting language front-end (in this case PHP because that's what all our other client-facing applications use)
• a component to handle dynamic routing of requests, although I could really use web server rewrite rules instead of making the front-end code figure it out. My mod_rewrite skills have gotten a little better in the last year or so.
• An RDBMs because the source data is in a database already, and so few NoSQL solutions allow you to do the kind of grouping and aggregate math functions I need to do that MySQL can handle for me.
• An authentication component. I've written enough of these to know it's easier to use someone else's that just works
• An access control component. I've written enough of these to know it's easier to use someone else's that just works too.
• Grid-based CSS framework for layouts so we can use as few tables as possible.
• An easy-to-use Javascript library to take care of everything else that I cannot do in the scripting language or via CSS.

Man, I'm starting to think that Rasmus was right with apologies to my friend Kevin Beckford. Kevin is of the opinion that main reason to use a framework to build something is because unless you will be the only one ever to write and maintain that application, the rules and structure that a framework gives are essential to help your fellow coders stay on track. It's like having a common language that you both speak, and easy access to a dictionary (the documentation for that framework).

In my case, I am using PHP + Zend_Framework + Zend_Application + Zend_Auth + Zend_Acl + Doctrine + MySQL (plus Python on the server to do some data munging) to build this thing out. Luckily, no Javascript needed at this point. So what could I strip out to build it the way I put it above? We could end up with PHP + Apache rewrite rules + Zend_Auth + Zend_Acl + MySQL + Python, with Blueprint CSS thrown in to help me with the layouts. Not a *whole* lot of difference from before.

But of course, I was able to quickly build out admin functionality for the front end due to the ability to map out relationships between tables and having Doctrine pull in all that data and return it to me in a ready-to-go package for easy display. Also, to be realistic, we are too far into the project to rip all that stuff out and quickly refactor it. I mean, how do I justify stripping out Doctrine and writing all the "query, process and stick into variables" code now? It's a delicate dance, and it's obvious those decisions need to be made before you start, not when you're experiencing some mild Buyer's Remorse.

So, in the end, why did Geoffrey Grosenbach make me feel stupid? It was because he made me realized that I have a tendency to what to overengineer things a bit, wrapping them in framework solutions when perhaps the more direct route is to couple existing modules together to have a more flexible solution.

But I have at least considered that the tools I was using were going to make it easier to build out the behind-the-scenes stuff. Sure, I could write all the SQL code myself (I've finally understood the power of joins), but once I've learned how Doctrine does things it becomes super simple to build on the existing work.

Look at the Peepcode blog setup shows that he was using a bunch of tools already in widespread use in the Ruby and Rails communities. There are not many equivalences for that stuff in the PHP world (HAML and SASS only have incomplete implementations for use with PHP as far as I can tell) but the approach he used is the one to really think about. I mean, what would this blog look like if I decided to go the minimalist route, but with tools that I am more comfortable with? Could I do it with Werkzeug + a simple blog engine + Textile + Blueprint CSS + jQuery? I have no spare time as it is, what would this type of project do to me?

Rallyhat is for the most part a programming experiment for me, designed to learn Python *and* produce an web application that is actually usable by someone other than me. One of the technologies I'm using is CouchDB to store things. I was having some problems getting both couchdb-python working with permanent views (meaning views stored in CouchDB itself) as opposed to temporary views (meaning views generated in the code).

Now, couchdb-python looks to be a nice, simple interface to use. The problem I found was the documentation is, what's the word I'm looking for? Lacking. Since Jan is an amazing teacher, he was patient with me and showed me that I could look at the documentation for couchdb-python via the Python REPL (aka the Python shell). So, armed with this "how the hell could I not realize this" knowledge I figured out how to use it. Yet another reason why I'm warming up to Python.

It was as simple as starting up the Python shell and then doing the following:

In there you will find code examples for each method, and then explanations of the parameters for each method. All you really need in this case. So I take back what I said about the documentation for couchdb-python being lacking. I was just looking in the wrong place.

So, task #1 was to create the view I needed. I decided to use Futon, the built-in interface to create my permanent view:

This view would then live at http://couchdb.server//rallyhat/_design/searches/_view/by_user is you want to get every doc for every user. To get just the docs belong to a specific user, you append ?key= to the URL above. Is that easy to understand or what? As an aside, I am finding the online version of CouchDB: The Definitive Guide to be an awesome reference guide (once you remember to actually use it, thanks again Jan). With the view then created, the next step was to create the Python code to read in those views.

With *any* Python library that you use, make sure to see if the author added help files to the package. You'd be surprised to see what's in there...

So as we start 2010, and you're a web developer, what is out there for you to play with? Let's see...

• Mature web application frameworks for the Big 3 open source scripting languages (PHP, Python and Ruby) that can help you, when used properly, rapidly build applications, if you are willing to spend the time understanding the limitations of frameworks. You might hate frameworks, but you certainly cannot ignore them. Yes, you could consider WordPress itself to be a framework
• Multiple languages that will run on the JVM that can be used for web development, if you are willing to spend the time understanding the JVM and how to get these things talking to Java as well. Ignore the Java haters, I think this is a good trend.
• Multiple actually-usuable-outside-of-small-projects alternatives to RDBMs, if you are willing to spend the time to figure out when you should use them. The trend is towards having ridiculous amounts of data, and it appears that in the open source world the RDBMs is not up to the task. Think Cassandra, Redis and Hadoop along with Pig.
• multiple mature Javascript libraries that make creating all that awesome Ajax goodness easy, if you are willing to spend the time to pick one with the features you like
• multiple mature (or soon-to-be mature) functional programming languages that can be used to build web applications, if you are willing to spend the time to learn functional programming. I mean, really, how can you not look at Clojure and Erlang and not be impressed?

But again, who cares what I think? If you do care, then you are at the same place as me: paralysis by analyses. What framework should I pick? Do I go document database or key-value store? If I don't want to learn Java, can I do some stuff with Scala or see if Jython is up to the task? Can jQuery do long-polling / comet stuff? Is functional programming too weird for a web monkey like me to figure out?

Crap, that's a lot of stuff to worry about, isn't it? Is it any wonder I get stressed out trying to figure out how to build stuff at work? Or what stuff to dabble with in my spare time? We are suffering from an embarrassment of riches, which is being hidden by the constant religious wars that developers seem to get embroiled in. If it's not "my language is better than your language", it's "my editor is better than your editor". "My programming paradigm is better than your programming paradigm." "If you did interesting work, you'd get accepted to speak at conferences more". (Sorry, that was me arguing with myself).

If anything, I have found that my focus is starting to change. I've started realizing that what is really happening is that despite there being ridiculous amounts of choices of tools to solve problems, it has become EASIER to solve these problems. In my planning for a work project I stumbled across the now-infamous Rasmus Lerdorf "The no-framework PHP MVC framework" blog post. While Rasmus is a programming god, he is also a very smart guy. He understands that programming is about solving problems, and he (at least to me) seems to not have the ego that says you must build up your own complicated solutions to problems.

This has gotten me to consider how I had been approaching problems. Too often, I have been approaching them from the code out to the display. Worrying about the technology, instead of how people will use it. Rasmus proposes an easy problem - a sample Ajax app that uses exactly 4 things: a scripting language, a Javascript library, a data storage source, and a data exchange format. In his specific example he used (in order) PHP 5, Yahoo YUI, sqlite and JSON. You could've done this with Python, JQuery, MySQL and XML. Or Ruby, Dojo, CouchDB, and JSON. My point is that we often find ourselves caught up in what we are USING to build something rather than solving the problem.

I have really dislike the idea of New Year's resolutions, because they are almost always broken. But in this case, I think a few are warranted: this year I resolve to do a better job of figuring out the problem I need to solve first and worry about the technology needed second. Sometimes my desire to feed the part of me that wants the new and shiny takes control and creates a potentially bad solution. Sure, it would be awesome to get paid to learn a new programming language and a new web application framework to go with it, but if it gets in the way of solving the problem, well, you've suddenly become the problem instead of the solution. Sounds so cliched, doesn't it?

So, in the spirit of that, let's look at 3 problems I need to solve in the next little while that I perhaps can also use to feed the Imp of the New and Shiny:

• Analysis of some MySQL slow-query logs are needed at work. First solution was to play with a tools specifically for analyzing the slow query log. They provided summaries that were simply not useful to me. Perhaps Pig + Hadoop will give me the high-level solution to collect the information we seek.
• I have a very simple idea for collecting absurd stories you've told to your kids. My first instinct was to create the project using a web application framework. Upon reflection, perhaps the combination of WordPress and then a custom-written story submission and approval system is the better approach.
• Plans are being made to create the next platform our company will use for online fantasy games. Since we cover multiple sports, I have no desire to create a separate version of the platform for every sport. Instead, inspired by the realization that Rails is nothing more than a DSL (or Domain Specific Language) on top of Ruby, I have started work on a very simple DSL for defining scoring rules. Wouldn't it be awesome for the games admin to be able to select a stat category for a sport (say, rushing yards in football) and then use "(1 point each 10) + (5 at 100 total)" as the scoring rule? Parsers are actually pretty easy as long as your DSL is consistent.

I hope I've made sense here, as I'm trying to show how I wish to change my approach to solving programming problems. There is no doubt that I will pick up the new and shiny as I find a way to solve these problems. Some might be written in PHP. Some will be written in Python. Some will be written in ScoreRule (to pull a name for my sports DSL out of my ass). But the real winner is the people who will use these things. Shhh, don't tell anyone: I'm thinking about the end user for once instead of the programmer.

Concurrency is coming, better be ready

The Continuation Mondad in Clojure. Even if you hate Lisp (I'm looking at you, Kevin) or don't give a damn about Clojure it is worth reading about to understand how you can write concurrent applications.

Real-time web applications are coming, better be ready

I have a new project I am going to be starting at work, one that can potentially involve real-time information. Check out this series on how to build real-time web applications using Django, Python and a javascript library called Orbited. When you combine this with the you-can-see-it-on-the-horizon-but-not-quite-here-yet concept of web sockets, I can see some of the concepts of what I called Web Pi (3.14159, get it? Get it? Um okay, any way) starting to take shape.

TDD With PHP is already here, where have you been?

Giorgio Sironi has gone and written the book I was thiking of writing this winter. If you want to learn more about doing TDD with PHP, go and grab a copy of his e-book. It's 100% free, but I liked it so much (and admired him for tackling a topic I have interest in) I donated $10 to him. You should do the same.

Chris should not post links on Twitter that cannot be discussed in 140 characters or less

Finally, on Twitter I posted about this blog post which talked about how it's only people who are financially secure who pound away at the "only do what you love" message. When you don't have to work, that's easy to do. I got into a long conversation with Amy Hoy about this topic, although Twitter is a crappy medium for doing this sort of thing.

I respect Amy a great deal, so I was kind of bummed when she and I appeared to disagree on some the key points of the blog posting. Basically, "do what you love" is a lie because it is not always possible. "Do what you do, with love" is a better approach to thing. Not everyone gets to work on exactly what they want to, so perhaps I am showing some signs of neo-puritanism or something when I said "well, even if you have a crummy job you should be doing it as best as you can." I proceeded to be smacked around by several people who do not share that philosophy. Amy seemed to feel that the blog article was (to use her words) bullshit and was advocating "eating a shit sandwich and liking it" by giving people justification for "hiding behind the mortgage" as an excuse for not taking chances. It will no doubt surprise Amy that I happen to agree with her, but utterly failed to get that across on twitter. My apologies to Amy for that. We've all worked crummy jobs, and it has been my personal experience that the best way to get out of a crummy job is to work as hard as you can to build up your skills so you are ready to leap on better opportunities when they come around.

Both Ruby and Python offer a Read-eval-print loop that can be run from the command line to allow people to do the opposite of what conventional testing tools do: allow you to run interactive tests of your code. I used the REPL available in Python (you know, just typing 'python' from my command line) to test out how to use various geocoding libraries for a side project. My favourite Ruby programmer (who is offering career consulting that I might find too enticing to pass up even at $100/month) uses irb all the time to work on stuff.

So in a way, a REPL is the anti-automated test. I think that if there was a good REPL available for PHP, testing best-practices might take better hold. Perhaps there is hope for a good PHP REPL in Alan Pinstein's iphp project. I remember trying to continue work on the testing console I had created a number of years ago for CakePHP, and quickly realized that what I needed was a REPL.

Since I did not have the desire to write my own limted PHP parser by messing around with eval (although, to be honest I did try playing with it and realized I did not know what the hell I was doing), the testing console was dead. Besides, how many PHP devs really use PHP on the command line? 10%? 1%? Hard to know, as the group I follow is a self-selecting one and I'm sure most of them have used PHP on the command line. Hard to become an advanced PHP developer without trying to do stuff on the command line with it.

I was also pleased to find out (after a year of using it, no less) that eXist had it's own REPL for XQuery - they call it the "sandbox". It's awesome because I can take my "wrote this 6 months ago and can't remember if it still works" XQuery scripts and test them out. Sort of like a command line client to your RDBMS. It even tells me when I've got syntax or formatting errors in them BEFORE I try and execute them. Talk about saving a developer time.

Do things like this exist for RDBMs like MySQL and Postgres? Imagine an interactive client that is checking your syntax out while you type it. If such a thing exists, please tell me!

Once again, I see I have started to diverge. Back on track again.

So, from a testing perspective it's easy to see how shells and sandboxes can become essential tools. For example, all the good advice I have ever seen about unit testing clearly advocates a sandbox approach. You create fixtures containing data. You create mock objects to simulate functionality that might potentially be destructive. You might even create a separate database for doing test operations on. Clearly, the goal here is to create an environment that is the same as the one your code will eventually be running in, but separate from the "real" one.

A REPL, on the other hand, is a very powerful and very dangerous. If you're not careful, you could very easily insert data into your current "working" environment instead of a "test" one. As with so many other things, power comes with a price. Sure, I can run some code to figure out how to use, say, SQLAlchemy but I better make sure that I am not writing data to my production database. That would be silly.

I'm guessing the main reason PHP does not have a easy-to-use REPL is because it grew up on the web first and the command line second. Ruby and Python started on the command line first, and out to the web second. It's really that simple I think. Will PHP ever get a widely-accepted REPL?