I'm using:

1. Zend Framework 1.10.5
2. ...and Zend_Application
3. ...and building functionality out using modules

So off I go duck hunting and find lots of samples on how to do it. Except none of them fit how my application is currently built. Plugins? Front controllers? Placing code in preDispatch? Yikes, this will take some time to sift through all the available info and come up with a solution custom-tailored to my application's structure.

I ended up using info from here and here to form my solution. The first step was to create a plugin to hold all the info about my ACL's and roles.

Then, I needed a way to actually call this plugin and do something with the results. I was using a base controller for the 'football' module for my application, and after messing around, I figured out that the code interacting with my ACL plugin needed to go in the init() method of the controller:

This used to be a lot more crufty, but I slowly whittled away at the code until it has what I think is the simplest implementation.

Oh, I almost forgot. Here are the tests I wrote for it. I bet you thought I forgot to write tests, didn't you.

For a project at work, I am using PHP (specifically Zend Framework) for the front-end but are using Python scripts run as a cron-job (and also on-demand when statistical corrections occur) to collect raw stats for a variety of sports, and then generate fantasy point totals for the games we run. I'm already using PHPUnit for tests of the front end, and I decided to to use py.test to test my Python scripts.

Setting up tests in Python was pretty simple. Here's one of my test scripts:

Very similar to tests with PHPUnit, right? So now that I had both PHPUnit tests and py.tests tests (hrm, is there are better way to say that?) to run, I had to figure out how to automatically run them. More specifically, how to get our installation of Hudson to run them.

Getting PHPUnit to play nice with Hudson was relatively easy. I installed the NUnit plugin for Hudson, made sure I installed phpunit, and then I added it's use to my build scripts. However, the strength of Hudson is that with the use of another plugin I could read reports of all those tests. So when things failed, I would not have to look at the console output to figure things out. There's a place in the Hudson config where you can configure this:


Now, I figured that the same thing could be done with py.test. It had an option so that at run-time you could tell it where to put JUnit-compatible test result files. After a little tinkering, I got it to work. First step was adding execution of it to my build script. Here is the latest-and-greatest version of that script:

mkdir /var/www/games-hudson/${BUILD_ID}
cd ${WORKSPACE}/games
/usr/local/zend/bin/php doctrine-cli migrate
cd ${WORKSPACE}/games/tests
/usr/local/zend/bin/phpunit --log-junit=${WORKSPACE}/build/logs/phpunit-results.xml
cd ${WORKSPACE}/games/scripts
/usr/bin/py.test --junitxml=${WORKSPACE}/build/logs/pytest-xmlrunner.xml
cp -R /var/lib/hudson/jobs/${JOB_NAME}/workspace/games/* /var/www/games-hudson/${BUILD_ID}
chmod 777 /var/www/games-hudson/${BUILD_ID}/tmp
rm -rf /var/www/games-hudson/current
ln -sf /var/www/games-hudson/${BUILD_ID} /var/www/games-hudson/current

Next, I then told Huson where it could find the JUnit-compatible files generated by py.test:

So there you have it. Now, when I do a commit and trigger a Hudson build, both my PHPUnit and Python tests get run. And there is output to check, so I don't have to dig through console output to figure things out.

PHPUnit

Since Kevin and I are travelling together through the world of snake handling, we have been looking at the available options for building web applications in that environment. Our conclusions have been surprising to ourselves, but I think they can be very instructive to others. These two lessons are applicable to other programming languages as well, so feel free to substitute your favourite language where applicable.

Lesson 1: Full-stack gets you in the door, lightweight lets you find your niche

If you want to learn Python for the web, the first place you go is Django. It is a full-stack solution for building things in Python. Awesome documentation, and with some help from Python's online documentation, you can build an app in a reasonable time frame without knowing a ton of Python. Awesome tutorials and code samples FTW! I have built a Django app and briefly made it public. It was, like so many other things, a piece of crap missing functionality so I am trying to rectify that in my ever-dwindling spare time. I did not find any serious obstacles to building this app, except for figuring out how to do some dynamic form-field-generating code. It took me a while to find the right info (amazing how having the correct keywords in your search helps) but I did.

As an old warrior of frameworks (10 and counting across 3 languages) I found that because I already understood the concepts behind MVC (or MVT in the case of Django) I could concentrate on learning Python first, and then the unique features of the framework after that.

In many ways, my early experiments with Rails (back in 2004-2005) helped me understand frameworks a lot better. All frameworks have their own magic methods for doing things, with Rails making heavy use of Ruby's ability to create Domain Specific Languages to make Rails the incredibly useful framework that it is today. But eventually, your skill with the language and the framework gets to the point where you see the limitations of the conventions and magic methods the framework uses.

Now that I've gotten my hooks into building stuff with Django, I now recognize the same seductive promises that I found in things like CakePHP: the full-stack takes you away from actually figuring out how to code things, and instead you find yourself just using all the built-in methods. Don't get me wrong, the built-in stuff for Django is awesome. But if I want to push my skills forward so I can say "I can build stuff for the web using Python" instead of "I can build stuff for the web using Django" I need to use other tools. What if I want to run my stuff on Google App Engine? Sure, there are these hacks available for Django that make it "usable" on GAE, but that's not what I want. And what about the use of WSGI instead of relying on mod_python? Forgive me if this stuff comes across as technobabble. It was technobabble to me at one time too.

So now the next step is (after finishing the current Django app) is to build an application using a much more lightweight framework. Why? Because it's time to actually learn how to do something instead of relying on magic methods. Because once I learn to write code to duplicate those magic methods, I'll be able to use Django *and* Flask. And that, my friends, is how you make yourself useful. That and the ability to run the app both as a standalone WSGI app or on Google App Engine seems like a bonus to me.

So now that you've gone from full-stack to lightweight, it's time to look at the next lesson...

Lesson 2: Don't use anything made by One Guy

This will undoubtably be a touchy subject for some. Let me try and explain it.

In the quest for the lightweight solution, I ran across web2py. Nice and lightweight, awesome documentation, but it's all driven by ONE GUY. Experience has taught me that projects driven by ONE GUY end up looking like how the ONE GUY would do it. Sometimes the ONE GUY is brilliant and it all works out. That is rare, in my experience. Open source projects are littered with the wreckage caused by egos and control freaks who were unwilling to accept outside help to solve problems.

So look for projects where it appears there are solid contributors beyond ONE GUY. web2py sure has an impressive list of people who contribute, but is Massimo the ONE GUY when it comes to actually making non-trivial code changes? I sure hope not.

There will be no test afterwards

While I do not expect other people's experiences to be the same as mine, I'm pretty sure I am not the only person who has learned these lessons. So the next time you are evaluating using a web application framework (in a new language or an existing one) think back on these lessons.

After I got over being insulted that a programmer was not required for every tiny tweak to the site (I'm kidding) I tried to figure out how I was going to accomplish this. I'm not a big fan of storing actual HTML content in a database. Call me old-fashioned, but I feel my HTML content belongs on the filesystem, and nowhere else. Besides, it makes it harder to do deployments of code on multiple servers if all the templates aren't there to begin with.

The solution was, of course, obvious: create a custom Zend_View and override stuff until you get it behaving the way you want! The logic for doing this is actually quite simple:

1. Check to see if the script in question exists in the file system
2. If not, figure out what template we need to pull from the database.
3. Read in contents of that script from the database
4. Write contents to the filesystem

NOTE: I am not sure if this is the most elegant solution for solving this particular problem. I am open to a different solution so long as it meets the same needs.

Since I've made a commitment to doing TDD for this project, naturally I started out with a test:

For those wondering, yes all the tests pass.

The "Template" is a Doctrine (1.2) model that represents the table where I'm storing information about the templates. Yes, it has unit tests too. That pass.

So how do we accomplish this? First, I hit up my peeps on Twitter and the advice from Matthew Weier-O'Phinney himself was to simply create my own Zend_View and override what needed to be overriden. Also, he was very wise to tell me to look at Zend/View.php itself for guidance. That and some helpful messages from exceptions during testing.

Here's the initial implementation, for which all tests pass.

So, what did I learn from this process?

1. I needed to create my own private _file and _filter variables to match what exists in so that if I create any custom filters they can be applied to my custom view *and* to keep compatibility with existing code.
2. Because the _run method is protected AND abstract, I needed to implement my own _run method in my custom view, or else it would spit out errors. Even if it doesn't do anything special.

You can see that the logic in my render method is pretty much as I outlined before. I realize I will take a slight hit the first time we go to load a particular template. I'm assuming at some point I will want to run a test to see just how long it does take to pull in the template from the database and write it to the file system. I'm guessing this will not be a serious performance hit, but you never know. All those file_exists() calls can't be that good for performance,

I look forward to seeing other potential solutions to this problem

Zend Framework

When I read a book about a programming framework, I'm expecting to see that they start off with the basics of using the framework, then keep expanding on the abilities of the framework. By the end of the book, you should be able to build something OTHER than the example application (if they provide one). I mean, if I just want a list of functions then I will hit up Google/Bing/Duck Duck Go to find the info I want. This book starts with the basics of the framework, and patiently builds on the concepts you previously learned. Good, clear examples of how to do things.

I thought another nice touch was the expanding into concepts like scaling the application, and extending the framework itself. If there is one thing I've learned building web apps over the years, it's that you (a) inevitably need to worry about scaling the application and (b) you end up having to make your own modifications to the framework itself to meet unique challenges for the application.

All in all, I recommend this book to anyone who is planning on using CodeIgniter 1.7 (and beyond) to build web apps. The sections on the first steps towards scaling your app, and extending CodeIgniter itself make this book stand out from the crowd.

As a lapsed tester, I've made the commitment to build out our Zend Framework powered application using tests going forward. The first two modules for the app *should* have been done with tests, and for that I hang my head with shame. Now that I've invested the work in creating a continuous integration environment using Hudson, there is no longer an excuse to not write tests. Especially when I can get tests run automatically every time I commit code.

Anyway, I was asked to implement a new feature for the application: a list of all transactions for a fantasy baseball league, sorted by date. What a perfect excuse to write some tests! The initial problem though was how do I simulate logging in a user so I can see this page, which was protected by authentication using Zend_Auth. So I started my scouring the internet for answers.

I found quite a few examples on how to test a controller, using the $this->dispatch('/path/to/action') method but found that I wasn't getting redirected properly to the post-login page. I was passing proper credentials in and everything. Then finally after deciding to go back to basics and read up on just how to test Zend_Auth. After some swearing a few face-palms, I realized what I had been doing wrong.

The tl;dr version: I was forgetting that the unit test itself needed to manually log in the user by speaking with Zend_Auth directly. Here's the testing code for that particular controller:

I'm using a custom authentication adaptor with Zend_Auth, as we need to limit access for a user to a particular league. Perhaps better to do it with Zend_Acl, and since the platform is likely to expand I will end up using that as well. But I am drifting off topic here.

The magic is in that loginUser() method. What I did not realize at the time was that running a test where I asked it to dispatch users to the login page and passed along what would've been entered in the form was not working. Perhaps it was creating an authenticated session OUTSIDE the scope of the test environment. Perhaps it is my sucky code. Either way, it wasn't working.

So, by creating a authenticated login Zend_Auth *inside* the test, I could then safely dispatch my testing code to look at pages requiring authentication. I hope this blog posts helps out others trying to write tests for controllers that are hidden behind Zend_Auth.

Taking a quick peek at my bookcase, these are a few of the books that I consider essential reference guides for me:

• The Pragmatic Programmer
• Practices of an Agile Developer
• Mastering Regular Expressions
• Beautiful Code / Beautiful Architecture
• Javascript: The Good Parts

I have more books, but those are the ones closest to my desk.

There is now a new book that has earned a spot on my bookcase, and should be in one of the slots closest to my desk. I am reading a borrowed copy of it, but I plan on rectifying that soon. "Building Scalable Web Sites" by Cal Henderson (he of Flickr and "Why I Hate Django fame) is that book.

I don't know how I can describe how awesome this book is in this blog. Just check out the topics in the table of contents:

1. Introduction
2. Web Application Architecture
3. Development Environment
4. i18n, L10n, and Unicode
5. Data Integrity and Security
6. Email
7. Remote Services
8. Bottlenecks
9. Scaling Web Applications
10. Statistics, Monitoring, and Alerting
11. APIs

Pardon my vulgarity, but that is pretty much every single fucking thing you need to worry about when building a web site that is going to be used by anyone other than you and your close circle of friends.

This book is old in internet time: published in 2006, meaning it was probably written in 2004-2005. A lot of technology has changed, but the principles themselves have not. Also, a lot of the code inside uses PHP for it's examples, but I really think this applies to any programming language that is stateless and/or promotes a "shared nothing" architecture. That means (as far as I'm concerned) PHP, Python and Ruby. It's very simple: scaling anything is difficult, but scaling something where there is information that needs to be shared between nodes is extremely difficult. The best way to use this book is to ignore the specific technologies being mentioned and focus on the ideas and practices that are being promoted. I mean, let's be brutally honest: Cal Henderson helped build one of the most massively scaled web sites, using a programming language at it's core that the hipster programmer crowd sneers at. Ignore his advice at your peril.

Go buy this book. Right now. You will not regret it.

A programming language is only as good as the community that supports. By supporting it, I mean willing to get together several times a year for conferences and bust their asses to have people share interesting talks with everyone else. Look at Python and Ruby. Awesome languages to work with. But even more awesome are the conferences that serve as anchors for the community.

There seems to be lots of little regional Ruby conferences, which give the language itself a unique flavour in return. Of course, it helps that Ruby can be used quite easily outside the traditional web application building environment. I don't see so many regional Python conferences, just PyCon and DjangoCon both in the US and over in Europe. It could be that I'm not looking hard enough.

As for PHP, to be fair, it shows up at pretty much any open source web-related conference. OS Bridge. OSCON. Codemash. I could go on, but I am not in the conference-promotion business. I mean, why is there no Great Canadian PHP Conference? I know there was phpworks in Toronto back in the mid 2000's. It was my first conference I ever went to. But I'll tell you why there is no Great Canadian PHP Conference: there is no PHP community in Toronto.

Oh sure, there are tons of PHP jobs to be had in Toronto. Lots of code monkey positions to be filled. But the Toronto PHP User's group has been dormant since last year. I sent an email to the organizer telling him I wanted to help out at least 3 days ago. Haven't heard a damn thing back. Perhaps it is time to go around such a person and just do it myself.

See, without the PHP community I do not have a career. Once I took the plunge, submitted a talk, and got accepted (I gave a talk about what the PHP community could learn from Ruby on Rails) I found I had suddenly joined the PHP *community*. Instantly I had people asking me questions AND people willing to answer my questions. Like I was their long-lost buddy. That was an awesome feeling, let me tell you. But even more important were the people I met.

As a result of the community, I have found myself in the awesome position that every time I needed a job, it took less than a week to get one. It was a combination of my blogging and my work in making those connections in the community that put me in this position, but it is sure a nice feeling to be able to convince your nervous wife that we were not about to face a rough patch while I struggled to find work. So it's a no-brainer for me to continue to submit talks to conferences and try to attend even when I don't speak. It's about giving something back to the community that has helped me grow my career.

So don't just *use* your language of choice. Try and become part of that community. Doesn't matter if it's Python, Ruby, or good-old-solved-the-web-problem-first PHP. You'd be surprised what doors are opened up by your participation.

See, I normally only go to conferences when I speak. Why? Being away from the family is tough to arrange when you work from home. I've got two little girls, and they are used to Daddy being around to help out with stuff. Like taking them to day care, or picking them up from school. Making dinner. Helping with school lunches. Grocery shopping. I'm sure you get the picture. So to abandon the family to go to a conference requires lots of logistical work.

But damnit, I hadn't been to a conference since March of last year (the old PHP Quebec conference) I was determined not to get shut out. Luckily my boss agreed to foot the bill and so off I go. I'd love to meet any of my readers, and even sign paper copies of my book if you got a print-on-demand version. Hell, I don't even have one. So don't be intimidated by my size, appearance, or lack of knowledge on certain topics. Come and say hi. I'll the big, bald, grumpy Canadian with a pot-scrubber on his chin stomping around the conference.

Anyway, I wanted to talk about the sessions I'm looking forward to seeing. In no particular order, here they are:

Building Real-Time Applications with XMPP

I've known Travis for a while through the PHP community, and never seen him do a presentation. However, his numerous IM sessions helping me understand decorators in the context of Django and talking me through fixing totally busted Git commits lead me to one conclusion: he knows his shit so I should pay attention to what he has to say. The building of our next-gen fantasy sports gaming platform starts when I get back so I'm hoping he imparts some pearls of wisdom.

Continuous Inspection and Integration of PHP Projects

Being the only guy working on projects gets frustrating: nobody to share ideas with and also very hard to make the case for the types of tools and procedures that the TDD/BDD and continuous integration crowd advocates. I mean, all this work so Chris stops complaining about bugs slipping through the QA cycle? Besides, I always have a soft-spot for Ze Germans who are always at the PHP conferences.

Graphs, Edges & Nodes: Untangling the Social Web

I first met the presenter (Joel Perras) at PHP Quebec last year through the CakePHP community. Now that he's moved on to the Lithium project, I see his ego has grown to the point where he is okay giving presentations. Don't worry whippersnapper, I'll cut you down to size during this (probably awesome because it deals with stats and nodes and graphs and all this cool stuff I've been reading in "Programming Collective Intelligence") talk.

Now, I'm sure that I will be attending other sessions while I'm there. But those are the main ones I'm looking forward to seeing. We are spoiled for choice at this conference, and I am definitely looking forward to seeing old friends and making some new ones.

"Hey Chris, can you set things up so that whenever someone runs one of those deployment scripts, we send an email to our support mailbox of all the files that were pushed up since the last deploy?" After pausing for a minute to figure out how I would do it (the quick answer was "no idea") I said "Sure, I'll see what I can do."

So I stood in front of my whiteboard and proceeded to sketch out an idea of how I would want to do such a thing. Luckily I could cheat and look at how Capistrano did it. Because the way that they do it makes so much sense, so I duplicated it quick-and-dirty style. A big thanks goes out to my friend Derek for pointing me in the right direction.

So, the way to make this happen is as follows:

1. Read in a file inside your current deployment that contains the date of your last deployment
2. Check your code out of your version control system into /releases/
3. Symlink /releases/ to /current
4. Write a file into /current that contains the date of the last deploy.
   

Here's what it looks like as a Phing script

The key was figuring out the best way to send SVN log information via email. Leave it to the UNIX way of piping output to provide with the best solution totally within Phing.