@TheKeyboard » Django

Coding Lessons Learned

Chris Hartjes — Thu, 22 Jul 2010 00:51:39 +0000

In discussions with my friend Kevin, I have come to realize that we have arrived at the same conclusions about the use of web application frameworks. Much of these conclusions have driven from 12+ years in building applications using them, with various rates of success. Suffice it to say, only the most l33t programmers out there are using their own frameworks, be it a custom one or contributing to the creation of an open-sourced framework. Everyone else is like me: a user of frameworks, and extremely rare contributor back to those frameworks.

Since Kevin and I are travelling together through the world of snake handling, we have been looking at the available options for building web applications in that environment. Our conclusions have been surprising to ourselves, but I think they can be very instructive to others. These two lessons are applicable to other programming languages as well, so feel free to substitute your favourite language where applicable.

Lesson 1: Full-stack gets you in the door, lightweight lets you find your niche

If you want to learn Python for the web, the first place you go is Django. It is a full-stack solution for building things in Python. Awesome documentation, and with some help from Python’s online documentation, you can build an app in a reasonable time frame without knowing a ton of Python. Awesome tutorials and code samples FTW! I have built a Django app and briefly made it public. It was, like so many other things, a piece of crap missing functionality so I am trying to rectify that in my ever-dwindling spare time. I did not find any serious obstacles to building this app, except for figuring out how to do some dynamic form-field-generating code. It took me a while to find the right info (amazing how having the correct keywords in your search helps) but I did.

As an old warrior of frameworks (10 and counting across 3 languages) I found that because I already understood the concepts behind MVC (or MVT in the case of Django) I could concentrate on learning Python first, and then the unique features of the framework after that.

In many ways, my early experiments with Rails (back in 2004-2005) helped me understand frameworks a lot better. All frameworks have their own magic methods for doing things, with Rails making heavy use of Ruby’s ability to create Domain Specific Languages to make Rails the incredibly useful framework that it is today. But eventually, your skill with the language and the framework gets to the point where you see the limitations of the conventions and magic methods the framework uses.

Now that I’ve gotten my hooks into building stuff with Django, I now recognize the same seductive promises that I found in things like CakePHP: the full-stack takes you away from actually figuring out how to code things, and instead you find yourself just using all the built-in methods. Don’t get me wrong, the built-in stuff for Django is awesome. But if I want to push my skills forward so I can say “I can build stuff for the web using Python” instead of “I can build stuff for the web using Django” I need to use other tools. What if I want to run my stuff on Google App Engine? Sure, there are these hacks available for Django that make it “usable” on GAE, but that’s not what I want. And what about the use of WSGI instead of relying on mod_python? Forgive me if this stuff comes across as technobabble. It was technobabble to me at one time too.

So now the next step is (after finishing the current Django app) is to build an application using a much more lightweight framework. Why? Because it’s time to actually learn how to do something instead of relying on magic methods. Because once I learn to write code to duplicate those magic methods, I’ll be able to use Django *and* Flask. And that, my friends, is how you make yourself useful. That and the ability to run the app both as a standalone WSGI app or on Google App Engine seems like a bonus to me.

So now that you’ve gone from full-stack to lightweight, it’s time to look at the next lesson…

Lesson 2: Don’t use anything made by One Guy

This will undoubtably be a touchy subject for some. Let me try and explain it.

In the quest for the lightweight solution, I ran across web2py. Nice and lightweight, awesome documentation, but it’s all driven by ONE GUY. Experience has taught me that projects driven by ONE GUY end up looking like how the ONE GUY would do it. Sometimes the ONE GUY is brilliant and it all works out. That is rare, in my experience. Open source projects are littered with the wreckage caused by egos and control freaks who were unwilling to accept outside help to solve problems.

So look for projects where it appears there are solid contributors beyond ONE GUY. web2py sure has an impressive list of people who contribute, but is Massimo the ONE GUY when it comes to actually making non-trivial code changes? I sure hope not.

There will be no test afterwards

While I do not expect other people’s experiences to be the same as mine, I’m pretty sure I am not the only person who has learned these lessons. So the next time you are evaluating using a web application framework (in a new language or an existing one) think back on these lessons.

Expand your programming mind: filters in Lithium

Chris Hartjes — Tue, 16 Mar 2010 15:27:17 +0000

Out of the ashes of a severe personality clash in an open source PHP web application framework project rose a new framework proclaiming that it simply sucks less than all the others. Lithium, otherwise known as li3, is the latest PHP application framework to come onto the scene. I know what you’re saying. *Yawn*.

As far as I can tell, Lithium is trying it’s hardest to be fast, lightweight, and use all the features that PHP 5.3 has to offer. I am unable to use it at work for a variety of reasons (not stable, required PHP 5.3 only, Nate is always picking on me via IM) but I am more interested in the ideas that are coming out of it.

It is using all the newest hawtness in the PHP world: namespaces, closures, Phar, late static binding. I could go on, but probably most of these things mean nothing to you until you actually need them. In the framework world, namespaces are a big deal, an essential tool to avoiding collisions with 3rd party code you might want to integrate into your project. Closures are very familiar to those who’ve done work with Ruby or Javascript. Phar allows for some unarchiving on the fly of code, even potentially distributing your application as an archive that can be run. Late static binding is something I’m trying to comes to grips with, as I have either never come across a need for it or never realized that I did and have implemented a workaround for it not realizing there is a simpler solution.

None of these are what I want to talk about. I want to talk about something that I feel is a very underrated part of Lithium, the ability to define filters. This is a concept that you find in Aspect-oriented programming, and one that I imagine most programmers have never considered. Here’s why I think it’s a big deal.

Most frameworks are designed around an OOP paradigm. Yes, some mix them together (I’m looking at *you* CodeIgniter), but by and large you get the job done by extending on a base model / controller / helper / whatever to create new functionality. Sometimes this is not avoidable. But what AOP says is that there is another way, a way that Lithium has adopted as well.

Hence the creation of “filters”. As far as I can tell (and I’m sure I will be jumped upon if I am wrong) the purpose of filters in Lithium is to allow you to add functionality without extending the class itself. When I saw that I immediately understood why that is so awesome.

The idea of callbacks is also very similar: if X happens, run code Y. A staple of jQuery, Rails and probably other projects I have forgotten to mention. The idea is still the same though. In Garrett Woodworth’s presentation about Lithium to the Orange County PHP Users Group he goes over some solid examples of filters including:

automatic use of the XHProf for code profiling
automatic setting of parameters upon saving a pasted bit of code

It’s also been used to automatically change string record ID’s into a MongoDB ID and graceful failing of an integration test suite when an error occurs.

These filters remind me of behaviors in Rails (and in CakePHP as well) and makes me wonder if signals are the same thing in the Django world. All in all, this looks to me to be a better way to extend functionality than actually overriding existing methods or hacking at something deep inside a class. Figure out what functionality you want, figure out where it needs to be called, and add your filter.

Rallyhat lives!

Chris Hartjes — Fri, 12 Mar 2010 22:00:58 +0000

Some of my long-time readers have seen me mention a long-festering (I believe that is the correct term at this point) project that I dubbed Rallyhat. It first started out as a “figure out when some baseball teams are in town and buy tickets” and then morphed into “a tool for helping people plan road trips by tagging destinations with information”.

What it really became was a tool for me to learn how to write a solid web-app using Python and Django. I’m happy to say I finally have a version that I feel comfortable sharing with the rest of you. So feel free to check it out. Pound on it, try and break it and I will also read your feedback and comments. I’m sure you can figure out my email address…

Also, I’ve found it has been easy to pick up not only Python and Django itself. I’m ready to tackle some other applications using Django, and I have one just in mind.

Yes, I am aware at how spartan it looks right now. I’m more focussed on functionality than how it looks right now. I’ve already gotten some interesting ideas on expanding the app to include adding in images to go with locations, so we’ll see how that goes.

For those who are curious, I built it using the following:

Python 2.6
Django 1.2 beta 1
CouchDB to store the trip plans
couchdb-python
GeoPy
django-registration
Google Maps services
Django Debug Toolbar
deployed using Nginx + Gunicorn, configured with the help of Eric Florenzano’s awesome Django Advent article about deploying Django using FastCGI because it mentioned how you could use Gunicorn

What Thrashed Around In Chris’ Brain This Past Weekend

Chris Hartjes — Tue, 24 Nov 2009 02:12:49 +0000

(The blog experiment continues: 1 post every weekday in November and I think I only skipped one, but made up for it with a Saturday post)

Node.js is something I want to use, but I do not know what for
For those who don’t know, Node is a “toolkit for writing extremely high performance non-blocking event driven network servers in JavaScript”. In other words, it’s for writing server-side Javascript with an emphasis on providing concurrency. Damn you Tim Bray, I cannot avoid seeing concurrency everywhere since you started your Concur.next series on your blog. It fascinates me, and I don’t even do any kind of work where I need to worry about concurrency. Not yet anyway.

Simon Wilson (someone I follow from the Django community) has a great article explaining why Node.js is genuinely exciting to him, and my old CakePHP landsmann Felix shows how to handling streaming file uploads in node.js. Like so many new technologies, Node.js starts out as a solution looking for a problem but as I become more familiar with concurrency it all becomes a little clearer.

While pgAdmin (and my favourite trick of creating an SSH tunnel to the server the database is on). Well, I'd had enough of that. I also wanted to set things up so when my boss wanted changes done, I could tell him "go do it yourself using this neato-keen admin interface I whipped together".

So I got to work creating a Django project, installed the django-admin application and got to work.

Have I mentioned how I *love* code generation these days? Can I get "hell yeah!" from the lazy programmers out there? Django comes with this awesome tool that you access via manage.py called inspectdb. What it does is introspect the database you configured for your Django application and generates the output for your models.py file. However, keep in mind the following:

You *will* have to fiddle with the order your classes that represent the models appear in the generated code

You *will* have to go in and alter any field definitions where inspectdb was not able to determine the proper field type and instead guessed.

So, after firing off inspectdb, I then issued a 'python manage.py syncdb' to create all the tables needed for django-admin, created a user for the admin site...and then it got hairy.

The documentation for customizing django-admin is good. And I was able to create just about everything I wanted by actually, you know, READING the documentation. But I found a problem that I was not able to solve. I even asked my favourite consultants for help but I could not get it to work the way I *wanted* it to work. When I told my buddy via IM that it would be easier to whip up this admin in PHP than slog through learning how to do it the Django way, he was not impressed. He stopped short of saying "you're dead to me now", but I got that vibe.

Here's the problem. I have these two models:

PLAIN TEXT

PYTHON:

class Permissions(models.Model):

league_key = models.CharField(max_length=15)

commissioner_id = models.ForeignKey(Accounts, db_column='commissioner_id',

to_field='id')

status = models.IntegerField()

key = models.CharField(max_length=50)

publisher_key = models.CharField(max_length=25)

publisher_display_name = models.CharField(max_length=50)

base_league = models.IntegerField()

share = models.IntegerField()

default_fixture_key = models.CharField(max_length=50, blank=True)

def __unicode__(self):

return self.publisher_key + ' / ' + self.league_key

class Meta:

db_table = u'permissions'

verbose_name = 'Permissions'

verbose_name_plural = 'Permissions'

class PermissionsAdmin(admin.ModelAdmin):

list_display = ('league_key', 'publisher_key', 'commissioner_id', 'status',

'share', 'default_fixture_key')

list_display_links = ('league_key',)

exclude = ('base_league',)

Now, if you look at the definition in there for the base_league field, you will notice that I gave up and called it an IntegerField. But in reality, it is pointing to another record in the same Permissions table. We have a concept in Sportso of a "shared league model" where you can base a league on another league and import all sorts of properties and data from it. I was hoping that I could do something like this:

PLAIN TEXT

PYTHON:

base_league = models.ForeignKey('self', db_column='base_league', to_field='id')

Whenever I did that, I would actually crash Python on my MacBook. Not a good sign. If I tried swapping them, I would get an error complaining about violating permissions_pkey, which is the Postgres sequence that was created for the auto-incremented ID for the table. I did manage to take a look at the SQL it was trying to execute, and it keeps trying to update the 'id' field (which is the primary key) instead of the 'base_league" field. So I gave up and will continue building out the other parts of the admin. I just wish I could figure out WHY this self-referencing foreign key setup was not working.

If you know what's going on here, let me know via the comments.

Chris’ Brain – Almost October 2009 Edition

Chris Hartjes — Thu, 24 Sep 2009 20:56:37 +0000

Can you tell I have nothing earth-shattering to talk about? Here are the two things taken up processing space in my brain

I won't use frameworks without built-in testing support any more

I'm doing some very hairy refactoring of code at work to support a new concept of ownership of content generated by the application. If I had been honest with myself and started the project with TDD practices, I'd be in much better shape. While the framework that it was built with is all the things it talks about, it is a non-trivial task to add a testing harness to it. I'm trying to setup this environment but I've run into a problem with being too stupid to figure out the changes to my Apache .htaccess file to make it work (UPDATE: Thanks to @spooons for helping me out to get it working).

So you can take your lightweight frameworks and chuck them out the window if I have to waste more than 15 minutes of my time figuring out how to write tests for it. TDD is an essential tool when you get to the maintenance phase of a project.

Django Applications are doing things right

If you use Django, you can find lots of third-party applications that you can literally drop into place into your existing one and start using it. Can you imagine doing this with your favourite PHP framework? Being ignorant, I have no idea how easy it would be to create something like Django-Socialauth for Zend Framework, where you can drop it into your application and it comes with templates and models and controllers for handling all these things. I was astounded by both the foresight that went into Django that allows people, as long as they follow specific conventions, to create standalone applications that you can use in pretty much any Django application. As long as you haven't hacked Django itself too much.

Digging Through The Bookmarks

Chris Hartjes — Fri, 08 May 2009 21:01:29 +0000

Although I never got into the habit of using online bookmarking services like Delicious (never really understood how/why to use them) I do tend to bookmark a lot of stuff. So as a change of pace I thought I'd go through my programming bookmarks and show you some of the stuff I found interesting enough to save:

Limonade, a PHP microframework

I am Mr. Framework (Rails, ZF, CakePHP, Code Igniter, Django, I've tried a ton of 'em) so I found this project to be very interesting. I think it's only really missing easy-to-do URL rewriting. I could *probably* figure it out using whatever the web server for that project uses, but I am spoiled by the easy routing stuff in most frameworks.

jQuery and Google Maps

Marc Grabanski is a guy I met at the original CakeFest in Orlando, and he does so much amazing front-end work that I find myself going back to his site over and over again. This is the second tutorial he has up there on using jQuery with Google Maps. So well written, and so easy to implement with your own stuff.

The Testability Explorer Blog

Misko Hevery has this amazing blog where he talks about great testing practices. For the love Murphy, please go read his blog and start testing your code. I mean it!

The startup immune system

Great article on some recommended practices for making your company's handling of defects in the software they produce better.

Sharpen the Vim Saw

What? You don't use Vim?!?! That's okay, what Ivan Idris talks about here is finding inefficiencies in the way you use your editor-of-choice and eliminate them.

Understanding Git Conceptually

Want to know what the big deal about Git is? Why are all the cool kids using it? Well, many of *them* don't even know why Git is so awesome. This site explains HOW Git works so you can understand WHY you should be using it.

Django admin for your PHP app?

A quick guide to how to sneak Django into your PHP project as the rock-solid start of an admin area. I have done this with a project at work at the prototype level (no code committed yet) and it just makes so much sense.

Quick DB Setups with MySQL Sandbox

Although I am lucky that I do not need to test my work code against different versions of MySQL, the techniques described here show how to do that exactly that in a way that is so simple I thought to myself "Why did nobody ever tell me you could do this?!?"

Library of Free Data Models

This site is awesome in a very subtle way. It shows you all sorts of samples of building relationships between things in a data model. Want an example of how to link agents and leads together for a recruiting company? They have one. Ever wanted to see how a hotel would handle booking reservations? They got one.

Okay, last one. I promise

Rope, a python refactoring library

I'm starting to get into the world of refactoring tools as I dig into Python. Rope is a tool to let you go through your Python code and identify things that need to be refactored, like duplicate code blocks that can be combined into one. I so wish tools like this existed for PHP, as I would use them all the time. Add in the fact that it can be integrated into Vim, well, I've got a tool that will become an essential part of my Python toolkit going forward.

So there you have it. What hidden gems do you have to share?

Reader Feedback: Experiences With Django

Chris Hartjes — Tue, 26 Aug 2008 21:18:41 +0000

Welcome to Day 2 of my answering reader feedback questions here in the month of August. I was asked to share my experiences with Django. I will confess that I am a complete n00b when it comes to Python (wrote a few scripts for work-related stuff) and got an early alpha of my baseball road-trip planning site actually working using Django. Sure, it was ugly and very rough around the edges but I got it to work. But I will say this: if I had the opportunity to start working with Python and Django every day (and for the same pay I'm getting now) I would take it in a heartbeat. Seriously.

So why all the love for Django and Python? As a guy who's used PHP for a very long time, I'm really blind to the syntax of it. All the griping about needle vs. haystack and "PHP is ugly" is just wasted on me. Just so much pissing and moaning by people who should know better and lack the courage to learn the language they way they should. I heard it described by Evan from Identi.ca as being "the working man's language". Nothing wrong with that, as PHP is great at Getting Things Done. But Python...Python is vastly different from PHP. It's about objects. It's about consistency. It's about the most amazing interactive interpreter I've seen since I was a kid fooling around on my VIC-20 typing in programs from Compute! magazine. Bet you didn't think I'd been using computers for that long. To shift into bragging mode, I first got paid for programming when I wrote a program for my mother's school (she's a teacher) to create the salary grid for the teachers. I was 12. I think.

Also, there's a dirty little secret about using frameworks that nobody really tells you. After a while, it makes you lazy. You forget how to do certain things because, well, the framework does it for you. The other day there was a tweet from someone asking about the best way to sanitize data before you put it into the database. My first thought was "sheesh, that's built into the framework" and then I stopped myself and said "you are getting goddamn lazy!". My response was "try using mysql_real_escape_string(...)" but was also told about using PDO's binding of parameters as another way to do it. See, those things are all done by the framework meaning lazy old Chris doesn't have to remember how to do it. Until the time he's confronted with non-framework code that he cannot simply rewrite as part of a framework. *gulp*. So, since Python and Django are so different from the other frameworks I've used it forces me to STOP being lazy and actually learn how not only Django itself works but how the insides of it work as well.

Now, being an old hand at frameworks means using Django is not that big a stretch. I spent a lot time googling around for the "Python (or Django) Way" to do things. Except for a few concepts that are not familiar to a long time PHP guy (generators?) it wasn't that hard to get up and running with Django. The documentation is awesome, and once you learn to stop writing PHP code in Python and instead embrace doing things the Python way, it becomes a lot easier.

In the end it's still datasource-backed models talking to some controller code and passing it onto some template. I find the Django templates to be very limiting after my initial experiences and my "PHP is the template" background, and I think that as I get deeper into it I really have to find out how to do things the Django way. What CakePHP calls Model-View-Controller, Django calls Model-View-Template, but it's really the same thing.

When I compare Ruby to Python, I don't get the feeling that I will have to resort to dynamic functions and closures to Get Things Done in Django like I see happening in Rails. Besides, they seem to be two very different communities. That's probably because the leaders of the Django community are so different in temperament and I don't see the same type of "Django rules, everything else sucks!" mind sets that I saw in the Rails community. Maybe I'm just looking in the wrong places, but I feel more at ease in the Django community than the Rails community, albeit from the outer fringes of it.

So don't be lazy and complacent about the tools that you use. Go and checkout Django and you might feel the same way about it as I do.

What’s In Chris’ Brain – May 2008 Edition

Chris Hartjes — Fri, 02 May 2008 20:35:43 +0000

Yes, it's the day after May Day (workers of the world, unite!) and yes, I only managed to get one blog post done this week. So, it's time for a round-up of the stuff I've been thinking about this week.

For all the bleating I did on the CakePHP mailing list this week about using free and open software, the editor waffling continues and I was back to Komodo today. I needed the security blanket of an IDE today after spending way too much time with CSSEdit trying to get stuff lining up properly for a work project.

Halfway through a code review project as a side gig. Old code, running on PHP 4.4.8 (those of you following me on Twitter saw me moaning about getting that working with CentOS on Parallels). Needless to say, the code is a mess but I have to actually talk about WHY it's a mess and what can be done.

Trying to see if I can get a caching plugin for WordPress to work with nginx. Not that I ever expect to get Dugg, but you never know. Almost there...

Finally managed to get a Django app running properly inside the Google App Engine SDK. Work on that got torpedoed this week by two slow-pitch games...that got rained out...after me waiting around for 20 minutes or so. Next step is to build out my web service for Rallyhat. Should only be a few hours work (he said, not knowing now what he will know later)

YASP (Yet Another Side Project) is going to get rolling this weekend. Some cool stuff using Amazon S3, integration with OpenID and PayPal. This time I actually have a plan to make some money.

I've submitted my "Deployment Isn't A 4 Letter Word" talk for ZendCon.

Been updating the IBL web site to use the latest CakePHP goodies, meaning just refactoring a lot of stuff that was garbage and using newer techniques I discover via the mailing list and Nate Abele's rantings.

Special thanks to Travis Cline for sending me a copy of "xUnit Test Patterns" from my Amazon Wish list! Many thanks to a fellow budding Djangonaut for the gift.

Building Rallyhat: Proof-of-concept Deployment

Chris Hartjes — Sun, 13 Apr 2008 01:39:49 +0000

Well, that went relatively well. As a proof-of-concept for Google App Engine I decided to see what I could whip together today in between domestic duties, looking after the kids and praying that my wife gets better from her massive cold because I'm off to Vancouver tomorrow.

So, I pushed the database for Rallyhat up into the cloud and wrote a super-simple front end just to see if I could do it. You can take a quick peek at http://chrishartjes.appspot.com. To get it to spit something out, you need 3 parameters:

home_team -- nickname for a baseball team (i.e. Yankees, Red Sox, Padres). Don't forget to urlencode the spaces with a +

begin_date -- date in YYYY-MM-DD format

end_date -- date in YYYY-MM-DD format

It's a small start, but a start nonetheless. Next step is to configure my app to use the dev version of Django, not the built in version.