2 Jan
Just Build It, Damnit!
There was a posting on the CakePHP mailing list today from someone asking for sources of advice on “starting big applications”. Having gone through this at my previous job, I posted the following response:
Just to inject some sanity into what should be a very interesting
thread, I urge anyone building an application to simply BUILD IT and
worry about the profiling / benchmarking for AFTER you’ve built it.
Trust me when I say this: no application of any significant size
survives long enough to be scaled past it’s original intent without
wholesale architectural changes. You *WILL* be rewriting things, you
*WILL* be rethinking how your application is organized. Like Death
and taxes, it is inevitable. But you can worry about it later.I say this as someone who has been through the “let’s build something
massive” process, only to see it fail horribly due to all sorts of
preconceptions at design time that resulted in major rewrites that no
benchmarking would’ve helped. We didn’t know what the bottlenecks
were until we actually saw them get hit, and then we set about
rewriting the bottlenecks. A little slipshod for sure, but there were
many things that happened that we could not have anticipated when we
were building it.If your site is so busy that you need to do benchmarking and other
refactoring work, then you have accomplished more than 99% of other
websites out there and deserve a round of applause. Until then, Just
Build Something, Damnit and worry about fixing any performance issues
later.Before everyone jumps on me, please understand that I am saying this
out of experience. Yes, there are certain things you can do in terms
of building a cache-friendly app, or configuring a web server for
optimal use but those are so application-specific I hesitate to
recommend those things until you actually know what you need. Premature
optimization lies on the path to developer madness.You won’t know what challenges your application will face until people
other than you are actually using it.
Hope that helps.
My new mantra for programming in 2007 is Just Build It, Damnit! I’ve been procrastinating too much on several personal projects so it’s time to put my money where my mouth is.
Posted by Travis Swicegood on 02.01.07 at 3:31 pm
“Just Build It, Damnit!” - I like it! It’s so easy to get caught up in the minutiae of code and making it fast that you loose site of the app it’s a part of. I do it all the time…
Posted by Paul M. Jones on 02.01.07 at 3:31 pm
Chris, you are exactly right. Building and benching in iterations is reasonable and realistic.
Having said that, when you get to point where you **do** need to benchmark in order to improve performance, you will need to know the limits of the framework. No matter how much optimizing you do in the application and database code, you won’t be able to exceed the framework’s base responsiveness, unless you change the framework foundations in some way.
Posted by Chris Hartjes on 02.01.07 at 3:31 pm
Hey Paul, I think your concept (sent to me in an email) of doing releases in iterations and benchmarking your results is probably the best method to approaching bookmarking.
Posted by ThinkingPHP and beyond » Cake 1.2’s Set class eats arrays for breakfast! on 02.01.07 at 3:31 pm
[...] “But doesn’t it cost more performance to loop through the array twice in the Set example?” I hear some of you cry. Yes it does. And? Have you built your application yet? Does it implement all features you are dreaming of? And most importantly: Do your web stats indicate you are going to have 1 million hits / day soon? If so go back into your code and remove the Set example with the less succinct foreach alternative. If not, listen to Chris Hartjes who’s motto for 2007 is Just Build It, Damnit!. [...]
Posted by huz on 02.01.07 at 3:31 pm
cool, early optimization is an evil…
Posted by Nils Hitze on 02.01.07 at 3:31 pm
Give this man a cigarre .. he earns it.
You are 100% totally right.
And if someone writes a new myspace.com out of the box he is either crazy
or has to much time, or both, or he has a developer TEAM behin him.
Posted by Chris Hartjes on 02.01.07 at 3:31 pm
@nils: You know, great applications can come from one lone, crazy developer who has an idea and builds something. But that’s the key: it got built, got used, got rewritten, and maybe morphed into something totally different. Flickr started out as an idea for an online game. Ruby on Rails started out as an application for doing online chatting. You never know where the fickle finger of fate wil push you when you start to build something.
I’m as guilty as the next egomaniacal programmer in that I start a lot of projects and don’t finish them (like rallyhat.com *cough*) but just getting something finished and usable is a major accomplishment.
Besides, everyone knows the next MySpace is Facebook…
Posted by z on 02.01.07 at 3:31 pm
Spoken like a true asshole. If you have been down this road many times, then you would know what types of obstacles you are likely going to encounter. Giving an appropriate amount of effort to a well thought out architechture can save you an incredible amount of time. Also, once you have fully developed a project plan you, anybody can write the code and get it done much more rapidly. Young developers cannot wait to just write code. That is why you find yourself always having to go back and “tune” things. 90% of that is not “tuning”, its fixing crap that you knowingly did halfassed in the first place that have finally bitten you in the ass. Learn from your mistakes! “Just do it” is a recipe for failure. If you cannot devote 20% of your total project budget to initial architechure and planning, you are seriously misguided, and are welcoming the long and painful life of total application rewrites. Enjoy! Wait, what am I saying! I’m sick of cleaning up after you people!
Posted by Chris Hartjes on 02.01.07 at 3:31 pm
Hi “Z”…well, lets just say it takes one to know one!
In seriousness, I think you’re totally missing the point here. Sometimes, there are problems that no amount of up-front planning will fix. Sometimes, you have to actually just build the damn thing in order to find out.
You throw the number 20% out there like it’s some sort of magic bullet. “If only I had spent 20% of my time planning, this would’ve worked!” Don’t make me laugh.
Experience is the best teacher, and I highly doubt that every project you have ever worked one where you did all sorts of planning up front for worked out exactly the way it was supposed to. It never does, and never will. I’m telling people that you are far better off actually getting something built then spending endless amounts of time tweaking for performance levels or problems you haven’t faced or might not ever face.
Mistakes happen since code is written be people, not machines. Even the best architecture documents can be misinterpreted or just flat-out wrong for solving the particular problem.
I’ve never done a total application rewrite, and I never plan on doing one. Just not worth the time and effort required. You’re sick of cleaning up after “you people”? Buddy, I’m sick of listening to people like you claim that everything would’ve been fine if you had just planned for problems that you didn’t even know existed. If you can see the future, why aren’t you playing the lottery?
Posted by z on 02.01.07 at 3:31 pm
20% is not a magic bullet number. It is at least some amount of planning. So many times I have worked with people or had to work on code written by others were no up front planning was done at all. You are correct in that you are not going to get it all 100% correct the first time around. However, most applications employ business logic that has intertwined dependancies. The just build it approach always leads to dealing with those dependancies in a hackish way. The expense being that it is terribly difficult to correct further down the road than if one had simply taken the time to understand them in the first place and develop a solution appropriately. Time is a killer when it comes to application development and getting things to market fast while they still are relevant or can compete. I think I may have been a bit too tongue-in-cheek in my first post. The main point I was trying to make is that “just build it, damnit” implies “don’t plan, just build!” which i strongly disagree with. “Don’t optimize early, just plan as best you can, then build” is a statement I can get behind. Everything won’t be just fine if you do up front planning, but it will be a hell of a lot better and you’ll look back thinking “I’m glad we thought this out at least a little bit”, not “what the hell were we thinking?!”.
Posted by Chris Hartjes on 02.01.07 at 3:31 pm
@z
I think we’re on the same page here now. I definitely agree with “Don’t optimize early, just plan as best you can, then build” as a sound philosophy for building an application. You do have to have a plan, or else nothing will actually get done.
Posted by Maxim on 02.01.07 at 3:31 pm
I read your article from http://cakebaker.42dh.com/2007/04/01/three-myths-about-testing/ and sure you’re right, especially when developping with a framework!
P.S : quite out of topic but could there be a link for feed subscription through netvibes?
Posted by jean-Rémy Duboc on 02.01.07 at 3:31 pm
“Launch it now” -> Quotes of the day : Launch now and php arrays
Posted by The dot com dream... « Rudimentary Art of Programming & Development on 02.01.07 at 3:31 pm
[...] http://www.littlehart.net/atthekeyboard/2007/01/02/just-build-it-damnit/ (funny, and straight to the point) [...]
Posted by Yahoo! Mash layout editor :: {one true brace on 02.01.07 at 3:31 pm
[...] ti?p cái editor này. ??n sáng hôm nay thì t? t?m hoàn t?t version 0.1. Codename Just Build It, Damnit! Link: [...]
Posted by A on 02.01.07 at 3:31 pm
Is not prevention better than cure?
Is not it better to find out the problems/ bugs early in the development stage?
Hope someday web people will understand how a quality software should be built. Pity on you guys!
Posted by Chris Hartjes on 02.01.07 at 3:31 pm
*sigh*
Right, because only desktop developers understand how to build a “quality software”.
Posted by A on 02.01.07 at 3:31 pm
Not actually.. there are also smart web people out there. Who think twice and do it once. But it is true most of the (startup+php) kind of people are software engineering illiterate and only think testing is the only way to assure software quality! how stupid!
Posted by A on 02.01.07 at 3:31 pm
BTW, I don’t do desktop
Posted by Chris Hartjes on 02.01.07 at 3:31 pm
@A
Some of the biggest and most successful web sites were started up by “software engineering illiterate” people, who were smart enough to bring the right people in AFTER things got hairy.
Sorry to lump you in with the desktop developers, although that was not my intent. My point was that it’s okay to Just Build Something, but have a plan in place on what to do when things go wrong. Worrying about what might happen instead of just getting it done and out there is the mindset I want to fight against.
Posted by A on 02.01.07 at 3:31 pm
Google was not started up by “software engineering illiterate” people! Neither facebook nor youtube! But I agree few successful websites were started by such people. But it was not the quality of the app that made it successful. Rather it was the idea of the app that sold. And hence later on, right people were chosen to maintain the quality of those apps.
I think you personally understand the necessity of short-term planning and designing in every short development cycle and you do understand ‘agile development’ is an offspring of iterative/spiral development process . IMHO, It is very important to follow a process to improve quality + maintainability + managibility.
We should understand agility and never encourage cowboy coding! People misunderstand a lot from these type of articles.
Ofcourse performance of an application is an issue but in most of the (web-apps) cases it is not that kind of a MAJOR one - in most (web-apps) cases we can IGNORE this (performance thing) in early development cycles (forget optimization) and KEEP focus on other qualities.
A quality app is that app which can be maintained, improved and manged easily. They are coded in a way so that less bugs can be entered, and if entered they can be hunted easily. Code is so well organized that we often mistakenly think it is an art but in fact building quality app (+ manageable, maintainable quality code) is a pure science (in fact engineering).
Sorry Chris, I did not get your point first. Infact you are right. Again sorry if i have offended anyone.
Posted by Bill on 02.01.07 at 3:31 pm
Right on!