There was a posting on the CakePHP mailing list today from someone asking for sources of advice on “starting big applications”. Having gone through this at my previous job, I posted the following response:
Just to inject some sanity into what should be a very interesting
thread, I urge anyone building an application to simply BUILD IT and
worry about the profiling / benchmarking for AFTER you’ve built it.
Trust me when I say this: no application of any significant size
survives long enough to be scaled past it’s original intent without
wholesale architectural changes. You *WILL* be rewriting things, you
*WILL* be rethinking how your application is organized. Like Death
and taxes, it is inevitable. But you can worry about it later.I say this as someone who has been through the “let’s build something
massive” process, only to see it fail horribly due to all sorts of
preconceptions at design time that resulted in major rewrites that no
benchmarking would’ve helped. We didn’t know what the bottlenecks
were until we actually saw them get hit, and then we set about
rewriting the bottlenecks. A little slipshod for sure, but there were
many things that happened that we could not have anticipated when we
were building it.If your site is so busy that you need to do benchmarking and other
refactoring work, then you have accomplished more than 99% of other
websites out there and deserve a round of applause. Until then, Just
Build Something, Damnit and worry about fixing any performance issues
later.Before everyone jumps on me, please understand that I am saying this
out of experience. Yes, there are certain things you can do in terms
of building a cache-friendly app, or configuring a web server for
optimal use but those are so application-specific I hesitate to
recommend those things until you actually know what you need. Premature
optimization lies on the path to developer madness.You won’t know what challenges your application will face until people
other than you are actually using it.
Hope that helps.
My new mantra for programming in 2007 is Just Build It, Damnit! I’ve been procrastinating too much on several personal projects so it’s time to put my money where my mouth is.
“Just Build It, Damnit!” – I like it! It’s so easy to get caught up in the minutiae of code and making it fast that you loose site of the app it’s a part of. I do it all the time…
Chris, you are exactly right. Building and benching in iterations is reasonable and realistic.
Having said that, when you get to point where you **do** need to benchmark in order to improve performance, you will need to know the limits of the framework. No matter how much optimizing you do in the application and database code, you won’t be able to exceed the framework’s base responsiveness, unless you change the framework foundations in some way.
Hey Paul, I think your concept (sent to me in an email) of doing releases in iterations and benchmarking your results is probably the best method to approaching bookmarking.
[...] “But doesn’t it cost more performance to loop through the array twice in the Set example?” I hear some of you cry. Yes it does. And? Have you built your application yet? Does it implement all features you are dreaming of? And most importantly: Do your web stats indicate you are going to have 1 million hits / day soon? If so go back into your code and remove the Set example with the less succinct foreach alternative. If not, listen to Chris Hartjes who’s motto for 2007 is Just Build It, Damnit!. [...]
cool, early optimization is an evil…
Give this man a cigarre .. he earns it.
You are 100% totally right.
And if someone writes a new myspace.com out of the box he is either crazy
or has to much time, or both, or he has a developer TEAM behin him.
@nils: You know, great applications can come from one lone, crazy developer who has an idea and builds something. But that’s the key: it got built, got used, got rewritten, and maybe morphed into something totally different. Flickr started out as an idea for an online game. Ruby on Rails started out as an application for doing online chatting. You never know where the fickle finger of fate wil push you when you start to build something.
I’m as guilty as the next egomaniacal programmer in that I start a lot of projects and don’t finish them (like rallyhat.com *cough*) but just getting something finished and usable is a major accomplishment.
Besides, everyone knows the next MySpace is Facebook…
Spoken like a true asshole. If you have been down this road many times, then you would know what types of obstacles you are likely going to encounter. Giving an appropriate amount of effort to a well thought out architechture can save you an incredible amount of time. Also, once you have fully developed a project plan you, anybody can write the code and get it done much more rapidly. Young developers cannot wait to just write code. That is why you find yourself always having to go back and “tune” things. 90% of that is not “tuning”, its fixing crap that you knowingly did halfassed in the first place that have finally bitten you in the ass. Learn from your mistakes! “Just do it” is a recipe for failure. If you cannot devote 20% of your total project budget to initial architechure and planning, you are seriously misguided, and are welcoming the long and painful life of total application rewrites. Enjoy! Wait, what am I saying! I’m sick of cleaning up after you people!
Hi “Z”…well, lets just say it takes one to know one!
In seriousness, I think you’re totally missing the point here. Sometimes, there are problems that no amount of up-front planning will fix. Sometimes, you have to actually just build the damn thing in order to find out.
You throw the number 20% out there like it’s some sort of magic bullet. “If only I had spent 20% of my time planning, this would’ve worked!” Don’t make me laugh.
Experience is the best teacher, and I highly doubt that every project you have ever worked one where you did all sorts of planning up front for worked out exactly the way it was supposed to. It never does, and never will. I’m telling people that you are far better off actually getting something built then spending endless amounts of time tweaking for performance levels or problems you haven’t faced or might not ever face.
Mistakes happen since code is written be people, not machines. Even the best architecture documents can be misinterpreted or just flat-out wrong for solving the particular problem.
I’ve never done a total application rewrite, and I never plan on doing one. Just not worth the time and effort required. You’re sick of cleaning up after “you people”? Buddy, I’m sick of listening to people like you claim that everything would’ve been fine if you had just planned for problems that you didn’t even know existed. If you can see the future, why aren’t you playing the lottery?
20% is not a magic bullet number. It is at least some amount of planning. So many times I have worked with people or had to work on code written by others were no up front planning was done at all. You are correct in that you are not going to get it all 100% correct the first time around. However, most applications employ business logic that has intertwined dependancies. The just build it approach always leads to dealing with those dependancies in a hackish way. The expense being that it is terribly difficult to correct further down the road than if one had simply taken the time to understand them in the first place and develop a solution appropriately. Time is a killer when it comes to application development and getting things to market fast while they still are relevant or can compete. I think I may have been a bit too tongue-in-cheek in my first post. The main point I was trying to make is that “just build it, damnit” implies “don’t plan, just build!” which i strongly disagree with. “Don’t optimize early, just plan as best you can, then build” is a statement I can get behind. Everything won’t be just fine if you do up front planning, but it will be a hell of a lot better and you’ll look back thinking “I’m glad we thought this out at least a little bit”, not “what the hell were we thinking?!”.
@z
I think we’re on the same page here now. I definitely agree with “Don’t optimize early, just plan as best you can, then build” as a sound philosophy for building an application. You do have to have a plan, or else nothing will actually get done.
I read your article from http://cakebaker.42dh.com/2007/04/01/three-myths-about-testing/ and sure you’re right, especially when developping with a framework!
P.S : quite out of topic but could there be a link for feed subscription through netvibes?
“Launch it now” -> Quotes of the day : Launch now and php arrays
[...] http://www.littlehart.net/atthekeyboard/2007/01/02/just-build-it-damnit/ (funny, and straight to the point) [...]
[...] ti?p cái editor này. ??n sáng hôm nay thì t? t?m hoàn t?t version 0.1. Codename Just Build It, Damnit! Link: [...]
Is not prevention better than cure?
Is not it better to find out the problems/ bugs early in the development stage?
Hope someday web people will understand how a quality software should be built. Pity on you guys!
*sigh*
Right, because only desktop developers understand how to build a “quality software”.
Not actually.. there are also smart web people out there. Who think twice and do it once. But it is true most of the (startup+php) kind of people are software engineering illiterate and only think testing is the only way to assure software quality! how stupid!
BTW, I don’t do desktop
@A
Some of the biggest and most successful web sites were started up by “software engineering illiterate” people, who were smart enough to bring the right people in AFTER things got hairy.
Sorry to lump you in with the desktop developers, although that was not my intent. My point was that it’s okay to Just Build Something, but have a plan in place on what to do when things go wrong. Worrying about what might happen instead of just getting it done and out there is the mindset I want to fight against.
Google was not started up by “software engineering illiterate” people! Neither facebook nor youtube! But I agree few successful websites were started by such people. But it was not the quality of the app that made it successful. Rather it was the idea of the app that sold. And hence later on, right people were chosen to maintain the quality of those apps.
I think you personally understand the necessity of short-term planning and designing in every short development cycle and you do understand ‘agile development’ is an offspring of iterative/spiral development process . IMHO, It is very important to follow a process to improve quality + maintainability + managibility.
We should understand agility and never encourage cowboy coding! People misunderstand a lot from these type of articles.
Ofcourse performance of an application is an issue but in most of the (web-apps) cases it is not that kind of a MAJOR one – in most (web-apps) cases we can IGNORE this (performance thing) in early development cycles (forget optimization) and KEEP focus on other qualities.
A quality app is that app which can be maintained, improved and manged easily. They are coded in a way so that less bugs can be entered, and if entered they can be hunted easily. Code is so well organized that we often mistakenly think it is an art but in fact building quality app (+ manageable, maintainable quality code) is a pure science (in fact engineering).
Sorry Chris, I did not get your point first. Infact you are right. Again sorry if i have offended anyone.
Right on!
Very true. I think this applies easily to most things, not just software development.