I think I can give you an executive summary of this blog post: if the documentation for an open source project sucks, nobody but the most hardcore developers will use it.
What has prompted this incredibly brilliant stating-of-the-obvious? The reaction by those in the Twisted community over Facebook’s pre-emptive strike in the non-blocking Python-based web server space by open sourcing a framework they are using internally called Tornado. Why are the Twisted folks getting all twisted (look how funny I am!) up about this? Like any good Open Source project for people who are interested in Getting Things Done, Tornado is dissing Twisted by basically saying “yeah, we thought about using Twisted but it wasn’t easy enough for us to use and we think it sucks”. I’m pretty sure I’m paraphrasing here, but you get the idea.
Now, I have looked at Twisted and have come to the following conclusions:
- It looks awesome
- It is built for people who are supposed to understand what they are doing
- The documentation sucks
- The examples on how to build anything of any significance are, to this developer, non-existant and/or difficult to find without shelling out for a possibly very-outdated book
Contrast this with Tornado. I’ve looked at Tornado and have come to the following conclusions:
- It looks awesome
- It is built for people who are supposed to understand what they are doing
- The documentation is inside the comments on the code which sucks only if you’re not into reading the code
- It includes a sample application so you can see something other than Hello World
Am I biased towards Tornado? You bet. Why? Because as hard as I tried, I could not find USEFUL documentation on how to build things using Twisted. Maybe I didn’t use the right search terms. Maybe I didn’t dig down enough links deep on the Twisted site. Maybe I didn’t fully understand the terminology or architecture of Twisted and how they different components speak to each other.
I’ll tell you this though: I am not having any such problems with Tornado. This is coming across as a criticism aimed towards Twisted, and I suppose that’s fair. I’ve noticed complaints from the Twisted community which really amount to whining. I understand that attitude totally. I’ve been there, when faced with competition that just might be doing a better job then I am doing. But complaining that Facebook should’ve simply fixed up Twisted instead of writing their own is not productive. Productive is understanding that Facebook looking at Twisted and said “this is not for us because *we* don’t find it easy enough to use” is a sign that something serious is wrong with Twisted’s documentation. You could have the most awesome non-blocking web server available, but if people with who WANT to use it can’t figure out HOW to use it, well, what can you say to that? Only the most dedicated open source contributors would’ve done that. Most will say “meh, I want to do this myself so I understand it” and then Twisted will die an obscure death while Tornado plunges forward gaining mindshare amongst people interested in the type of applications you can build with this technology. In my opinion, it’s all because of the lack of clear, freely available documentation for Twisted.
If there is an online source BETTER than the Twisted site, please share the link love in the comments so I can try and do a better comparison of the two at a documentation level. For someone like me, who is new to Python but old to the idea of web frameworks, I think that would be the fairest comparison.
Tags: documentation, Python, Tornado, Twisted
I completely agree! I was trying to use twisted.words the other day to build a jabber bot. I managed to get it working eventually but by scraping several examples together so I could figure out how to get everything set up. The documentation for twisted is quite crappy I have to agree.
Agree completely. This is exactly why the django framework is picking up so much steam: it’s documentation is second to none. Not only does it explain how to do things, it explains why they’ve taking certain design decisions and why you should too. And it’s constantly updated to keep it in-line with trunk, it really is fantastic.
Agree with usenetshack. I wanted to use twisted but was put off by the lack of examples and documentation. Django is completely the opposite, with plenty of tutorials, which is why it is now so popular and I am now using it.
Phillip.
It looks to me like we now have two projects for non-blocking network i/o without good documentation (unless you’re really counting code comments). I don’t see how this is an improvement.
Twisted is actually very easy to use too. I’ve taught it to teenagers before for making games with Flash interfaces, and I don’t even know it that well.
This is just pure NIH syndrome. It happened because it’s easier to write code than to read it, and it’s more fun to create than to fix.
@Cainus
I totally agree that there is definitely an element of NIH in this scenario. And it’s awesome that you are able to build stuff using Twisted, I just couldn’t find any documentation that I found useful enough to help me build something.
I have used twisted a couple of times, and I couldn’t agree more. Its damn near impossible to “Get Started” with twisted without ripping apart someone elses twisted code to see how they did it. I wasn’t able to find a single piece of useful up-to-date documentation and a lot of what I found was broken, outdated, or useless.
I built something I could’ve used twisted for, as the documentation seemed so vast, I didn’t know where to start..
My next experience was that the functionality I wanted was not in a seperate branch, so I gave up again.
Twisted should have example apps right on the front page – chat, ftp, http etc… they should all be fairly simple too
Bah: http://twistedmatrix.com/projects/core/documentation/examples/ http://twistedmatrix.com/trac/wiki/TwistedWeb
Both are vastly superior to the examples and documentation that Tornado is offering and easily googlable. (I’m not trying to say they’re perfect; I agree they’re not dropped in your lap as nicely as the django docs).
I agree. I was evaluating twisted words a long time ago for Jabber like the first poster, and really it was the most confusing project I’ve ever used. The issue for me was that I didn’t know whether the examples that were provided were recent or not, and everywhere I went there was a warning that this library wasn’t mature and shouldn’t be used in production (in reference to words), yet they acted like it could be and that reference was just outdated.
I have built several twisted apps which are in use in production environments, including web and jabber/xmpp apps. I love twisted, and use it whenever possible…
…and I agree completely with the op.
Twisted is powerful, flexible, well architected, and reliable. It is also damn near impossible to get over the steep learning curve and to actually USE.
Show me a competent Twisted developer, and I’ll show you someone who had a gun jammed roughly in their mouth, breaking several teeth, and was FORCED to learn it at some point. We are all glad we know it now, but that learning process SUCKED.
In addition to the comments, Tornado also has this documentation: http://www.tornadoweb.org/documentation
Hi Chris,
I think you’ve making a good point here. Almost by the definition of
popularity can we see that something is inaccessible to the majority of people,
it will not be popular.
And the people who are saying Twisted’s documentation is bad are on to
something too. It’s silly to argue with someone who’s saying that your
documentation doesn’t help them. They can’t really be wrong (short of simply
lying, of course, which I don’t think is the case here). However, it’s not a
black and white issue. To simply say “Twisted’s documentation is bad” is an
oversimplification. Casey’s comment above gives a good example of this, I
think. He started off by saying that he wanted to use Twisted Words, but
clearly his experience with the documentation wasn’t a good one. His
conclusion is that Twisted’s documentation is “quite crappy”, which an example
of one kind of this oversimplification: Twisted is a lot of things, and the
poor state of the Twisted Words documentation (and I agree, they’re some of the
worst in the project) doesn’t mean that the Twisted Web documentation is
equally bad.
I’m not just trying to split hairs here. There’s a practical consequence here.
If you look at the Twisted Words documentation and find them bad, don’t assume
the rest of Twisted’s documentation is bad! It’s easy to get fed up and go
away, and I can’t blame anyone for doing that, but giving up at this point is a
lost opportunity and it’s one based on a false extrapolation.
I’m not trying to get out of the responsibility for this either. It’s
absolutely true that if Twisted were uniformly well documented, this problem
wouldn’t come up. And we’re definitely working on that, we know there’s a
problem.
Aside from the good points being made here, there are a couple specific points
I’d like to address as well. First, Chris, you said that there are no examples
of how to “build anything of real significance”. I think that the “Finger
Tutorial” is a prime example of this. I wonder if you disagree, or if you
weren’t able to find it in the jumble of our website? Second, you mention that
Tornado’s documentation is all in the source code. This is a pretty minor
point that I won’t expect to convince anyone of anything with, but this doesn’t
actually differentiate Tornado from Twisted by all that much. While Twisted
has lots of (perhaps often poorly organized/written/etc) external
documentation, it also has lots and lots of documentation in docstrings, and
(to a much lesser extent) actual comments. ohloh reports 30% of Twisted’s
source lines as comments (which they include docstrings in):
.
Lastly, I’ll just reiterate that this isn’t an either/or story. As is so often
, but I think it’s going too far when people say
the case, the truth is somewhere in the middle. I’m more than willing to
entertain suggestions that Twisted’s documentation needs to be improved (it’s
easy – I already know this
that *all* of Twisted’s documentation is bad (unless they’ve actually read it
all, which few people making this complain have), or that it lacks examples (as
of right now, by *extremely* crude measurement, there are 147 runnable example
files included in the documentation). People who get Twisted and like it tend
to want to come out and “whine” when they see claims like these because they’re
just not true. And it being the internet (where it’s so easy to misunderstand
the tone or content of a post), it’s easy for things to polarize and people to
start taking absolute stances, and everyone goes away in disgust at the other
side. And the worst thing is, none of this misunderstanding helps anyone’s
software get better.
Ah, I lied, that wasn’t lastly – but this is: in response to Stu, I think
that’s a great comment. Some of us have been thinking about ways to improve
the front page of the Twisted site to make it easier for people to navigate to
the documentation (which may well exist! but that only matters if you find it)
they need to accomplish their goal. I’m not sure when these changes will
actually be made, but they’re coming.
@Cainus
Thanks for the links to the documentation. I had looked at those, and I think the big thing I obviously neglected to mention was how difficult I was finding it to find documentation on using Twisted.web to do something simple. The finger tutorial is clear, but I also feel that it seems to gloss over some things. Perhaps it is my n00bishness with Python, or limited experience writing servers that are not simply serving up content over the web that is the problem.
At one time I considered it a badge of honour to figure out how to use a tool with really bad documentation. As my mind atrophies with age as I approach 40, I’ve decided I don’t have the masochistic leanings to slog through that sort of stuff any more.
@ Jean-Paul
As I see it, there is a great BREADTH of documentation for Twisted but not a lot of DEPTH. Again, maybe a gross simplification but if you have to really hunt around to figure out how to do something then you can imagine why the documentation AND the tool itself gets labeled as sucking.
As someone who was involved in the CakePHP community quite a bit, I felt the same as the Twisted people: there is lots of documentation, you just have to dig around to find it. These days the documentation for CakePHP is quite good and it is now centralized. With a lot of hard work I can imagine Twisted getting there. See, that proves my point that Open Source is about DOCUMENTATION.
I am not knocking Twisted, as I think it is a very powerful technology and promotes ideas that more developers should be aware of. Tornado simply has focused on the web side of things, so perhaps the real comparison is not Twisted vs. Tornado but Twisted.web vs. Tornado. When you read stuff like “Twisted.web is being deprecated and Twisted.web2 is under development”, well, you can understand why people look at Tornado and say they would rather use it than Twisted.web.
Hi,
I agree with your statement completely.Tornado is not only a fast web server, it acts as a very lightweight application framework as well. Tornado is an open source version of the scalable, non-blocking web server and tools that power FriendFeed. The FriendFeed application is written using a web framework that looks a bit like web.py or Google’s webapp, but with additional tools and optimizations to take advantage of the underlying non-blocking infrastructure.
so, all in all, i think most people that “didn’t get twisted” didn’t understand the concept of Deffereds, async workflow or tried to use twisted as a webframework like Django.
I’ve written several irc bots using twisted, and some other smaller scripts and it was quite easy to use.
You just have to read through the short examples, which are really easy to understand.
What is not that easy is the web part of twisted, because there still needs to be a consistent solution.
And, sorry… the have “comments in the code”? I am sorry, but using docstrings, generating documentation isn’t that hard. I most probably won’t read the freaking source of the framework to understand how to use it! (and i’ve never done in twisted)
Yes, Twisted’s documentation sucks and it really kept me from using it and even trying again. It already starts sucking at the tutorial: the explanations of the code snippets basically reiterates what you did and nothing about why it is necessary, what the concepts behind it is or how it affects the behavior of the rest of the system. Point in case: Deferreds.
I didn’t know about Tornado before reading this post, so thank you!
I use Twisted at work, but I’m not a Twisted fanboy. I have to say IMHO Twisted’s docs are fine for me. But you need to actually read them because the approach is so different. Like READ them. Each and every paragraph. Even the prose. Other projects like Django you skip over 90% of the text cause its fluff and go to the example and thats fine. The docs for Twisted aren’t like that. You must pay attention and read carefully.
You said a whole lot of “Maybe I didn’t…”. I wouldn’t say maybe, I’d say you didn’t. Maybe I didn’t dig deep enough? For sure you didn’t dig deep enough.
Twisted is a whole lot more than just a webserver. That extra power comes with a steep learning curve. But to climb that mountain you have to exert a lot of effort. You have to actually pay attention and look into it in detail. But when you finally climb the mountain, the view is fantastic.
But also, I don’t know why the Twisted guys are so upset. So someone else has made a faster, lighter, single threaded webserver. Great! The more choices the merrier. I don’t see why there is even a problem here. The bickering is retarded. Someone can say that Twisted is complete and utter shit. And thats fine. People are entitled to their opinion. Those with more understanding will know it’s hyperbole.
Good point. Which brings me to Cassandra. Great software, documentation subpar (compared e.g. to something similar: MongoDB which has excellent documentation).
Cheers
Stephan
Same experience.
I even bought the twisted book and it’s not all that helpful in my opinion.
If the book had been: 1/3 lower-level networking basics, 1/3 architecture overview / design discussion, 1/3 serious twisted projects twisted would be in very widespread use.
As is the twisted book is a hurried tour of twisted usage for some simple apps, kind of like a bound volume of annotated hello-world for various twisted modules. It’s short, too.
It is a nice extra bit of info but pretty far away from what it’d take to turn you into a competent twisted developer.
It’s really too bad, too, b/c from what I can tell twisted is an amazing piece of work, it’s just well-nigh unapproachable.
> Why are the Twisted folks getting all twisted (look how funny I am!) up about this?
Actually, it’s not very funny because ‘getting all twisted up’ doesn’t pass for normal English; you want ‘getting all wound up’ – ‘wound up’ being the same as ‘twisted’, it’s even cleverer.
Hey again Chris,
Thanks for the response. I just need to correct one thing. You said “When you read stuff like “Twisted.web is being deprecated and Twisted.web2 is under development”, well, you can understand why people look at Tornado and say they would rather use it than Twisted.web.” but Twisted Web is not being deprecated. Can you tell me where you read this so I can correct it?
Thanks.
@cris
I think it IS obvious that I did not dig deep enough, but this raises an interesting question: how much work should you expect users of your tools do in order to figure out how to use it? Twisted appears to have a very high level of expectations for those who want to use it. I imagine I am not the only one who could not make the commitment to read every single piece of documentation in order to figure out how to build something with it. Laziness maybe, or perhaps just different expectations on what documentation should contain.
@Jean-Paul
I thought that is what this page said http://twistedmatrix.com/trac/wiki/TwistedWebPlan
After another read through, I realize what confused me: that you are taking stuff from twisted.web2 and rolling it back into twisted.web. This quote from there is very telling:
“This is mostly a problem of perception, but there are some real issues.”
I thought things were going the opposite direction, with twisted.web being phased out and rolled into twisted.web2.
Its weird because this bickering seems to be about twisted.web[2] and not twisted in general. The twisted framework touches so many more protocols than tornado – I’d like to see tornado switch all of its interfaces from https to xmlrpc or in one line of code like twisted can do. I’ve never used twisted for handling http requests but get lower in the stack and I doubt you’ll find a tool better for the job. If you are programming tcp you’re probably crazy enough to not need thorough documentation anyways.
Tornado is documented like a web server which is probably why tornado seems “better documented”. Apples and oranges.
[...] Open Source Is Really About Documentation Ja, aber ob man es so generalisieren kann? [...]
[...] http://www.littlehart.net/atthekeyboard/2009/09/14/open-source-is-really-about-documentation/?utm_s... a few seconds ago from Gwibber [...]
Hi!
Congratulations! Your readers have submitted and voted for your blog at The Daily Reviewer. We compiled an exclusive list of the Top 100 php Blogs, and we are glad to let you know that your blog was included! You can see it at http://thedailyreviewer.com/top/php
You can claim your Top 100 Blogs Award here : http://thedailyreviewer.com/pages/badges/php
P.S. This is a one-time notice to let you know your blog was included in one of our Top 100 Blog categories. You might get notices if you are listed in two or more categories.
P.P.S. If for some reason you want your blog removed from our list, just send an email to angelina@thedailyreviewer.com with the subject line “REMOVE” and the link to your blog in the body of the message.
Cheers!
Angelina Mizaki
Selection Committee President
The Daily Reviewer
http://thedailyreviewer.com