To Build or To Buy?
Recently a friend sent me a link to an old post on Rick Copeland’s blog titled Three Reasons Why You Shouldn’t Write Your Own Web Framework. Although Rick talks specifically about open source web frameworks, his comments raised the more general question of how should we draw the line when deciding whether to build a new anything versus “buying” or using an existing project. The answer is not always clear, because it depends on a lot of factors.
Rick’s arguments against starting from scratch are primarily about avoiding wasting time, yours and other developers’. You end up wasting your time because you’re always playing catch-up, implementing features and fixing bugs that have already been handled by other tools, instead of building the features your customers wanted in the first place. You waste other developers’ time because you add to their burden of evaluating the solutions available in a larger pool of options. Although the cliché example for Python developers is the over-abundance of web frameworks, the situation with template languages is even worse.
So why do people start new template languages, web frameworks, ORMs, GUI toolkits, et cetera, instead of using and contributing to an existing project? Before open source became so prevalent, contributing to an existing project wasn’t really an option, so the economic considerations for building vs. buying were straightforward: Could you build the thing, either via contractors or full-time employees, for less than it would cost to buy it from a vendor, adapt it to your needs, and pay for the support? These days, open source eliminates a lot of the questions about up-front cost, but that doesn’t necessarily make the question any easier to answer.
For most open source projects, there isn’t anyone to pay for support and, as Titus Brown points out in his post It’s just too easy, “It’s less work to code a few hundred lines of Python than it is to understand someone else’s few hundred lines of Python”. That’s where we fool ourselves, though. Developers tend to have tunnel vision, seeing the easy parts of a project clearly and glossing over the tricky edge cases that are going to trip us up. Starting from scratch may only look like less work because we don’t see the hard parts.
On the other hand, sometimes starting from scratch can have benefits. For one thing, picking up in the middle of a project can be difficult, too. There’s a longstanding joke in the Zope community that its learning curve is “Z-shaped”. A few years ago I made the effort to learn its inner workings and was a serious Zope user. But when I put it down and stopped using it regularly, my skills became rusty very quickly. Like the saying goes, “it’s a great place to live but I wouldn’t want to visit there”. Maybe that says more about me than Zope, though. In any event, a smaller framework may do less, but can be easier to pick up and learn. Fewer features also mean less stuff to keep in your head once you become an experienced user.
“it’s a great place to live but I wouldn’t want to visit there”
An obvious reason to build your own is if none of the existing tools do what you want. When I built HappyDoc, back in 1998, I only started the project because the other similar doc-extraction tools had a limitation that made it impossible for me to use them: They all imported your code to find the classes, functions, and docstrings. Unfortunately, the code I wanted to document had import side-effects, so generating the documentation with tools that imported it required a special server configuration with resources that weren’t easy to replicate. The key feature of HappyDoc was that it worked by reading and parsing the source code, without doing a full “import”, so there were no side-effects. Since then, other tools have come along that work the same way, but at the time it was enough to differentiate HappyDoc from the other contenders. I ended up writing more than a few hundred lines of Python, though, so, re-examining my decision with the benefit of hindsight, I see that I might have been better off contributing to an existing project instead of starting a new one.
There’s still no easy way to answer the build-versus-buy question. I lean toward using open source projects when possible, but there are times when you’re just better off building exactly what you want than adapting your code to someone else’s way of thinking. How do you decide when to build and when to buy? Let me know what you think.