To Build or To Buy?

Recently a friend sent me a link to an old post on Rick Copeland’s
blog titled Three Reasons Why You Shouldn’t Write Your Own Web
Framework
. Although
Rick talks specifically about open source web frameworks, his
comments raised the more general question of how should we draw
the line when deciding whether to build a new anything versus
“buying” or using an existing project. The answer is not always
clear, because it depends on a lot of factors.

Rick’s arguments against starting from scratch are primarily about
avoiding wasting time, yours and other developers’. You end up
wasting your time because you’re always playing catch-up, implementing
features and fixing bugs that have already been handled by other
tools, instead of building the features your customers wanted in the
first place. You waste other developers’ time because you add to
their burden of evaluating the solutions available in a larger pool of
options. Although the cliché example for Python developers is the
over-abundance of web frameworks, the situation with template
languages
is even worse.

So why do people start new template languages, web frameworks, ORMs,
GUI toolkits, et cetera, instead of using and contributing to an
existing project? Before open source became so prevalent,
contributing to an existing project wasn’t really an option, so the
economic considerations for building vs. buying were straightforward:
Could you build the thing, either via contractors or full-time
employees, for less than it would cost to buy it from a vendor, adapt
it to your needs, and pay for the support? These days, open source
eliminates a lot of the questions about up-front cost, but that
doesn’t necessarily make the question any easier to answer.

For most open source projects, there isn’t anyone to pay for support
and, as Titus Brown points out in his post It’s just too easy, “It’s
less work to code a few hundred lines of Python than it is to
understand someone else’s few hundred lines of Python”. That’s
where we fool ourselves, though. Developers tend to have tunnel
vision, seeing the easy parts of a project clearly and glossing over
the tricky edge cases that are going to trip us up. Starting from
scratch may only look like less work because we don’t see the
hard parts.

On the other hand, sometimes starting from scratch can have benefits.
For one thing, picking up in the middle of a project can be difficult,
too. There’s a longstanding joke in the Zope community that its
learning curve is “Z-shaped”. A few years ago I made the effort to
learn its inner workings and was a serious Zope user. But when I put
it down and stopped using it regularly, my skills became rusty very
quickly. Like the saying goes, “it’s a great place to live but I
wouldn’t want to visit there”. Maybe that says more about me than
Zope, though. In any event, a smaller framework may do less, but can
be easier to pick up and learn. Fewer features also mean less stuff
to keep in your head once you become an experienced user.

“it’s a great place to live but I wouldn’t want to visit there”

An obvious reason to build your own is if none of the existing tools
do what you want. When I built HappyDoc, back in 1998, I only started
the project because the other similar doc-extraction tools had a
limitation that made it impossible for me to use them: They all
imported your code to find the classes, functions, and docstrings.
Unfortunately, the code I wanted to document had import side-effects,
so generating the documentation with tools that imported it required a
special server configuration with resources that weren’t easy to
replicate. The key feature of HappyDoc was that it worked by reading
and parsing the source code, without doing a full “import”, so there
were no side-effects. Since then, other tools have come along that
work the same way, but at the time it was enough to differentiate
HappyDoc from the other contenders. I ended up writing more than a
few hundred lines of Python, though, so, re-examining my decision with
the benefit of hindsight, I see that I might have been better off
contributing to an existing project instead of starting a new one.

There’s still no easy way to answer the build-versus-buy question. I
lean toward using open source projects when possible, but there are
times when you’re just better off building exactly what you want than
adapting your code to someone else’s way of thinking. How do you
decide when to build and when to buy? Let me know what you think.

Originally published in Python Magazine Volume 2 Issue 10 , October, 2008