tools for literate programming with Python?
I’ve had an idea kicking around in my head for a couple of days. It’s one of those things I just can’t seem to let go of, but I don’t really have time to build it right now. I’m hoping someone out there had the same idea already and written something that works sort of like what I want. Failing that, maybe someone looking for a project to start will like this idea.
I’ve seen discussions in the past of something called “literate programming”. My understanding of the gist of the idea is that you write prose and code together in a file, then use a set of tools to split them apart as part of a build process. The benefits are supposed to be exceptionally well documented code, since you’re essentially writing the documentation first and code later. I never really bought into that idea; it seemed like a lot of extra overhead. Unless you’re writing libraries to be shared by other developers there’s just not a need for so much documentation.
On the other hand, if what you’re doing is writing about code, then it seems like a great idea. I’ve been doing a lot of writing about code lately, so I want to see if I can improve my tools. I’m specifically thinking of this for working on my Python Module of the Week series, but it could be useful in other areas as well. The way I blog now means that I have to make sure I regenerate all examples before posting, in case I’ve edited any source files along the way. So my cycle goes: read docs, write code, write description (pasting in code and examples), change code, fix pasted code and examples, repeat until done.
A quick Google search turned up a few tools I need to look into, but I’m not sure what I envision is literate programming as originally defined.
I want a tool that lets me write prose and code in the same file, then extract the code for use separately, but also run the code and re-process the input file. The idea would be to edit a single file, mark the sections that are code, mark an output area for each code block. I would then use a tool to extract and run the code with the output inserted back into the original file (replacing any output from previous runs, of course). Ideally the source file would be HTML or something close, since I have to convert to that for posting anyway. I would rather not have to learn some random new markup language that I could only use with this one tool, but something like Markdown or reST would be OK.
It doesn’t seem like it would be that complicated to write a tool to do what I want using a library like BeautifulSoup to find the source blocks and output destinations. And I could set up my own trigger in TextMate to run it, so I wouldn’t have to change editors. Before I spend a bunch of time implementing something I could just download, though, I thought it would be prudent to ask a couple of Dear Lazy Web questions:
Does something like what I describe exist?
If not, what literate programming tools for Python do you recommend? I may use them as inspiration for a design.
Thanks in advance for any suggestions.