Speed is good!

Most of the work that has been done on this new trip to doItLater has been just refactoring the original code.  I’ve done a lot of interesting things since back then, learned a lot more, and the first time I threw it together it was a complete side-project with a short-term goal, so I didn’t spend any time deleting unneeded functions or making things really solid.

I’ve been cleaning up that mess for a little bit now.  It’s not a hard mess to clean up at all… there is a lot of quickly-written stuff and old code that isn’t needed, but at least it was very well organized.

The most recent releases of doItLater that I pushed out to the site included a profiler which lets me see what parts of the code are being slow, and a new way to connect to the database only once per page instead of nearly once per function as was being done before.

The speed improvemens have been significant.  Getting rid of multiple connections is a great way to get a huge gain across the whole site.  On top of that, the main page has been just embarassingly slow for a while.  I thought it might have had something to do with being on a super-inexpensive web-host (DreamHost), but that turned out not to be the main problem.  Since the profiler was out there, I ran it on the main page, and sure enough, the script was executing for 15 seconds, and almost all of that time was spent generating the “Random Stuff” thumbnails which shows any random content.

After looking at the query, it really is an inherently slow process.  To get around this, I now cache those results for 15 minutes (so it’ll only be slow for 4 people per hour).  I had to write a caching class which was a mix of some caching used on Pedlr and LyricWiki, then with some new stuff too.  I’m really excited to have the profiler and the “Cache” class around.  I probably won’t have to do much more performance work for a while (since it’s running fairly fast now), but it’s good to know that once we need to scale – these tools already exist.  When you need to scale an app, you’re generally in a panic because traffic is growing and everything is slow or breaking and you’re losing people.  That’s not the kind of time when you get to casually sit back and make some really shweet tools to help yourself along.

Anyway… the site is fast now.  Enjoy 🙂