I was confronted with a pretty common programmer’s problem at work a few weeks ago. I inherited a script which was taking 20+ seconds to render a map for a mobile site. 20 seconds is a long time in a mobile user-experience – 5 seconds is enough to lose a visitor. 20 seconds would just infuriate them!
No biggie I thought, probably just some query that needs optimising. However on looking into the generation of the data, I found a rabbit warren of legacy code (all code is legacy though really!) with functions calling other functions in loops over enormous data sets which called other functions which called other queries in other loops. Not the kind of thing you can optimise in a hurry, and to be fair the code was fairly well ‘encapsulated’. Often the problem with object oriented methodologies where black boxes are reused without too much thought is that the true power of SQL is lost in the ORM abstraction layers.
So what to do in a hurry? Well the script was just generating a big XML document for a Google map, so it could be cached. The data would be good for 15 minutes or so, so that’s a reasonable cache expiry time. However the shared, clustered hosting setup didn’t allow for a smart solution like memcached, or a simple file cache. I needed something that would act like a write-back cache across a cluster of 10 dumb web servers.
Thus was born LayerCache.
It’s a pretty flexible cache provider which allows you to build a stack of as many layers of caching as you like: from memory, local files, shared files, de-normalised database, CDN, memcached whatever you like. The cache tries the fastest one first, then traverses the layers downwards (potentially across architectures into shared facilities), until it either reaches a dead end, or optionally the source provider.
Once it has the data, it propagates it back up through the layers in almost write-through fashion (I call it write-up) as its transparent to the caller, but not done asynchronously so kind of half-way between write-through and write-back.
The coding and unit tests were fun – I really enjoyed burrowing deep into this one. But the decision on the OSS licence nearly lost me a week’s sleep!