Our Caching System, Guide Improvements, and the Exciting Next Step

Continuing the theme from Adding To The Foundation [Of Our Cart] and A Better Search Experience on Detailed Image, I recently wrapped up a few more important features.

Caching

One big opportunity for improvement on any database driven website is caching. Let’s take the example of our sitemap. When someone used to land on that page, it would run a database query to extract the product name and URL of every product on our site. If it’s hypothetically accessed 100x/day, that would have been 100 queries. They’re not particularly taxing queries, but there are hundreds of similar scenarios happening simultaneously all day long, which in sum do make a difference.

The solution is to cache – save a file on the server with the sitemap in it, and only update that file at specific regular intervals. For the sitemap, we update it once per day. So now those 100 queries become 1. A simple example, but I think it illustrates the point.

I wanted to build a caching system that could be used for any aspect of our site, from those sitemaps to product pages to stuff we haven’t thought of yet.

To accomplish that, I decided to create a PHP class* that does a few simple things:

  • Converts an array to JSON and then writes that array to a file in our cache directory on our server
  • Retreives the file and then converts that JSON back into an array

It’s a pretty simple 35 lines of code that can then be used across the site to quickly store and retrieve information.

As we rework the entire site, this will be a staple in what we do: anything that can be cached will be cached, to speed up load times and minimize resources, the latter being particularly important during crazy shopping spikes like Cyber Monday. Caching product pages will likely have the greatest impact.

Right now though, this system is only in place for a few select sections of the site, like the sitemap and our Auto Detailing Guide…

Reworking Our Guide

In the past two years we’ve done two substantial rewrites to our Auto Detailing Guide. The first was solely about improving the content, the second additionally improved upon our linking, product recommendations, and introduced a downloadable PDF version.

Amazingly, none of this was in our database. It was all old school HTML files. The PDF was manually updated, uploaded, and linked to each time we made even the slightest grammatical change. It was a pain in the ass. This was just one of those things that we started doing one way in 2007 with the initial site launch, and never improved upon because we did only a few revisions per year.

Now we’re looking to revise much more often, as should be the case with any web based how-to guide. To accomplish that though, we first had to make it less tedious to update. I started by getting the existing guide into our database. All of the content, structure, and product recommendations are being pulled from the database. Of course, since these pages aren’t updated all that often, they take full advantage of the aforementioned caching system and are only regenerated from the database when there’s a change in the content.

The last – and most difficult part – was auto-generating the PDF. I used the robust HTML2PDF to accomplish this. There were a lot of little problems that took a while to work out. The most difficult** was properly handling when one section of the guide links to another section of the guide. Instead of the link within the PDF bringing the user to their browser, you want it to jump internally within the PDF to that other section. It took some time, but the result is a better PDF than we had been producing manually before.

As a last step, the PDF also is only re-generated when the cache is updated. Now someone can make the change once in our database, update the cache, and be done. Eventually we’ll build this into our Admin so our employees can make changes, but for now this is a monumental leap.

What’s Next?

Now comes the fun part. The foundation is in place for us to devote all of our attention to the HTML5 responsive design that I touched upon in that first post last month. More to come soon!

*A goal of mine is to open source this code and other code that may be helpful to other developers in the future…not the imminent future, but hopefully in the next year or two.

**It was also quite challenging that the library was written and documented in French, with some English translations available, but many of the examples and variables still in French. Those five years of French I took in middle school and high school finally paid off a bit! I’m far from fluent, but was able to understand enough to get done what I needed to get done.

4 comments on Our Caching System, Guide Improvements, and the Exciting Next Step

  1. Rob says:

    Sounds good. Are you noticing any immediate changes to your analytics? Time-on-site, pages viewed and ultimately revenue would be the ones I’d imagine you hope to increase.

    What happens when a page needs changing before your cron job makes a new cache file? For instance, if an item goes out of stock or if a customer leaves a new review.

    • Adam McFarland says:

      Are you noticing any immediate changes to your analytics?

      I haven’t even looked since January is such a slow month for us relative to the rest of the year. February picks up a bit, and then the spring rush hits in March. Once the new design is released (hopefully coinciding with said rush) we’ll be monitoring all of those things very closely.

      What happens when a page needs changing before your cron job makes a new cache file? For instance, if an item goes out of stock or if a customer leaves a new review.

      Good questions :) That’s why I held off on the product pages for now. They’re quite a bit more complex. The qty in stock will probably always be database driven so that we can put something on backorder or take it off instantly. The product descriptions themselves are more apt for caching. We manually approve each review before it’s published, so I’ll probably just have it rebuild that cache upon approval. Where it gets tricky is if one of our employees edits a review (say to remove profanity or a promotional link), we’ll again have to build the cache. It certainly can be a pain and has it’s downsides, but overall I think it’ll be worth it for most of our critical pages.

  2. [...] and I have been pushing really hard to release the aforementioned new version of Detailed Image. As a company we’re probably devoting close to 40% of our time [...]

  3. [...] almost ready to launch the new Detailed Image site. The planned launch day is Saturday, April 20th. We’re putting the finishing touches on [...]

Leave a Reply

Your email address will not be published. Required fields are marked *

Commenting Rules

Wow. I'm honored that you found this post interesting enough to leave a comment. Before posting, I have a few ground rules:

  • Please keep your comments as relevant to the post as possible.
  • No personal attacks or any other nastiness.
  • Your first comment is subject to my approval.

Thanks!