Music Alerts Fixes – More Speed and No Duplicates

I received a handful of great emails/comments about Music Alerts in the last 24 hours.  I’ve also heard from a few hardcore fans who tried creating feeds of 200+ artists.  I call them hardcore because mine consisted of about 25 :).  As with any really raw program the faults were exposed for those large feeds so I spent a few hours today cleaning the programming up:

  1. Faster feed generation.  Previously, I was generating the feeds on the fly every request…meaning each time you or your feed reader requested the feed, the system called Amazon and requested data back.  Aside from being inefficient, this took A LONG time and timed out the larger feeds.  My temporary solution was to build a “caching” system that stores the feed on Music-Alerts.com and checks for updates every 3 days.  The first time you load a large feed it might take a while, but after that you should be in the clear.  I’ll look into a better permanent system down the road.
  2. Duplicate elimination.  I noticed my feed had a few duplicates in it.  I also noticed larger feeds had like 5 per album!  This was 50% my fault and 50% Amazon’s:  I had messed up on one of my loops and turns out if an artist didn’t have an album being released soon it showed the last one that did instead of skipping to the next artist.  On top of that, Amazon returns duplicates by nature as it is (do a quick search and see for yourself).  For that, I built a filter that compares album titles and weeds out any duplicates.

If you happened to build a feed prior to this post, it *should* work but your feed reader (which has the old one cached) might give you hell so it might be easier to just set up a new one.  The <description> tag in the feed has a comma-separated list of your artists so you can just copy and paste to make a new one…just remember to delete the word “and” before the last artist.

Whew.  This “weekend project” turned into a little more than that.  I’ll probably spend some time this Saturday or Sunday trying to market it a bit.  It’s definitely the type of thing that can catch on virally, so my hopes are I can get a TechCrunch or similar style tech blog to pick up on it…we’ll see.  I’ve got a few other tricks up my sleeve too…I’ll keep you posted if anything works.

Now back to “real” work.

1 comment on Music Alerts Fixes – More Speed and No Duplicates

  1. […] plan it to scale.  I also wasn’t as good of a programmer as I am now.  When I made a database caching system for feeds, I figured I’d done everything I needed to for it to […]

Leave a Reply

Your email address will not be published. Required fields are marked *

Commenting Rules

I'm honored that you found this post interesting enough to leave a comment. Before posting, I have a few ground rules:

  • Please keep your comments as relevant to the post as possible.
  • No personal attacks or any other nastiness.
  • Your first comment is subject to my approval.

Thanks!