Our Custom Built Newsletter System

In December of last year we released a project that we’d been working on for a long time. It’s since saved us countless hours, reduced errors, improved accuracy, and opened the doors for some really unique future opportunities. The project was a complete overhaul of how we perform one of our most critical business processes, one that is one of our largest revenue drivers. Yet none of our customers have noticed. And that’s the way we wanted it. That meant it was a success.

From the title, you’ve probably already figured out that I’m talking about a custom built newsletter system. For years we had used phpList, a powerful open source email manager. Overall it did a great job. There are some downsides to running your own email server, but we put the work in to configure both our server and IP addresses for sending large amounts of email (there’s a good post on the 37Signals blog that outlines some of the best practices for running your own mail servers).

There were some disadvantages to phpList though. The process of sending a newsletter was very manual. It involved a lot of exporting from our database and importing into their system. Unsubscribes and bounces didn’t sync with our database so we had manual processes for updating those. We weren’t using it to send our Daily Special newsletter, which is almost as large as our regular newsletter list. And as with any third party software, it needs to be updated regularly to fix bugs and prevent vulnerabilities.

We finally decided it was worth bringing it in house. In doing so, we wanted to build our dream system that made every part of sending emails better. I thought it was going to be a monster project, but it turned out to only take about a month from start to finish. One of those rare projects where everything sort of worked on the first try, and even a year later we haven’t had any issues at all (knock on wood).

Here’s a list of features:

  • A newsletter can be created in a matter of minutes: pick a list, enter a subject, enter some HTML content, preview it, send a test to yourself, and then schedule it (or save it as a draft for later).
  • A web version is automatically created and linked to from within the newsletter.
  • Links are auto-tagged for Google Analytics.
  • Real-time stats. Leave the browser window open and see a completion % bar as well as opens, clicks, and unsubscribes.
  • Pause a newsletter during sending. This is helpful if we notice an error while the newsletter is going out. It can then be edited and restarted.
  • Email throttling based upon variables we set (i.e. send no more than 100 emails per minute).
  • Creating a new template is simple.  There are support for “widgets” that one might want to add to a newsletter, such as the latest Ask a Pro Blog articles.
  • Lists are pulled live from our database.
  • Adding a new list is as simple as writing a MySQL query for the users you want on the list.
  • One-click unsubscribe links are automatically placed in each template, and if someone clicks the link the database is updated instantly.
  • Automated bounce management. If someone’s email bounces X number of times, we automatically unsubscribe them from our list.

Mike can literally now send better newsletters in a fraction of the time with a fraction of the stress.  To this point, it has been a big accomplishment to seamlessly transition our rather large lists over to this system.

Now that it works though, it opens up doors for all sorts of opportunities.  With our newsletter system so tightly integrated with our shopping cart, we can tie data together that other companies can’t.

For instance, we could send a special offer to any customer from New York who has placed 2 or more orders with us, hasn’t placed an order in the past year, but has opened more than 50% of the newsletters we’ve sent to them.  That’s just the first thing that popped in to my mind.  Really any combination of content, tips, or offers could be paired with any combination of user behaviors or traits.

We’ve done some segmented newsletters in the past, but nothing like this.  If done well, this has the potential to increase revenue per order, orders per customer, and overall customer satisfaction.

5 comments on Our Custom Built Newsletter System

  1. Tim says:

    Interesting, for some reason I thought you used a third party for email blasts, really cool you have a totally custom solution that works for your needs. It’s one of the projects I’m currently exploring, doing it internally is not even on the table, and there’s a 50+ person IT department here that handles development. Funny how small businesses can do things more effectively and better than large “enterprise” business.

    One of the things I’ve been doing is reading thoroughly through the CAN-SPAM act and it’s surprisingly flexible and can be sender-centric if the message is not promotional.

    • Adam McFarland says:

      Agreed on the CAN-SPAM act, it’s surprisingly pretty fair to all parties!

      You’ll have to let me know what your company ends up doing. I’d imagine at your scale there are problems I’ve never even thought of! Multiple databases, security issues, etc. For sure one of the advantages of being small is that it’s much easier to accomplish everything in one fell swoop with a system like this.

  2. Dale says:

    Adam, how does what you do compare with using something like Mailchimp?

    • Adam McFarland says:

      Good question Dale. Mailchip and similar services are great. If I was starting a business from scratch I’d definitely go that route (similar to how I’d probably use a shopping cart like Shopify if we were starting over again). The most difficult part of sending mail yourself is setting up your mail server, which is something we had already done when we used phpList.

      There are two main disadvantages to Mailchimp for us. First, we send far too much volume and their high volume plans are expensive (although getting cheaper). The second is that we need super tight integration in to our database. The Daily Special email is completely auto generated from our database each morning, for example. And down the road we want to be able to more of this. Mailchimp does have an API that could potentially accomplish some or all of that.

      But given the position we were in with the server already set up, it was easier, cheaper, and an overall better solution to do it ourselves. Starting out though, I’d absolutely use Mailchimp.

  3. […] few years ago when we built our custom newsletter system for Detailed Image I wrote about all of the extensive features it included. One feature we overlooked though was user agent tracking for opens, which can be used to determine […]

Leave a Reply

Your email address will not be published. Required fields are marked *

Commenting Rules

I'm honored that you found this post interesting enough to leave a comment. Before posting, I have a few ground rules:

  • Please keep your comments as relevant to the post as possible.
  • No personal attacks or any other nastiness.
  • Your first comment is subject to my approval.

Thanks!