DI Features and Lessons Learned

Since we launched the new Detailed Image site last week we’ve had a lot of “the site looks cool” compliments. While we certainly appreciate the positive feedback, 99.99% of visitors will never ever recognize the changes we made to help improve Pure Adapt as a business. This post is dedicated to those features, why we put them in place, and the lessons learned in building a custom shopping cart from scratch. The goal was to have at least the functionality of the old cart for about half the site (which was an osCommerce cart) while implementing a laundry list of improvements for the rest of it. I think this post will be particularly useful to programmers who are planning to program their own e-commerce platform – many of the problems I encountered I struggled to solve because there were very few resources available.

Let’s start with some of the features:

Rewritten URLs
One of the primary reasons that we decided to pull the trigger and begin developing the site was that DI received virtually no search engine traffic. The osCommerce site consisted of literally two files /store/index.php and /store/product_info.php and all pages were dynamically generated based upon a series of URL variables like ?category=12&cpath=4&productid=99. Search engines are getting better at indexing dynamic sites, but DI was clearly having problems (they have several thousand links and still were barely being crawled).

The solution to this problem for dynamically driven sites is URL rewriting, which is basically changing the URLs to become unique for each page and product without using any URL variables. For example, http://www.detailedimage.com/Poorboys-World-M1/Spray-Wipe-SW-P32/. These URLs are “hackable” (you could visit http://www.detailedimage.com/Poorboys-World-M1 and it would work fine), and also provide the added bonus of being more “clickable” when someone searches. Example – if you searched “Poorboys World Spray and Wipe” wouldn’t you be more likely to click if Google displayed that URL as opposed to http://www.detailedimage.com/store/index.php?productid=89 ?

If your site is on an Apache server, you can accomplish this relatively easily in your .htaccess file. The biggest thing to remember is that you need to 301 redirect all of your old pages to the new ones so that search engines (and people) looking for the old page will be redirected to the new one.

Product Upsells
We built a product upsell system that recommends “Complimentary Items” and “Works Best With” items uniquely for each individual product. The relationships were hand coded by George based upon his years of experience, so they are highly relevant. Users can purchase the item by itself, or as part of their own custom bundle based upon the upsells they select. We anticipate that these will raise the average sale price quite a bit.

Detailed Image Product Upsells

Google Analytics E-Commerce Integration
I think that by now most people know that Google Analytics is the preeminent web analytics tracking software available. Aside from being the right price (free) and easy to install, the automatic integration with Google AdWords and configurable tracking ‘goals’ make it a hands down choice. But one of the features that most people don’t know about is the e-commerce revenue tracking that they have built in.

This is one of the most useful features I’ve ever seen in analytics software. We pass the product information for each transaction to Google on the receipt page via a simple javascript, and in turn it tracks EVERYTHING related to the transaction. We now can determine how much revenue is derived from organic search vs. PPC vs. forums we sponsor – it brings web analytics full circle. We can even see revenue and products sold for each individual forum – so now we can precisely evaluate the ROI on a $300/month forum, for example. We can also see which type of visitor spends the most on average per transaction…or which type of visitor purchases which products. Data can also be segmented by location (country, state, etc), browser, and just about any other way you’d want to splice and dice it. Oh, and you can even use it to track affiliate sales if you have an affiliate program (we’re planning one on in the next few months). You can’t put a price on the value of this information to a small company like us. We can make much more informed decisions now.

Backend Automation
For the past two years, George and Greg have spent several hours each day manually processing each order: they charge out the customer, enter his/her billing info into Quickbooks, request a FedEx quote, print out invoices and shipping label, pack and ship….for each and every customer. Now we’ve got all of that down to a process that takes less than 3 minutes (not including packing time of course). In addition to saving them 3-6 hours each day, it eliminates the mistakes that they made manually entering customer info over and over into Quickbooks.

In addition, we also have full tracking capabilities for FedEx packages through our site for both us and the consumer, further eliminating customer service inquiries related to tracking.

CSS Overlay for Pictures
The art for the site was primarily done by artist Sam Li, who does absolutely spectacular work. His talent allowed us to bring the “driver cockpit” idea for the site to a reality. The coolest innovation was the navigation screen on the right that displays category information, specials advertisements, and product photos.

One thing I always HATE on other sites is a pop-up window when enlarging an image to view a close-up of the product. Problem solved with a slick CSS overlay for product photos.

Detailed Image Product Overlay

Auto-generated Sitemaps and Froogle Feed
The Sitemap, Google Sitemap, and Froogle feed (Google Product Search now) are all completely automated. If we add a new product, they are all updated appropriately. Previously, George was manually compiling them.

Of course, the best benefit of all is that this new system is completely scalable without limitations. We created it, so we can expand it however we want, something that wasn’t really possible with osC.

NOW, on to some of the problems I encountered and how I overcame them. This section will probably only be of value to programmers, particularly those looking to build an e-commerce site themselves.

The #1 thing I learned from a programming perspective is that most companies use an off the shelf cart (Yahoo Stores, osCommerce, CubeCart, etc), so don’t expect there to be a ton of information out there if you’re looking to build an extremely customized cart. The majority of problems I solved on my own.

FedEx/USPS Integration
This was by far the most difficult. Surprisingly, neither company makes it very easy to integrate with their site. We began the project with the idea of generating shipping labels ourselves for 100% integration…which we quickly dismissed because of the lengthy approval process that both have. We only ship USPS for International orders (relatively uncommon), so George and Greg decided to continue to do those labels online themselves.

So I needed to be able to generate rate quotes for FedEx and USPS, and be able to ship FedEx with our system. In addition, I needed tracking integration for ourselves and our customers. Here’s the solution I came up with:

  • Generate FedEx rate quotes using Vermonster’s Open Source FedEx Direct Connect. Support is nil (I tried emailing them for help once), but if you take the time to read the documentation and comments in the code it shouldn’t take too long to get up and running. Of course, you’ll need to register with FedEx to get testing (and later, production) keys.
  • Generate USPS rate quotes using Aldirgas Varnagiris’ USPS PHP rates calculator. It worked perfectly right away, however you can only run the specific examples (as opposed to using your own address) with your test account. You also can’t run any international examples using the test account (took me a day to figure that out). My advice – try the domestic example, if it works, apply for the production key and test international using that.
  • Use the FedEx Ship Manager to print invoices and generate tracking numbers. The software sucks ass, but using it means you’ll generate 100% perfect shipping labels…much easier than trying to make them yourself and then go through FedEx’s verification procedure. Our backend generates a text file for the days orders, which you import into the the Ship Manager (using what they call the “Hold file”). You click to ship, the labels print, and you export a text file back to the website to insert the tracking numbers. Sounds relatively simple, but “Hold file” documentation is scarce, so I relied on some of the FedEx DC documentation from above and also the really-hard-to-find Tagged Transaction Guide to get the file formatted properly.

If you’re trying to replicate those steps, I’ll be happy to answer any questions that might arise throughout the process.

Redirects and URL Rewriting
When moving a site or changing URLs it is critical to implement 301 search engine friendly redirects to tell both people and search robots where your new files are.

I’ve always written a 301 redirect in the .htaccess file like the following:

Redirect 301 /feedback.php http://www.sportslizard.com/contact.php

And I never had a problem. Turns out in this case, there were some higher level server issues messing with our redirects. In addition, we had dynamic URLs for our old site, which can cause problems if redirecting that way. After many, many, many hours searching around and using trial and error, I came up with the following solution for redirecting a dynamic URL to a static one using the aforementioned URL rewriting:

RewriteCond %{QUERY_STRING} ^cPath=21_113&products_id=266$
RewriteRule ^store/product_info.php$ /Menzerna-M10/Auto-Shampoo-P71/? [R=301,L]

In this example, we’re redirecting http://www.detailedimage.com/store/product_info.php?cPath=21_113&products_id=266 to http://www.detailedimage.com/Menzerna-M10/Auto-Shampoo-P71/. The QUERY_STRING is the URL variables (anything after the .php in the URL). The SUPER IMPORTANT thing to remember is that last question mark – that prevents the query string from being appended to the new URL. You’ll see that the R=301 tells the user agent that this is a 301 redirect as opposed to a normal URL rewrite.

PHP Dynamic PDF Generation
I decided to take a fresh approach to invoices and receipts. Instead of the standard “view printable invoice” we provide each customer with a link to a printable/downloadable PDF receipt. In addition to always printing correctly, it gives them the option of just downloading it to their hard drive (saving the paper). I got the idea from Mike (our fourth partner) when he told me he used PrimoPDF to PDF any receipt from an online purchase.

We’re also using dynamic PDF generation to store digital copies of all of our invoices.

After trying a whole bunch of different options, I finally settled on using FPDF – a simple open source PDF-ing software for PHP. While it’s really easy to generate a simple PDF with different text sizes, colors, and pictures, I had a ton of trouble printing the tabular data necessary for an invoice. FPDF works much like a dot-matrix printer: it prints line by line and you control the location on the line based on left/right padding. This caused problems when trying to line up an invoice. Eventually I found the Write HTML Tables add-on scrip that allowed me to format the invoice like an HTML table which worked perfectly.

CONCLUSION
Hopefully this will be a valuable resource to anyone developing an e-commerce platform, or anyone who currently has one and is looking to make a few upgrades. As I mentioned last post, this was one long and exhausting project, but it was worth every second: 3 months of work for years and years of cost savings and improved efficiencies. Over time I’m planning on posting examples of how these features help take the company to the next level.

11 comments on DI Features and Lessons Learned

  1. […] few posts ago I mentioned our PDF invoice system for Detailed Image, one of the main benefits of which is that it brings us a step closer to being a […]

  2. […] are a great operations company. The primary reason we overhauled the Detailed Image site was because of the back-end automation.  Most e-commerce sites have several steps from purchase to […]

  3. […] we’ve been spending a lot of time improving our shopping cart for Detailed Image.   When we launched the site back in September our primary goals were to get the site indexed properly and to automate the back-end shipping […]

  4. Wow Adam, this is one of the most informational posts I have found on the web regarding e commerce! Thanks for taking the time to layout all of the pitfalls you and your team encountered while creating the new features in your DI site!

    I am a software engineer and I am in the discovery stage of starting my own business.

    Kudos to you! I have added your blog to my RSS feed and I will be glued to it.

  5. Adam McFarland says:

    Thanks Ryan. Drop me an email if there’s anything I can ever do to help…or if you just want some feedback on a site I’m always happy to give my thoughts 🙂

  6. Justin says:

    Adam, thanks for this very informational post. I’m looking to creating my own side ecommerce business and this definitely helps. Will keep you updated! Keep up the good work!

  7. […] there was no need to have pre-made packages for people if we had our dynamic upsell system.  So we launched in September without packages, and eventually added them in January.  I spent less than a day programming the […]

  8. […] the launch of Tastefully Driven I’ve been intending to do a post similar to the DI Features and Lessons Learned.  But since this project had less “unknowns” there weren’t a lot of […]

  9. […] limited experience in projects of this scale, we were able to go from planning in June 2007 to a launch in September 2007.  How?  Because we focused on improving the user experience more than our experience.  Time was […]

  10. Dave says:

    What sort of automation were you able to do on the backend to incorporate having each order not be put into Quickbooks? Is it just batch loaded into the software?

    I spend a lot of my time processing each order as well.

  11. Adam McFarland says:

    Dave –

    I’ll do my best to explain how we have it set up.

    First a little background: we use FedEx to ship, using their API to give our customers quotes when checking out, we use PayPal Payments Pro to process our credit card payments, and we do our accounting in QB.

    Once an order comes through and payment is verified with PayPal the order goes in our queue to process. We actually do an automated double check with PayPal to prevent false positives leaking through, which do happen from time to time.

    The next day one of us enters the queue in our Admin system and clicks the “process” button. At that point a PDF pops up with all of the invoices automatically generated, ready for printing and packing with the customer order. A text file also opens in a new window. That file is imported into the FedEx Ship Manager software where the labels are printed as a batch. The tracking numbers are then exported to a different text file, which is uploaded to our site to send our customers tracking emails.

    The hardest part about all of this is correctly choosing a box size and weighing the order (see http://www.adammcfarland.com/2008/11/19/the-final-step-in-automated-our-order-processing-that-is/ ).

    However, it offers a huge advantage in the sense that it takes us 10 minutes to process orders whether we have 2 orders or 200.

    In terms of accounting, my partner George uses PayPal’s Quickbooks export system to export all of the transactions into QB. He usually does this twice per week.

    Hope that helps!

    Adam

Leave a Reply

Your email address will not be published. Required fields are marked *

Commenting Rules

I'm honored that you found this post interesting enough to leave a comment. Before posting, I have a few ground rules:

  • Please keep your comments as relevant to the post as possible.
  • No personal attacks or any other nastiness.
  • Your first comment is subject to my approval.

Thanks!