The Saga of the Weekly Newsletter

Probably the main reason my employer hired me was because the Exchange server had crashed and they needed help immediately. A biweekly newsletter had a “problem” and the Exchange server had run out of disk space.  Since I arrived at the peak for the season and they had several high priority items for me to work on, the initial solution was to stop sending the newsletter.

About six months later we decided to restart the newsletter and the results were pretty ugly. Exchange did not crash but there were a lot of error messages. So I set about fixing the newsletter.

The configuration consisted of a Visual Basic application and a local SMTP server.  This configuration was in response to some severe problems encountered when the application ran directly off of the production server. The processing is simple. A Visual Basic application pulls the current newsletter and the mailing list from our data base and then sends individual emails to the SMTP server. When I started working on the newsletter, the mailing list had about 120,000 people on it. There was an automated program that was supposed to clean up the mailing list based on the error messages returned. Obviously this wasn’t working.

To start fixing the problems I adopted a check list that is very similar to what John Pollard of ReturnPath recommended recently in a post. Although we are not big enough to use their services, I appreciate the efforts that ReturnPath has made in the industry. Here is John’s list.

What should a marketer do to establish Sending Permanence and a good sending reputation on a new IP address?

The answer to this question is to take the new IP address through a warm-up process that works within the ISP restrictions over a period of time. An ISP wants to protect its members from spammers, so working within their rules can help your IP establish a positive sending history and a good sending reputation. Below is a list of seven simple tips to help you get started warming up your IPs:

  1. Deploy from a sound infrastructure: A correctly configured DNS, such as reverse DNS (PTR) records and authenticating with SPF and DKIM tells an ISP that you are more likely a legitimate sender.
  2. Sign up your IP address with all available feedback loops: Maintaining a low complaint rate is a primary factor in determining your sending reputation. Spammers don’t care about complaints and IP addresses with high complaint rates experience ISP blocks and email being sent to the junk folder.
  3. Ensure your bounce process is set to remove unknown users after one bounce: Maintaining a low unknown user rate is another primary factor in determining your sending reputation. IP addresses sending to a large number of unknown users experience ISP blocks and email being sent to the junk folder.
  4. Clean up your email list of inactive and unengaged subscribers: Spam traps are inactive email addresses created by an ISP to catch spammers or marketers with bad list hygiene practices. IP addresses sending to a large number of spam traps experience ISP blocks and email being sent to the junk folder.
  5. Send to your best subscribers first: Your best subscribers are those that are actively engaged with your email program (e.g. high opens, click-throughs and conversions). These subscribers give your new IP address high engagement metrics which tells an ISP that your email is legitimate.
  6. Start slowly and increase volume over time: ISPs limit the amount of email they will accept from a new IP address (throttling). Start sending a small number of emails to each ISP and let their spam filters get used to the sending behavior of your new IP address before increasing volume.
  7. Monitor your Deliverability: Monitor how the ISPs are treating your email during the warm-up process using Return Path’s Mailbox Monitor. Your Inbox Placement Rate is a good indicator of when an ISP’s spam filter adjusts and can help identify other deliverability problems.

Here is my check list for fixing our newsletter problems.

  1. The first thing I did was to set up subdomains and reverse DNS (PTR) records for the static IPs we would be using to send out the mail. The reverse pointer record is a requirement for most feedback loops. Since we had ISP connections from two different ISP providers and we saturate the connection for several hours, I routed the newsletter to our backup internet gateway. That made the folks in the office happy.
  2. Next I set up feedback loops for those ISPs that did not require DKIM, AOL, Roadrunner, and Comcast.
  3. Then I ran our mailing list through a batch program to verify email addresses. This knocked off about 20,000 unknown or obsolete email addresses off our mailing list.
  4. Since the automated mailing list cleanup procedure did not work, I replaced it with two procedures. I used a semi-automatic procedure using a group of Regex expressions to categorize the SMTP errors I was getting off of the SMTP server. It seems that every ISP has a different way to tell the sender that an email address is unknown or inactive. The 5.1.1 and various flavors of the 5.5.0 code are the most common SMTP error codes for unknown or inactive users but I did find a couple of folks using the 5.7.1 code. I use this list to remove about 60 users off of the mailing list a week. I also used a semi-automatic procedure of examining the returned email in the newsletter inbox. Most of the 700 messages are mail box full messages and vacation replies which I eliminate quickly through the use of filters. Although this is tedious I chose to manually go through the remaining emails. This typically is about 250 emails and consist of miscellaneous error messages not caught by one of my filters, feedback loop replies, challenge and reply messages, some unknown user replies, and the occasional customer reply.
  5. My next trick was to install a second SMTP server(iRedMail) to sign the email and forward the email to the existing SMTP server. Yahoo likes DKIM and we needed to get back in Yahoo’s good graces. Once I confirmed that we were properly signing our emails, we could set up the feedback loop with Yahoo. 

So how has this worked out. In November of 2009 we went from biweekly to weekly mailings and our sales from the newsletter doubled. This surprised our marketing guy. Our mailing list which started out at 120,000 in 2009 is now down to 88,000. This drop has some benefits that are not obvious. Yahoo has not throttled us in over a year so we must be doing something right. Even when an ISP spam checking routine(e.g. Comcast) bans us, it is quickly resolved by filling out a form. Since we are complying with all of the best practices for senders, I can release the email in 24 hours. Even though our mailing list is down about 2.5% from 2011 and our sales are up 50%. These are pretty impressive numbers considering that our sales outside of the newsletter are flat compared to last year. For those of you who keep track of the sender score, our score went from 80 to 96. Here is our latest report.