Web Forefront

The first thing you need to realize is that while you may think your reaching Hotmail/MSN/Live users, your messages could actually be being sent to a black-hole. Yes that's right, no bounce messages -- which are normal in both Yahoo's and Google's mail services, and for that matter any other mail server on the web -- in fact, messages in Hotmail/MSN/Live are often not even delivered to a users 'Junk Mail' folder, so both the sender and receiver are kept in the dark. So what is one to do ? Well, Microsoft has setup numerous draconian processes in order for you to reach their customers -- and in all likelihood your customers too -- that make use of Hotmail/MSN/Live accounts.

While having your company newsletter, marketing spin or any other mass-mailing material not delivered to Hotmail/MSN/Live users can be a big blow, its crippling for any small company to have to go through hoops of fire just to get a vanilla message saying 'Thank you' to a customer, in fact, what is particularly irksome with Hotmail/MSN/Live is that the same message getting through to your customers using Yahoo and Gmail accounts can get snagged by the Hotmail/MSN/Live black-hole, so the remainder of this entry will cover my experience dealing with Hotmail/MSN/Live, with numerous technical tidbits along the way.

Your company's I.P server address, ah yes, this is your web footprint so to speak. Every email message you send now a days is screened to see from what I.P address it originated, in most cases you will be given the benefit of the doubt, but if your address is flagged for some reason that's were the trouble starts.

There are actually numerous lists keeping track of I.P addresses that are known to send spam on a constant basis, but alas, Microsoft is in another playing field, even if your I.P address is not on a widely used list like Spam Cop , you may still be taken by surprise, so how do you know if an I.P address is being blocked by Hotmail/MSN/Live ? First stop: http://postmaster.live.com/

I'll spare you the time needing to go through the PR spin, go to the Smart Network Data Services (SNDS) section, its a free sign-up that will shed some light on how your I.P address is being treated by Hotmail/MSN/Live servers. To get access, you will need a Hotmail/MSN/Live account and mail-receiving-privileges on the I.P address you are trying to see.

So knowing your I.P address go to the 'Request Access' tab and a list of all the authorized emails for that I.P address will appear, such as: postmaster@yourcompany.com, and also likely, postmaster@yourisp.com, abuse@yourisp.com. This is done because your access to SNDS will give you detailed traffic statistics about Hotmail/MSN/Live to this I.P address, and of course, they don't want just anyone poking around every I.P address in existence, so they determine which domain administration emails are associated to the I.P address in order to confirm authorization. In my case, SNDS fortunately guessed and listed an email I currently had access to and got the authorization email quickly, but lets hope your ISP has done its I.P address management correctly and had the I.P associated with your domain (Another friend was not so lucky, so he had to open a ticket with his ISP...and so on ).

You've now confirmed you have the right to see how Hotmail/MSN/Live is treating your I.P address, though a simple "Not Blocked" or "Blocked" would have worked, lets see what you got. Depending on how many Hotmail/MSN/Live users you are trying to reach, in the 'View Data' tab you will either see a 'No data for specified IPs on this date' or an image like the following:

To truly determine if you are being blocked, you will need to select a day on which you knowingly sent the most mails to Hotmail/MSN/Live users, above 200 emails seems to trigger a report as you can see by the previous image. At this point, I hope you don't see a red box on any calendar day, but in case you do, go for a break and take a deep breath.

The next step is contacting Microsoft support, oh yeah they do provide it but don't get to excited, its practically a 'canned' response, you can fill out their 30+ question form here: https://support.msn.com/eform.aspx?productKey=edfsmsbl&ct=eformts which needs to include your I.P address, sample messages you are trying to send, what OS are you using, what mailing list software, among other things.

Here again, I'll try and save you the trouble of going through the 'canned' response which contains their recommendations and ends with the rather dire sentence:The troubleshooting steps in this email are recommendations only. Microsoft makes no guarantees that following these steps will guarantee deliver ability to MSN, Hotmail, or Live.com customers.

Even though the 'makes no guarantees' sentence made me weary, I thought heck this is for those Viagra spammers which send out millions of emails, I only send out a few thousand a month to Hotmail/MSN/Live users, so lets try and comply with their guidelines to prove my emails are legit.

The next stop was Sender Policy Framework(SPF) or Sender-ID in Microsoft speak, a mechanism which prevents email 'spoofing' -- someone sending an email by any given name -- and nipping it before it arrives at a user's Inbox ( I actually wrote a technical article on this some time ago in case your interested : SPF - Sender Policy Framework for Email ).

To setup SPF/Sender-ID you will need to modify your domain's DNS records to include such values, in this case, Microsoft offers a wizard to create SPF fields , and having SPF for your domain is actually a good thing -- Hotmail/MSN/Live aside -- since it actually indicates 'email from mycompany.com can ONLY come from x.x.x.x address', granted it only works if the receiving party enforces it, your taking the necessary steps to legitimize where your emails can originate.

Next, and just to make my case air-tight was signing up for the Junk Mail Reporting Program (JMRP) by Microsoft. It took about 1 week with back and forth emails, bugging my ISP to send a confirmation email to Microsoft indicating I was the sole administrator of my I.P address, and actually qualifying to join, though I have no idea how they determine if you qualify or not.

The intent behind JMRP is pretty simple, MSN/Live/Hotmail forwards you users that were bugged by your emails, in other words, you get a list of people who 'junked' your email, to which of course you need to follow through eliminating them from future listings, its also good because you generally never know when someone 'junks' your email, so it can also shed some light on how your email is being perceived.

Ok, so here you are with your domain already participating in three recommendations by Microsoft so Hotmail/MSN/Live users can receive your emails: SNDS, SPF and JMRP. After having implemented this, I made another test run with my customer mailing list, the results: same, no bounces, no received messages, so back to Microsoft support in the form mentioned earlier.

This time, besides the 'canned' response I got a little more insight from someone actually reading what I filled out, indicating: "We have identified that messages from your IP are being filtered based on the recommendations of the SmartScreen filter. SmartScreen is the spam filtering technology developed and operated by Microsoft. SmartScreen is built around the technology of machine learning. SmartScreen's filters are trained to recognize what is spam and what isn't spam. In short, we filter incoming emails that look like spam. I am not able to go into any specific details about what these filters specifically entail, as this would render them useless", that literally means my newsletter is crap to Hotmail/MSN/Live and they cant tell me why it sucks, ok I can take it, especially since the same email in much the same volume goes through to clients with Yahoo and Gmail accounts, so it can't be that bad.

Still the response goes on..."While using the SNDS tool, enrollment in the JMRP or having your IPs registered with Sender ID will not allow emails from your mail servers to bypass our filters, these are in place to help legitimate companies deliver their emails to Hotmail Customers", well they told me no guarantees, but isn't there any other way to reach Hotmail/MSN/Live users from email originating on my domain ? Oh yes, show me the money.

The Sender Score Certified Mail Program is yet another recommendation made by Hotmail/MSN/Live to legitimize your emails, however, contrary to all the other options which are simply one-time events or free, this one actually involves shelling out a non-refundable application-fee between $400-$1500 (U.S) plus an annual fee between $1,000-$20,000 (U.S), depending on your email volume.

I can't say that I've looked deep enough into what I would actually gain from paying some third party to legitimize my mailing list, but for my email volume, I can probably say with certainty it would be cheaper to print out the email and send it out via first class snail-mail -- talk about those who said paying 'postage' for email would never work, this would be an indirect 'postage' to get by faster and with certainty -- may be this is a drop in the bucket for the like's of Dell, Amazon, P&G or GE, but for smaller companies I would be hard pressed to believe its that easy to justify this kind of investment just to reach one segment of web mail users, and even then, remember there are no guarantees you will be able to reach Hotmail/MSN/Live customers.

With my options exhausted, I started to think beyond these suggestions. One possibility I thought of was changing my I.P address, but after further review it was a very unpractical approach, for one because a change in I.P implies changes to practically everything on your website -- from DNS records to server settings -- and secondly Hotmail/MSN/Live would probably and eventually flag the I.P again after a few months since it would be producing the same email content, so an I.P address change was like changing houses every few months, something you want to keep to a minimum.

Dual or triple I.P addresses for my domain was yet another possibilty -- one address for sending newsletters, another for business email, another for personal emails -- additonal I.P addresses generally run a few bucks from most hosting providers, but on this possibility it was not so much the price, but rather the technical DNS/Mail Server/Application setup to operate this, more on why you would even consider multiple I.P addresses at the end.

Another option I thought about was telling my Hotmail/MSN/Live account customers to add me to their white-list, this seemed plausible at first, but again, its asking users to 'take action' and go beyond their normal navigation patterns, not to mention adding a user to a Hotmail/MSN/Live white-list is not a one click event.

At this juncture I pretty much confronted reality, Hotmail/MSN/Live had started tagging my newsletter as junk, and neither of their suggestions had helped me reach their users which were also my customers...oh well, I still had Yahoo and Gmail customers to work with and update them on a monthly basis, those using Hotmail/MSN/Live would just need to visit my site more often...BUT WAIT, what about those sporadic emails I would send to customers and partners using Hotmail/MSN/Live accounts ? like 'Thanks for your purchase', 'Can we talk by phone today at noon', surely those would not go into oblivion ? Well, guess again.

Turns out the same 'SmartScreen' that is 'built around the technology of machine learning' is as simple as: True or False for your whole I.P address! That's right, if for some reason your I.P is flagged -- which you can see in the SNDS mentioned earlier -- it doesn't matter what subject or content you use in any email originating on your domain, it will ALL get routed to a black-hole, no undeliverable warnings, no nothing. SPF and JMRP don't help either.

While I clearly understand and relate to Microsoft's attempt to limit spamm'y newsletters and mass-mailings, I don't have words for how Hotmail/MSN/Live deals with both its account users -- who are being deprived of legitimate short communication emails -- and the up until now draconian and useless processes required for third party's to reach these same customers.

Its been 5 months since I noticed the blockage, perhaps its been going on longer and its frustarting to say the least. But hopefully, this blog entry will help you deal with the overall technical process involved in reaching Hotmail/MSN/Live accounts and avoid you banging your head against the wall a few less times than I did. If the situation changes, I will follow up with an update in this same section.

Update Jan 08': Went ahead with a new I.P address setup, it worked BUT for about 12 hours, then it got blocked after sending around 100 emails. Then tried it with multiple I.P addresses, even changing the domain assigned to the host, numerous other Sendmail hacks and even DNS modification, in some instances it worked for the same 100 email range, but the server eventually got blocked again. I don't no about the filtering/Bayesian technology, but they are pretty good a blocking all together.

Update Feb 08': Seems they are finally and at least telling you the I.P is being blocked! We didn't bother to remove Hotmail/Msn emails from our lists since they were being destroyed anyway, but in our last newsletter mailing we got a handful of returned emails of the 'Warning: could not send message for past 4 hours' kind, which said: <<< 421 RP-001 The mail server IP connecting to Windows Live Hotmail server has exceeded the rate limit allowed. Reason for rate limitation is related to IP/domain reputation problems. If you are not an email/network admin please contact your E-mail/Internet Service Provider for help. Email/network admins, please visit http://postmaster.live.com for email delivery information and support, cold comfort I would say, but a nice sanity check for the yet uninformed. Just re-read the whole entry if you want a summary of what you can and will achieve upon visiting http://postmaster.live.com.

Update Mar 08': One more shot at trying to get through, now using QMail and a new set of I.P's. This is about the most obscure and esoteric reason we've gotten yet for not getting through and it came thanks to Qmail : Turns out our mailing application was composing emails with Bare Line Feeds in SMTP, so what is a bare line feed ? It basically means emails were using \n for breaking lines, instead of \r\n which is the recommendation by RFC 822bis Section 2.3... The nitty gritty here: http://cr.yp.to/docs/smtplf.html , when I saw this document and read Why should I worry about bare LFs? You can't get mail through to msn.com and thousands of other systems around the Internet. Your mailer is violating 822bis section 2.3, which specifically prohibits all bare LFs. I felt that was it, what else could it be but this obscure error! Well, corrected the bare line feeds, our Qmail setup stopped complaining, but still no dice getting through Hotmai/MSN/Live servers, even now we're getting a 'message accepted for delivery' which equals 'black-hole' to them, unbelievable.