Monday, March 09, 2015

Even when you fully test, something still bites... #MailEnabled #PublicFolders can NDR after an #Office365 cutover migration

The office 365 “big bang” cutover migration took place over the weekend.  Testing had proven everything we wanted to see, and - apart from the logistics of transferring GB’s email up a 3Mb down/710Kb up ADSL connection (it’s measured in days if you want to know!) - all went as planned.

And then we noticed incoming emails to some mail enabled public folders were not working.  Some desperate testing (including borrowed external accounts) proved that internally all email to Mail Enabled public folders worked; but that some of these public folders were receving external email, some not.

At which point the tech support incident was created.  Thankfully, within 90 minutes the problem was solved and the incident closed.  Again, nice one Office 365 support.

The lesson though.  It seems Microsoft have a bit of a bug in the system for newly migrated domains in a cutover migration; in that email addresses may not function for externally sent email and cause the sender to get a 5.4.1 NDR:

But didn’t I test this?  Yes, I did.  However the testing was done on a spare domain that was handed over to the Office 365 over a month ago.  So by the time I was testing the Mail Enabled Public Folder function, they all worked.  It seems the sequence that can create the problem is to add and authenticate the domain during the cutover (but that’s how you have to do it).

The explanation.
It seems that in a cutover migration when you add further SMTP domain aliases to Office 365, it is possible for the data not to propagate around the entire O365 world quickly enough.  The net effect is that when an external party emails that public folder, the address goes unrecognised and the message NDR’s.
But, there is a fix, whizz over to the office 365 portal and take the following steps
1. Go to the admin page
2. At the bottom, go to Exchange admin
3. Click on Mail Flow down the left hand side and select accepted domains across the top to get to here (domain names hidden to protect the innocent!)
4. Now double click on the domain in question and change it from Authoritative to Internal Relay, and accept the prompt that comes up and save this (see below).

5. The net effect is that when your sender’s email hits the office 365 setup, it will (instead of NDR’ing as an unknown alias) pass on the email through the environment until it reaches your public folder.  Internal Relay forces the server accepting the incoming email to assume that if it does not know the addressee, another server will, so it works it way through.  In authoritative mode it will just NDR the email immediately.
The advice Microsoft support gave was that this setup could be left on internal relay indefinitely, but if you give it a few days (well, a week), then you could move it back to authoritative, but feel free to leave it on internal relay.

Although it wasn’t stated, I suspect the propogation issue is one related to mail enabled public folders and not mailbox or shared mailbox aliases, but I have no evidence for htat.

In hindsight I might have been able to create all the aliases on the new domains in the Office 365 setup but that would have hampered testing in that during testing we could send and receive email between the live setup and office 365 testing world.  I don’t think I would want to have given that up.

But, my recommendation?  Unless there is a good reason for not doing so, setup up your domains as internal relay before, during and for a few weeks after your cutover migration.