Weekly Product Update: Geolocation In Webhooks, Custom Variable Support And More

There’s a lot going on at Mailgun, so we thought we would start publishing a list of updates each week, with the new features, bug fixes and improvements we’ve made. Some of these will be backend improvements that you won’t see (but will hopefully feel), and others will be things you interact with, like the API or control panel. If there’s anything burning you’d like us to work on, let us know in the comments. Now, onto the updates:

• Special character support for basic webhook authentication via URL

• Better false-bounce detection

• New reason code description for failed messages

• Geolocation and User-agent parameters now in webhooks

• Recipient-level custom variables available in batch sending

Special character support for basic webhook authentication via URL

One of our most popular features is webhooks for tracking email performance (opens, clicks, bounces, unsubscribes and more). For security reasons, it’s a good idea to protect your webhook endpoint with basic authentication using a username and password. This week, we’ve rolled out support for special characters in webhook basic authentication.

Let’s say you want to protect your webhook endpoint with the following username and password

Username- michael@mystartup.com

Password- impossible-to-guess-password

Simply escape the special characters in your webhook endpoint. In this case, you’d use https://michael%40mystartup.com:impossible-to-guess-password@mystartup.com/webhook” as your webhook endpoint

Now your webhook endpoint is protected with basic authentication, even when your username or password contains special characters like @

You can read more about setting up webhooks in our docs

Better false-bounce detection

One of the things that Mailgun does to protect our customers sending reputation is prevent repeated attempts to email an address that has received a “hard bounce”. A hard bounce happens when you try to deliver an email message to an invalid email address (e.g paul@yahooo.com where Yahoo! is spelled with 3 o’s OR paul786752@yahoo.com where the user paul786752 simply doesn’t exist).

If you repeatedly try to deliver to these bad email addresses, you can get blacklisted by recipient ESPs who may mistake you as a spammer (spammers often attempt to deliver messages to randomly generated usernames at the major ESPs in hopes that they will actually deliver to some real accounts).

In practice, understanding when a “hard bounce” has occurred, though, is pretty tough. There is not a uniform set of conventions as to what constitutes a hard bounce across ESPs. Since we’ve delivered billions of emails, Mailgun has created an algorithm to detect these bounces and suppress future delivery attempts to protect your reputation.

Like any algorithm, however, false positives happen, meaning a legitimate email address is suppressed due to an obscure response from an ESP. This week, we’ve made some major improvements to our bounce detection algorithm that helps us detect 2% more false bounces than before. This means that fewer of your recipients will be mistakenly added to the bounce table, preventing future mailings. Ultimately, more of your email will be delivered.

You can read more about how we handle bounces in our docs.

New reason code description for failed messages

We’ve also introduced an improvement in failed message tracking which allows customers to track all the reasons that a message was not delivered, above and beyond just bounces. There are now 3 failure conditions that Mailgun reports on:

“Hardfail” indicates a hard bounce or recipient has previously bounced, unsubscribed or complained of spam.

“Maxfails” indicates a soft bounce and the maximum number of attempts was reached.

“Old” indicates that Mailgun tried to deliver the message unsuccessfully for more than 8 hours.

Here is a full list of failed message webhooks.

Geolocation and User-agent parameters now in webhooks

To make geolocation and user-agent detection simple, we’ve started posting additional parameters in our webhooks. With the new geolocation parameters in open, click and unsubscribe webhooks, Mailgun provides the following:

IP– IP address the event originated from.

Country– Two-letter country code (as specified by ISO3166) the event came from or ‘unknown’ if it couldn’t be determined.

Region– Two-letter or two-digit region code or ‘unknown’ if it couldn’t be determined.

City- Name of the city the event came from or ‘unknown’ if it couldn’t be determined.

In short, you still get the IP address, but you also get the most common information that can be derived from IP address, without any additional parsing or decoding on your end.

For user agent detection, you get

User-agent– User agent string of the client triggered the event.

Device-type– Device type the link was clicked on. Can be ‘desktop’, ‘mobile’, ‘tablet’, ‘other’ or ‘unknown’.

Client-type– Type of software the link was opened in, e.g. ‘browser’, ‘mobile browser’, ‘email client’.

Client-name– Name of the client software, e.g. ‘Thunderbird’, ‘Chrome’, ‘Firefox’.

Client-OS- OS family running the client software, e.g. ‘Linux’, ‘Windows’, ‘OS X’.

You can read the full details here.

Recipient-level custom variables available in batch sending

Batch sending allows you to email up to 1000 recipients in a single API call. This makes sending to large lists, 1 million or 10 million users for example, much faster. You’ve always been able to set custom variables for batch sending, but the variables were at the batch level, meaning you could set a custom category for the entire group (say, a campaign ID), but not a unique value for each member (say a hashed account number). Now you can.

This feature is extremely useful if you are really into analytics, since it allows you to set unique IDs for each recipient in your send that you can use to match up the recipient to a specific customer in your database, even when email address is not a unique enough ID (like when you want to track account-level activity, but each account has multiple emails associated with it).

We wrote about this on our blog this week and provide sample code so you can try it out easily. Check it out.

That’s it for this week! Let us know what you think and anything else you’d like to see.

Happy sending,

The Mailgunners