Postmark Blog

New Feature: Postmark Inbound Auto-Retries

mgornik — Tue, 01 May 2012 14:43:30 -0400

New inbound feature - retries

Since launching Postmark Inbound, one of the features that our customers requested the most is the ability to retry inbound posts. Lot of things can happen that can disrupt an inbound JSON post coming from Postmark to the inbound hook server. The web application receiving emails or the web server hosting it can go down due to various reasons. Or, networking issues can render your inbound hook unreachable from Postmark.

Whatever the reason for the failed POST, you shouldn’t ever miss an email. That’s why we are adding automatic retries to Inbound, to ensure your web app receives the processed inbound emails.

How does it work?

When an email is sent to your inbound mailbox, Postmark will retrieve it and process it. Immediately, it will try to post the generated JSON to the inbound hook URL you defined. If the post was successful, it will be listed as Processed. The post is considered successful when the inbound hook returns one of the HTTP 200 codes. This is the same behavior we had until now and the presentation in the UI is the same.

However, if the hook returns a non-200 code, we will now schedule the JSON post for a retry. The entry in the activity screen will show how many retries were attempted, and will show the timestamp of the next pending retry.

You can even filter your emails by using the drop-down filter, looking for “Retrying” emails.

A total of 10 retries will be made, with growing intervals from 1 minute to 6 hours. This is done so that if your web app has longer downtime, it still has a chance to receive all of the posts when it is back online. Only if all of the retries have failed, your Inbound page will show the message as Inbound Error.

How much does retry cost?

Retries don’t cost you anything - a single credit is taken for processing inbound email and includes all of the potential retries.

What is the next thing we are working on?

We are improving our infrastructure to produce much faster processing times for inbound emails. Another idea we’re working on is manual retries initiated from UI. If you have any suggestions on what you would like to see with Inbound, please let us know! You can comment here, send us your thoughts over Twitter (@postmarkapp) or email support.

Inbound Improvements: Better Contacts JSON & New Date Fields

alexknows — Fri, 06 Apr 2012 12:35:00 -0400

We have introduced some changes to the posted JSON object.

In a nutshell, here are the improvements:

All contacts (From, To, Cc addresses) will contain both email and full name (display name). Before this change, we only extracted email address, but having the the full name will be a useful addition. Email and full name are separated into two JSON fields, so no additional parsing on your side is required.
Emails with multiple recipients will be parsed on our side and passed with the JSON object as JSON arrays. Now you can easily traverse through all recipients, without any parsing.

The Date property is added to JSON, holding the timestamp of the original message (presenting local time of the message sender). This can be useful to fetch local time zone of the people sending messages to your web app, among other things.

The specifics of these changes are included in the developer docs for handling contacts and send date.

Executing a non-breaking release

To be respectful of your apps, we are going to keep all existing fields in posted JSON and only add new ones.

We added new fields with similar information to some of the existing fields, but they offer more information and have additional formatting.

For example:

"From": "myUser@theirDomain.com",
"FromFull": {
  "Email": "myUser@theirDomain.com",
  "Name": "John Doe"
},

This will cause some duplicate data to appear in posted JSON but will allow you some time to change over to using the new format.

We’re looking for specific use cases where the legacy format is useful. If your app includes one of those cases, please reply to this email and let us know.

Finally, many thanks to the developers who have quickly updated their code examples for Postmark Inbound in response to this update. If you’re interested in providing examples in other languages/frameworks, we can offer free credits in return for your contributions!

We’re really happy to be hearing from customers about how they’ve been using Inbound, so if you’ve got stories you haven’t told yet, please share them with us!

DevOps Deals with Server Density

alexknows — Thu, 05 Apr 2012 13:45:00 -0400

Our friends at Server Density are running a series of deals in the coming months highlighting the useful sysadmin/development tools that they use and love. We’re thrilled to be their first featured deal.

In return, they’re offering a year of free server monitoring for 1 server to all Postmark customers!

What it does: Server Density provides hosted monitoring, analytics, and alerts for your servers and websites. You can easily start monitoring the status and response times of your websites which can be correlated with server metrics like CPU, memory, disk and other applications like Apache, MySQL and MongoDB.

How we use it: We use Server Density to monitor server health and statistics (on OS and DB levels). And we also publish custom metrics to Server Density which are used to monitor our own parameters in Postmark.

The deal: Server Density is offering Postmark customers 1 year of free server monitoring for 1 server to new accounts.

How to get the deal: E-mail hello@serverdensity.com with your Server Density account URL and mention this blog post.

Lessons learned from our recent outages

cnagele — Fri, 30 Mar 2012 11:34:00 -0400

I owe an explanation for the recent issues on Postmark. This week we had some pretty terrible performance on our MongoDB cluster, resulting in delayed emails, disabled activity and even disabled bounce API. No emails were lost during the outages, but I do want to explain what happened, what we did, and what we are doing about it.

Cause and effect

In short, one of our MongoDB clusters failed, forcing a failover to a secondary machine as a primary. Unfortunately this new primary did not have enough resources (Disk IO) to keep up with the write load that Postmark throws at it. Since the other replica failed, we basically had to wait until it rebuilt to bring it back, which took over a day with the amount of data we had on that cluster. Once we failed over again, everything instantly came back to normal.

The issue that occurred is not something new. We’ve had challenges with MongoDB in the past. I would say that it is not entirely about how MongoDB works, but how it fits our needs and how we use it. Postmark captures a lot of data each day, which we store both in MongoDB for archival and Elastic Search for searching.

We go through so much data that in order to maintain the performance that our customers expect, we need to purge data constantly. With a solution like MongoDB, this causes two problems.

First, we have to constantly delete data, which adds to the disk utilization on each primary in the replicaset. Second, when data is purged, it does not release the space on disk, meaning that we have to run constant compactions or resync to get it back.

As a result of these problems, our Bounce API was down for almost two days. We chose to disable that part of the API because it uses MongoDB queries to find emails, which puts additional load on the servers. We decided that it was better to have email sending services in good shape if it meant the Bounce API was offline.

While embarrassing, I still feel this was the right decision. The other side affect of the performance issues are inaccurate “In queue” messages in activity. While all emails were sent successfully (although possibly delayed in some cases), the activity statuses were still being pulled from MongoDB data which reflected some inconsistency while it was suffering. We are slowly updating the activity history to reflect the properly sent messages and you can expect them to be back to normal within the next day or so. All new messages are being shown accurately.

A silver lining

One really good thing that came out of the performance issues is an updated Bounce API. The bounce API has always gone directly to MongoDB when searching for email records. Since MongoDB is not really meant for searching, it was very slow, and if you use it you probably had timeouts in the past. One of our developers, JP Toto, was able to rebuild it to make calls directly to Elastic Search, which results in near instant responses. This change will also allow us to enhance how you search for bounce (and sent) records in the future.

Rethinking our architecture

We’ve been on MongoDB for a long time. At this point we are rethinking our architecture and have a plan for some very thorough benchmarking of other nosql (and sql) databases to see how they might fit our needs. The most important part of the results of this benchmarking is making sure that our email archive cannot affect the real-time sending that our customers rely on. This either means an eventually consistent solution or heavy background processing, or a combination of both. We still have not ruled out MongoDB, but we have some ideas on how we can use it differently that might fit us better.

Since most of our customers are also developers, we’ll make sure to post our benchmarking results once we are done.

More than anything, I want to sincerely apologize for the problems. For the most part, we know we are doing our job when you forget we exist because everything “just works”. We heard from many of you because that wasn‘t the case in the past few days.

We‘ll make sure that the next time we hear from you, it’s on a positive note. Thanks for your patience and support.

A really rough Saturday

natalienagele — Sat, 24 Mar 2012 15:38:00 -0400

Today we were going to have a really important, but low impact maintenance. This morning at 9am the team brought the application offline to migrate to a new, more powerful database. Email sending was not going to be affected during most of the process, we expected about 2-4 minutes of paused sending. Things did not happen as we expected and we had some long outages and delays in sending. I want to explain what happened in detail.

What happened

The process to move to the new database meant we had to transfer data. We were moving the database to our datacenter in Chicago (from Virginia). When we came back online we noticed significantly worse performance. While we investigated what the cause could be, emails were sending but at a much slower rate.

To combat the slowness, we added many more workers and about 20 minutes later sending was back to full speed. At this time we realized something strange was going on. Our test accounts that we created after the migration were reflecting inconsistent data. As we continued to investigate, we realized that the transfer of the old to new DB was incomplete. This was totally unexpected and took a while to realize. During this time, all accounts were not affected and sending chugged along very quickly.

We decided that the best way to fix the issue was to reimport the old DB dump. We had to put the app in emergency offline mode and run the import. After about 15 minutes it completed, but was still missing some records. We decided the only solution at that point was to revert back. About 10 minutes later the app was back online and sending was instant.

The damage

By reverting back we lost some data from today between 10am EST and 2:30pm EST. This includes a some new account sign ups as well as any credit deductions and sent email count. However, the sent records can still be seen in the activity page.

The good news is we didn’t lose any emails, and all emails are in Activity for searching.

What we’re doing now

Russ and the team are investigating what happened. In all our testing we weren’t able to predict that the sync would be incomplete. Until we know for sure, we will hold off on the migration. Our hope is to try again next weekend. Before we do, we will send an email notice about the maintance so you’re fully aware of any upcoming downtime.

We are very aware of the affect this kind of day has on our customers and their businesses. We use Postmark ourselves and rely on it’s uptime. We’ve gotten to a really good place testing and most days you never hear of any of our deployments or updates. On behalf of myself, Chris and our team, we’re very sorry. We will continue to work towards making sure these kinds of situations don’t happen again.

Emails getting caught in Gmail Spam? Find out why using...Gmail.

alexknows — Fri, 23 Mar 2012 14:14:00 -0400

You know that sending your transactional email through Postmark is the best way to improve your delivery rates. But sometimes, even with the best of tools and intentions, your emails will end up in spam and you won’t know why.

The next time you’re working to troubleshoot why an email is going to your (or your customers’) spam filter in Gmail, you can click on the email itself and look right below the sender and recipient information for more information about why Google thinks that email is spam.

These are just a couple of examples from my own spambox. Other options may include:

We’ve found that lots of messages from xyz@xyz.com are spam.

and

It contains content that’s typically used in spam messages

Even when you’re sending through Postmark, it’s a great idea to make sure that you are set up to periodically receive copies of your transactional emails, and take special notice when you make copy changes. If emails start going to spam, you can use this new Gmail feature to try to determine why.

Of course, if you’re ever stumped as to why an email sent with Postmark isn’t being delivered or is landing in the spam filter, Postmark’s expert support team is ready to help you diagnose the problem.

Two ways to reduce bounced welcome emails

alexknows — Wed, 21 Mar 2012 13:48:00 -0400

We encourage our customers to think of Postmark is a “good senders club” for transactional email. ISPs base part of a sender’s reputation on things like spam complaints and bounces and Postmark helps you make sure that you can keep those rates low in order to keep deliverability high for everyone.

A common technique in email marketing services is to help customers with “list hygiene”, largely for the same reasons. An ESP will often provide tools to help their customers scan the list for emails that are likely to generate bounces. This means that the tool will highlight emails with commonly fake or likely misspelled domains.

Transactional email requires a different approach, because a user signing up for a service is much less likely to be using an intentionally fake email address. They are, however, likely to have typos.

A common technique to avoid email typos is to have the customer type their email in twice and verify, but that’s extra work that can slow a user down during signup. Instead, we recommend doing a little bit of extra work to help your customers sign up quicker and still proactively catch email typos.

Here are couple of techniques for catching and fixing typo’d emails for your users in a way that’s likely to surprise and delight them.

Client-Side Suggestion

A brand new jQuery library authored by the developers at Kicksend allows you to specify a set of domains that are often mistyped, and it will catch domains that are similar enough to be a typo and suggest the correct domain to the user.

The library does require that you bring your own list of domains, which you can get ideas for from browsing your bounces in the Postmark activity filter.

This library - dubbed mailcheck - is new, so we strongly recommend testing it well before implementing it. Grab it on Github here.

Postmark Bounce Hooks

Have you ever had a user who didn’t get their confirmation email, only for you to realize after investigation that it was because they had a typo in their email address? Or what if someone changes their email address and forgets to update it in your app, and then stops getting important notifications?

If you use Postmark for your emails, we’ll detects bounces and their cause, and provide information back to your app when the bounce is caught via a webhook, where you can parse information about the bounce from a JSON post. This information can be used any way you like, but one of the ways we recommend using it is to provide useful information to your logged in users.

As an example in Beanstalk, we parse bounces and notify users that are logged in about the bounce. We even make recommendations based on the bounce reason and recommend updating the address in Beanstalk while they are still logged in.

If the user logged in is an account owner, we let them know about any addresses in their account that bounced recently.

With just a little bit of work we’re able to reduce customer support by a lot, and help keep our customers happy and informed.

Take a look at our bounce hooks API to set this up.

Less bounces = happier customers

Remember that reducing your bounce rate isn’t only a best practice for being a Postmark customer, but your customers will appreciate you being thoughtful and helping them catch their mistakes. Reducing their confusion and frustration will result in them spending more time enjoying your application, and little experiences like these help a lot!

New Code Examples Postmark Inbound in PHP & Python

alexknows — Mon, 30 Jan 2012 11:26:16 -0500

Hot on the heels of the Node.js example we shared last week, we recently had two new “mitt” libraries contributed to make it easy to process JSON data sent in Postmark Inbound’s webhooks.

NEW PHP Mitt:
Joffrey Jaffeux has built a well documented PHP mitt, and has even included instructions for using this code to process inbound emails in CodeIgniter projects.

NEW Python Mitt:
Inspired by Joffrey’s contribution, Jose Padilla developed a similar project for using Postmark Inbound in Python.

And don’t forget that there is still the Postmark Inbound rails gem developed by Randy Schmidt.

Many thanks to both of them for their contributions!

We are actively looking for your contributions for other languages to add to our developer documentation. If you write a code sample or library for Postmark Inbound, please email us at support@postmarkapp.com. Contributions are appreciated and quality contributions are rewarded with free Postmark credits.

We have a low-traffic API developer email list that you can join as well.

Learn to use Postmark Inbound with Node.js & CouchDB

alexknows — Fri, 20 Jan 2012 18:49:27 -0500

Since our launch on Tuesday, we’ve gotten lots of great questions and comments on Postmark’s new inbound email parsing service. Since, Mark Headd made a great open source contribution that we’ll be adding to our docs: a Node.js listener for Postmark Inbound that stores emails parsed by Postmark in CouchDB.

Even cooler than just open sourcing the code, Mark went above and beyond and recorded a handy screencast to walk you through what he built and how to use it - including the handiness of the way CouchDB handles serving up attachments (watch the video, you’ll see what I mean).

Big thanks to Mark for his contribution, and going the extra mile with the screencast which I’ve embedded below.

If you’d like to earn some free Postmark credits, email me and let me know what you’ve built that can help other people use Postmark Inbound easier! Code samples, screencasts, walkthroughs - the more useful and creative your contribution is, the more credits you’ll get!

Discuss this on HackerNews.

Introducing Postmark Inbound: Easily parse replies & other incoming email into JSON for your app

alexknows — Tue, 17 Jan 2012 04:24:00 -0500

Postmark started by solving the pain of sending emails from your web applications. Today, we’re closing the loop by making it just as easy for everyone to be able to parse incoming emails.

You already know that if your web app sends email, it’s sending it more reliably with Postmark. But wouldn’t it be great if your web app knew how to listen for incoming emails? We thought so too, that’s why we created Postmark Inbound.

Every Postmark server now includes an inbound email address. We’ll turn anything you send to this address into a beautifully formatted JSON object and post it to your application over HTTP, complete with email contents, headers, even attachments.

Postmark inbound is perfect for:

Replying to in-app messages.
Analyzing and acting on email content.
Creating tasks, reminders, and posts.
Using email to upload files.
And more!

We’ve worked hard to keep Postmark simple, including how you pay for it. Just pay for what you use, starting at $1.50/1000 emails. The same Postmark credits can be used for outbound and inbound email processing.

Try Postmark Inbound today

Postmark Inbound is available for all Postmark customers and new accounts immediately. Our developer documentation is available to help you get started.

You can also learn how popular issue tracker Done Done and our own hosted version control system Beanstalk use Postmark Inbound every day.

Enjoy!

P.S. Are you as excited about this as we are? Share that excitement on HackerNews!

We should have an emissions check for ESPs

natalienagele — Wed, 14 Dec 2011 16:32:55 -0500

Chris’s posted yesterday about the problems with selling dedicated IPs and along with the comments, it got me thinking about what we, the industry of Email Service Providers, are responsible for. At Wildbit, we feel an obligation to rid this world of spam and junk mail. It’s not like it’s a higher calling, it’s just the right thing to do as an email service provider (ESP). ESPs are the gateway to a lot of junk mail, not just spam mail. It’s the junk mail that’s difficult to police but be believe can and should be done by every ESP.

Junk mail vs. Spam mail

We talk a lot about spam in the email industry, but we don’t talk much about what I consider “junk mail”.

Like the random take out restaurant flyers you get in your physical mailbox, junk mail is the catalogue equivalent for email. These are newsletters and announcements and marketing emails that are mostly useless to many people, but don’t necessarily fit the definition of spam. You probably signed up for a list at some point, but the list owner is overstepping with the quality or quantity of their mailings. We dealt with this a lot when we ran our email marketing service. What’s important is that ISPs are now taking a stance against this type of email by keeping track of whether these get opened, read, etc (engagement).

Now, this junk mail does generate some spam complaints, but often not tons. Therefore ESPs allow these emails to go out day after day. Why? Because it’s not causing global delivery issues (at least not right away). These emails just get marked as spam, but anyone else sending through that IP won’t necessarily get flagged as spam.

Our view

At Postmark, we don’t deal with this problem nearly as much as we used to with our email marketing service. Most transactional email is expected and triggered by a user’s specific action. Even with that we keep our spam threshold extremely low. Some emails still trigger spam complaints, commonly things like invite friend tools, auto-responders, etc. We keep a strict eye out for that kind of activity and ban it from Postmark if it becomes abusive.

Strict? Maybe. There’s a fine line between what the marketing department wants to accomplish and what a customer or recipient wants and expects. Sending the same automated email 3 times a week is probably going to piss people off.

That’s junk mail, and we don’t stand for it.

Emissions Testing For A Better (Emailing) Environment

ESPs are polluting the environment allowing people to send this junk. There are so many groups and tools out there to block the spam after it happens, but I’m not seeing a lot of ESPs working on stopping the junk from being sent in the first place. The punishment to an ESP for sending junk just isn’t as high as sending spam, so everyone concentrates on keeping the African princes away, but not the retail newsletter that got sent out every day last week.

There’s a lot of money to be had in sending junk mail (both paper and electronic). We think that there should be more of an effort made by the ESPs to do better than the minimum requirements set by ISPs. Think of it like car companies getting awarded for having lower emissions than the minimum requirements. They are actually working to help the environment, not just keep it from getting worse.

The false promises of dedicated IPs

cnagele — Mon, 12 Dec 2011 15:01:25 -0500

I have to admit, I am disappointed in the false promises that companies are giving when it comes to offering dedicated IPs for their customers to “improve email delivery.” The reality is, dedicated IP addresses are not a sure way to improve delivery, and sometimes, can actually hurt email delivery. Before I jump in, let me first explain how IP addresses play a role in mail servers and sending email.

When confronted with the task of setting up an outbound mail server, there are a lot of things you have to consider. The first and foremost is making sure that all of your outbound emails are always sent from a consistent public IP address. This IP becomes the unique identifier that ISPs, accreditation services and blacklists use to track your activity and your reputation as you send. Once you have a dedicated IP, you can then continue the process of making sure you have proper reverse DNS records, adding it to your SPF records, setting up feedback loops and so on. I wrote a short guide on this back in 2007, most of which has stayed the same.

When it comes to Email Service Providers (ESPs) like Postmark, we go through this process on behalf of our customers and use a range of dedicated IP addresses that are shared amongst all of our customers. The other approach is to provide a dedicated IP address for each customer. I think this is a bad idea.

Why offer a single customer their own dedicated IP?

It’s pretty common for ESPs to offer dedicated IPs to a single customer. When we first launched Postmark this was one of the options (as a coming soon) when you created a new server. The idea behind a dedicated IP is to isolate reputation, throttling and blacklists to each customer. This helps with diagnosing issues and can avoid one customer killing the reputation of another if something goes wrong. The other advantage is that the customer can create a custom sub-domain for the dedicated IP, essentially looking like it is coming from their own servers.

From the ESPs side, it is a point of protection as well. If a customer decides to start spamming and the IP gets blocked, it will not hurt other customers on the system (only half true - covering this next).

Why offering dedicated IPs is a bad idea.

A dedicated IP sounds like something exclusive and attractive when customers look at what ESPs offer. In reality though, it’s a means of retreating from the overall responsibility of being an ESP.

By offering a dedicated IP for the majority of customers the ESP is basically saying “You do what you want, if you get blocked it’s your fault.” It also places a lot of heavy lifting on the customer, which defeats the purpose of paying for an infrastructure product in the first place.

In addition to this, new dedicated IPs are just as bad as IP addresses with a bad reputation, since it has no reputation at all. In order for an ISP (Gmail, Hotmail, etc) to trust email traffic, they need to know your history. If an ESP just gives you a new IP and you start sending a bunch of email through it, you’ll need some serious good luck. In most cases a new IP address get’s “warmed up” first. Basically you take a new dedicated IP address and send a small number of emails out over time, slowly increasing it each day until it has a good reputation. If the ESP does not do this for each new dedicated IP, the customer is going to have problems.

The other misconception with dedicated IP addresses is that each one is completely independent. For instance, if one customer gets blocked, all other IPs are fine, right? Wrong. ISPs and blacklists will monitor entire IP ranges and domains. If one IP causes enough problems, traffic from the entire subnet or domain could be blocked.

The final reason, and this one is important, is that ISPs are starting to place a lot of weight on domain reputation, not just IP reputation. My guess is that over time IP reputation will slowly fade away while more weight is given to domain reputation along with authentication standards like DKIM. This allows you to take your reputation with you and ties the reputation to the brand (your domain) which is a lot less disposable than an IP address. In a recent post from Laura Atkins, a highly respected consultant in the industry, she stated:

Domain reputation is where delivery is going. And I think a lot of senders are going to struggle with delivery as they find that IP reputation is not enough to get into the inbox.

We feel the same. And that is why we push DKIM and SPF so much in our setup process. It’s also why we are thrilled that over 60% of our customers use DKIM or SPF on our service, a huge accomplishment for us.

Why we decided to stick with shared IP addresses.

For the reasons above, we decided earlier this year to remove dedicated IPs as a feature. Our goal at Postmark is to make email delivery dead simple. Developers, designers and system administrators already have enough to deal with and email delivery should not be one of them. We also have a very acute focus on our email delivery rates. If emails are not getting to the inbox, using Postmark is not justified, no matter what additional features we offer.

To make sure our delivery is rock solid it’s our job to take 100% accountability of our IP addresses and our reputation as a sender to the ISPs. So instead of offering a dedicated IP address to a customer and send them off, we carefully and cautiously monitor a set of shared IP addresses for all customers. This helps in several ways:

We have a very high volume across each IP address, which gives a proven history of good sending practices with each ISPs. High volume with great sending practices means fantastic inbox rates.
We avoid the problem of constantly warming up new dedicated IPs, risking delivery issues for new customers. A customer can hit the ground running.
We take full responsibility for the reputation of our IPs, giving us extremely high reputation and delivery rates.
We can encourage our customers to use DKIM to focus on domain reputation, instead of IP reputation.

This also means we actively kick customers out if they are not following our sending guidelines or terms of service. We have a very low tolerance for spam complaints, 10 in 10,000 emails. If a customer throws any alerts and does not clean up the problem we act fast. Some might think our tolerance is too low. However, spam complaints should be almost non-existent for most transactional email, and anything that could hurt our reputation is not worth the extra money that the spammy customers bring us.

Joining Postmark is like being part of the “good senders club”, with all the benefits of emails to the inbox.

When dedicated IPs make sense.

Obtaining a dedicated IP (or many of them) can still make sense, but the decision is directly related to volume. For instance, we can’t just send on a single IP address. There is a point where too much volume can cause throttling at the ISPs, so you need to spread out that traffic across multiple IP addresses. In some cases we will recommend a dedicated IP address for customers who send an extremely large volume. This only works when the customer sends a lot of emails to all of the major ISPs. If you send a ton to Gmail but not Hotmail, you’re going to have issues delivering to Hotmail. Each ISP can only build a reputation on what they receive. Another benefit of a dedicated IP is getting accredited from services such as Return Path and ISIPP, allowing customers to track their individual reputation. In reality though, the majority of customers never have to worry about this. They leave it to us and we take care it.

Summing it up

While a dedicated IP address for an email service might sound sexy, it’s not a true solution for getting to the inbox. ISPs only care about proper sending practices and a solid reputation. Instead of requiring every customer to build that reputation from scratch we feel it is better to allow approved customers to leverage our reputation, helping them get to the inbox immediately and without the hassle or care of how it works. Sure, this means we spend more time policing our customer’s sending practices and kicking the trouble customers out, but that just means better email delivery for the rest of you.

Over 61% of Postmark customers authenticate their emails. Do you?

natalienagele — Wed, 07 Dec 2011 12:51:33 -0500

I don’t think we can stress enough the importance of authentication in sending email.

Every day we find customers who have delivery issues that are instantly solved by updating SPF records or signing their emails with their own Domain Keys. Because we know how critical it is, we’ve built it right into the Postmark setup process. Interestingly, we actually have customers who think it’s a required step to use Postmark, and we don’t think that’s a bad thing.

Amazingly, over 60% of all Sender Signatures of our customers have DKIM and/or SPF setup correctly.

And of course, custom DKIM comes with any account for free, whether you are using the first 1000 credits or you’re a high volume sender.

Sign your emails, remove the doubt

ISPs have a lot of tools at their disposal to check if what you’re sending to them is legitimate mail. As they continue to evolve, they get smarter and smarter at making that important distinction.

The new word of the day is engagement, the idea of ISPs watching what customers actually do with your email. For instance, whether they open it or send it to trash right away. If they open it, how long does it stay open?

All of this is meant to protect the recipient from not only Viagra email, but also useless junk mail. You can avoid a lot of delivery problems by simply telling the ISP who you are. Engagement, content scanning, etc are tools that ISPs use if they have doubts.

Your job, remove the doubt. And we’ve designed Postmark to help you do that.

The “why” to signing your emails

When an email is sent to the ISP, they:

check the IP reputation
check the domain reputation
check the content

We’ve got the IP reputation covered for you, but you are in control of the other two.

Content is the really tricky one, so you at least want to make sure the other two are rock solid. Your domain reputation is 2 parts being a good sender and 1 part making sure the ISP knows who you are. When you use an ESP, that means telling them that you have allowed us to send emails on your behalf.

What’s nice about transactional email, our speciality here at Postmark, is that most times you don’t need to worry about the tests of engagement, so long as you’ve shown the ISP that you are following the rules. Signing your email is key here, because once it’s signed, you’ve taken a lot of the doubt away. For example, a spammer can try to spoof @adobe.com. What they can’t do is access Adobe’s DNS, so they’ll never be able to sign their emails. When AOL sees an email from Adobe that’s not signed, it’s easier to make the decision to trash it.

“But I need to send on behalf of my customers”

If you’ve been whitelisted to send from many domains, you probably heard from our team that we don’t recommend it, but we’ll do it if you need it. The reason we advise against it is that you end up sending email without authentication.

When you send from your customer’s email address, you’re not going to be able to sign that email with DKIM. Instead we’ll sign it with our own DKIM, but that comes with issues as well - like GMail’s “via” tag for example.

Our recommended approach is to send from your own email, and then change the From Name and Reply To to be the customers. For example:

"Bob Smith" info@yourcompany.com, reply to: "Bob Smith" bob@smith.com

Takeaway

Getting to the Inbox is really hard already. As a company, we want to make it easier, but we can’t do it all for you. We take a really strong stance on recommending email authentication to our customers in Postmark and we’ve designed it into our workflow to make it simple. With what we’ve seen, you’re doing yourself a huge injustice without it. ISPs, even smaller ones, are starting to make this a requirement for getting through.

Do yourself and your recipients a favor and just follow the simple rules. It’s already a difficult job. Not using DKIM yet? Go to your sender signatures page and follow the steps or sign up for a Postmark account and see how easy it is.

Postmark Spam Score API in the Wild - Spamscore.me and Christmas is Saved on Reddit

alexknows — Wed, 16 Nov 2011 11:03:00 -0500

A few weeks ago we released a free API for checking the spamscore of a message. The folks on HackerNews loved it, and it’s been really great to see people find ways to get more value from this API.

Some of our favorite usages have shown up in just the last week!

Spamscore.me

We released our Spam Checking API with an easy and beautiful UI, but it does require that you grab the full email contents including the headers. Simple as this is, it’s work. Wouldn’t it be great if you could just forward an email somewhere, and moments later, get back a nicely formatted report about your email’s spamscore?

That’s exactly what Jim Carter built with spamscore.me. The cool thing about spamscore.me is there’s no interface other than your email client! Just forward any email to go@spamscore.me and get one of these bad boys back.

It’s free (but no less awesome), so please be kind to Jim’s servers. Have other ideas for useful mashups with our spamscore API? Let us know in the comments, or better yet email me once you’ve launched so we can write about it here!

Christmas is saved on Reddit

Our hearts were warmed to find that someone recommended our spamscore checker to a Reddit user whose christmas eCards were being sent to spam.

Chris was able to jump in and do a quick analysis of the person’s spamscore report, recommending the following to help with the resolution:

Set your Return-Path header to a domain and address that you control. Then, add an SPF record to that domain with the permitted SMTP servers.
Do not use your user’s email as the from address. This is spoofing and you can’t control the DKIM or SPF. Instead, use their “From name” but your email address.
Make sure Reverse DNS is setup for your SMTP server.

While we can’t promise personal spamscore report interpretation for every problem, we’re thrilled to see that this tool is being used to help solve real world problems in people’s applications.

Have you used the spamscore API to successfully track down a problem? Let us know in the comments.

New Postmark System Status Dashboard

alexknows — Wed, 02 Nov 2011 12:34:00 -0400

We work very hard to keep Postmark reliable and available for our customers, but sometimes things go wrong that are out of our control. We do our best to communicate the details of any issues we’re having, and make it a priority to prepare you for any planned maintenance or downtime.

Not everybody follows us on Twitter, and emailing our entire customer base isn’t always the best solution either. So today, we’re launching our new Postmark System Status page.

We’ll continue to post updates to Twitter and provide our in-app notices. We’ll still email you ahead of time for planned maintenance. But if you’re ever experiencing a problem and want to know if you’re the only one, you can now visit status.postmarkapp.com and look for recent updates about any known issues. If something is going on, the big banner on top will make that clear and the status updates below will provide more detailed information.

You can also subscribe to these updates via RSS in your favorite RSS reader, or even subscribe to email updates in your inbox if you prefer.

This is just one more step we’re taking to make sure that you know we’re working hard to provide an easy, fast, and reliable service for delivering your transactional emails to the inbox!

What Happened This Morning

cnagele — Thu, 20 Oct 2011 15:39:47 -0400

This morning at about 8:15am EDT we had an outage on our primary MySQL server. It affected sending, the API and our web interface. No emails were lost, but you would have noticed delays. I want to give a brief overview on what happened, how we recovered and where we are now.

The outage occurred when our primary server ran out of disk space. We have systems that monitor for things like this on each server to alert us. In this case, the space grew much quicker than normal, so by the time the alerts were noticed MySQL was already struggling.

In most cases when we have an issue with MySQL the impact is minimal. We run a high availability environment with MySQl-MMM, which usually detects any problems and automatically fails over to a healthy server. Unfortunately in this case MySQL was still running - it was just having issues writing certain data to disk, so it never got the trigger to fail over.

We struggled a bit to free up some logs and space and after about 30 minutes decided to just fail over to the second server. In hindsight we should have done that right away, but we wanted to make sure replication was healthy first.

Once we failed over manually, things were back to normal and we just had to recover any failed jobs. We have a nice fail safe in Postmark where if the DB is unavailable we queue emails locally on each server so we can successfully send them when we recover. This came in handy today!

Moving forward, we’re working on some upgrades to the cluster that will focus on availability and performance. We’re also going to make the alerts a bit more sensitive to make sure we catch potential issues sooner.

I’ve very sorry for the early morning chaos. It was definitely not the way I wanted to start my day and surely not how we wanted all of you to start your day either. As always, we’ll work hard to minimize issues like these and improve reliability. We know you depend on us, and thank you for your patience and trusting us with your email delivery needs.

Checking Spam Scores is now as easy as Postmark

alexknows — Mon, 17 Oct 2011 11:10:00 -0400

Postmark’s sending infrastructure works to carefully, quickly, and accurately deliver your transactional emails to your customers’ inboxes. The most common problems that occur outside of our configurations are related to content, and our customer service team often works with customers to improve dodgy email content that’s causing emails to land in the spam filter.

Today we’re announcing a new easy and free API to help you score the quality of your outbound (and inbound) emails. This JSON API provides easy and fast programmatic usage of the spam filter tool SpamAssassin.

Simply POST an email’s contents, all headers included, against our API and we’ll return a score. If you want, we can also return a full report in your call.

Not API savvy? No problem! Our landing page for the API provides a fully functional scoring & reporting tool for you without having to ever write a line of code.

Write an API wrapper, Get Free Postmark Credits

This API is silly-simple, but we want to make sure that it’s useful. One way to do that is to make sure that there’s a simple interface to it in everyone’s favorite programming languages & frameworks.

We’ll give 10,000 free Postmark credits to the first person to write and open source a complete API wrapper for our Spam Score API in a given language/framework. We’ll compile a directory of the wrappers and publish them along with the API.

Our own Oren Mazor (who wrote the API) has provided an example library in Python, if you’re looking for patterns to follow.

To submit an API wrapper, email a link to your repository to alex@wildbit.com. We’ll provide the free Postmark credits to one winner per language/framework, based on the timestamp of the submission.

HackerNews Discussion

We “soft” released this on Friday night and there was some great discussion on HackerNews, spending most of the weekend on the home page. Jump over to the comments thread to see what people think!

Major Performance Fix in Postmark's Activity Search

natalienagele — Tue, 27 Sep 2011 13:24:00 -0400

As too many of our customers know, we’ve had an ongoing battle going on with our search functionality in the Postmark Activity pages. With the rapid growth of Postmark compounded with storing more days of “Sent” message activity (we’re now storing 15 days instead of 10 days), we’ve been working hard to refactor our activity search to - well, actually work quickly and consistently, for quite a while.

Our largest senders, and even some of our medium sized senders, have consistently seen time outs when doing a search on their Postmark Activity pages.

This morning we released the newly improved search algorithm, and it’s performing extremely well. Searching our largest accounts takes just a few seconds, which means most accounts respond to search queries almost instantly.

What Changed?

Milan, who’s responsible for all of the improvements, will provide more details on how the search parameters have changed in a future post. For the most part, there are no differences to what you are used to. Just like before, you are able to search for a recipient, as well as the subject of the email. This is very helpful for troubleshooting delivery issues, inactive recipients, etc. Searching your Activity checks for successfully sent emails along with, bounces, spam complaints and SMTP API errors.

Speaking of SMTP API errors, we did include a small change to the way we will be storing these in your accounts. Prior to today, we would store SMTP API Errors indefinitely. These are generated when you use the SMTP endpoint, informing you that we didn’t send your email. The 2 main causes of this are: invalid Sender Signature or you’re trying to email someone that has been marked inactive (due to a hard bounce or spam complaint).

In both of those cases, you don’t need to know about these errors forever. We actually got a lot of requests to remove these from accounts to clean them up. So, from today on, we will be removing SMTP API Errors from the Activity feed after 15 days, just like sent emails. This will help with the UI as well as the search functionality.

Thanks!

We’re really excited about this update because we rely on search just like our customers. Having it reliable is crucial to troubleshooting, which is what we’re here for as a service. We just want to thank our amazing customers for their support while we completed this important fix.

Thanks so much, you are so awesome!

Wordpress Plugin Updated!

alexknows — Tue, 20 Sep 2011 11:48:55 -0400

We’ve updated our Wordpress plugin that was announced last month to incorporate a number of important bug fixes, including proper support for HTML emails. Thanks to the customers who reported these bugs.

We’ve also added a standard Wordpress license to our plugin to comply better with community standards and expectations.

And of course, a big thanks to our plugin maintainer - Andy Yates - for all of his hard work.

The plugin can be easily auto-updated from within Wordpress or installed from the Wordpress plugin codex.

Wordpress SMTP Problems are solved with Postmark.

alexknows — Thu, 11 Aug 2011 10:08:00 -0400

If you’ve ever been responsible for a Wordpress site but not been in control of the server you’re hosting it on, you might have run into issues where important emails - welcome emails, password resets, comment notifications, and more - didn’t send at all. This is usually because the server’s admin has disabled the default SMTP server, or perhaps disabled the PHP mail() method that Wordpress uses to send mail.

Or if you’re on shared hosting, you may have found these emails were sending after all - they were just going to your spam folder.

Today, we’re launching a Postmark Approved Wordpress plugin to make all of these problems a thing of the past.

Simply download our plugin from the Wordpress Plugin Directory or search for “Postmark Approved” from inside your Wordpress admin for the easy one-click install.

Once installed, simply plug in your Postmark API Key and the email address of a verified sender signature, and you’re ready to go. We’ve even made it easy to send a test email to make sure it’s working before you click enable.

Why go “approved” when there’s other options out there?

The biggest challenge in having “unofficial” plugins in the wild is debugging problems when they arise. It’s also hard for us to know about how people are using the plugins, and what’s good or missing. We’re hoping that being closer to an “approved” plugin will allow us to keep a closer eye on the problems that Wordpress developers have, and evolve the plugin as well as Postmark to better serve the massive and growing Wordpress community.

As Postmark features evolve, we can help keep the plugin fresh as well.

Postmark has grown in part thanks to the support of dozens of the people who’ve created and shared Plugin & API wrappers for their favorite languages and platforms. We’re thankful for everyone’s hard work, and continue to work with third party developers to help deliver the best tools for working with Postmark.

Feature requests? Hit us up in the comments, or email us.

A Special Thanks

We invited Andy Yates, who we knew from his work on Magic Bean (a 3rd party iPhone/iPad app for our other product, Beanstalk), to lead the development of this plugin with our guidance and assistance. Always vocal supporter of Wildbit products, we’re immensely appreciative of Andy’s effort dedicated to this project.