How #PhillyTech grappled with an Amazon Web Services outage - Technical.ly Philly

Software Development

How #PhillyTech grappled with an Amazon Web Services outage

Devs across town spent most of Tuesday fretting due to a regional S3 outage in the popular online platform. Here's how some companies fared.

Not that kind of cloud.

(GIF via Giphy.com)

A Slack tipster — namely Engine Room dev Mike Trischetta — slid into my DMs yesterday with a relevant question:

mike comment

As it turns out: it did.

Amazon’s Simple Storage Service (S3) platform, used by companies like Netflix and AirBnB, had a serious crash yesterday afternoon. Per the site’s status page, there were “Increased Error Rates” present in the US-EAST-1 Region (based in North Virginia.)

As you can imagine, the outage made Tuesday a bit of a day for some of Philly’s tech firms, many of which rely on S3 — or services that also use it.

“Until [S3] is back up our clients are experiencing delays loading their data,” said Stitch CEO Jake Stein of the outage. “Fortunately, we’ve set things up such that no data is lost when there are outages like this. In addition to our product, a lot of the tools we use rely on S3, so it’s definitely impacting our productivity.”

For Women in Tech Summit organizers (more on that event later this week,) the outage knocked out their ticketing sales.

To this, Ticketleap CEO Tim Raybould jokingly quipped:

However, Ticketleap didn’t survive the outage unscathed: Raybould told Technical.ly that images on the company’s event pages weren’t loading and order confirmation emails were interrupted, but orders were still going through.

MailChimp, the broadly-used email blasting service, had trouble loading images onto newsletters.

This inconvenienced even the ever-serene Charlotte Lee, Benjamin’s Desk‘s director of programming.

charlotte

Code for Philly comms lead and GIF connoisseur Pat Woods ran into some trouble on Slack.

Also heartbroken was former Technical.ly Delaware lead reporter Rana Fayez, who pivoted to the dev field.

CloudMine VP of Product Nick Borth said the firm hosts HIPAA compliant websites and backend storage in S3, so backup automation and the company’s failover strategy pretty much saved the day.

“This approach significantly lowers the risk of compromising data, and maintaining availability for our clients and partners in sensitive industries,” said Borth in an email.

The outage ended around 6:00 p.m., but before then, one solution was offered by Lancaster, Pa.- based entrepreneur Dave Conklin:

 

-30-
Subscribe to our Newsletters
Connect with companies from the Technical.ly community
New call-to-action

Advertisement

Technically Media

Sign-up for daily news updates from Technical.ly Philadelphia