Software Development

How #PhillyTech grappled with an Amazon Web Services outage

Devs across town spent most of Tuesday fretting due to a regional S3 outage in the popular online platform. Here's how some companies fared.

Not that kind of cloud. (GIF via Giphy.com)

A Slack tipster — namely Engine Room dev Mike Trischetta — slid into my DMs yesterday with a relevant question:

mike comment

As it turns out: it did.

Amazon’s Simple Storage Service (S3) platform, used by companies like Netflix and AirBnB, had a serious crash yesterday afternoon. Per the site’s status page, there were “Increased Error Rates” present in the US-EAST-1 Region (based in North Virginia.)

As you can imagine, the outage made Tuesday a bit of a day for some of Philly’s tech firms, many of which rely on S3 — or services that also use it.

“Until [S3] is back up our clients are experiencing delays loading their data,” said Stitch CEO Jake Stein of the outage. “Fortunately, we’ve set things up such that no data is lost when there are outages like this. In addition to our product, a lot of the tools we use rely on S3, so it’s definitely impacting our productivity.”

For Women in Tech Summit organizers (more on that event later this week,) the outage knocked out their ticketing sales.

To this, Ticketleap CEO Tim Raybould jokingly quipped:

However, Ticketleap didn’t survive the outage unscathed: Raybould told Technical.ly that images on the company’s event pages weren’t loading and order confirmation emails were interrupted, but orders were still going through.

MailChimp, the broadly-used email blasting service, had trouble loading images onto newsletters.

This inconvenienced even the ever-serene Charlotte Lee, Benjamin’s Desk‘s director of programming.

charlotte

Code for Philly comms lead and GIF connoisseur Pat Woods ran into some trouble on Slack.

Also heartbroken was former Technical.ly Delaware lead reporter Rana Fayez, who pivoted to the dev field.

https://twitter.com/MeetRanaFayez/status/836648959217106944

CloudMine VP of Product Nick Borth said the firm hosts HIPAA compliant websites and backend storage in S3, so backup automation and the company’s failover strategy pretty much saved the day.

“This approach significantly lowers the risk of compromising data, and maintaining availability for our clients and partners in sensitive industries,” said Borth in an email.

The outage ended around 6:00 p.m., but before then, one solution was offered by Lancaster, Pa.- based entrepreneur Dave Conklin:

 

Companies: Engine Room / Stitch, Inc. / 76 Forward / Amazon / Ticketleap

Before you go...

Please consider supporting Technical.ly to keep our independent journalism strong. Unlike most business-focused media outlets, we don’t have a paywall. Instead, we count on your personal and organizational support.

3 ways to support our work:
  • Contribute to the Journalism Fund. Charitable giving ensures our information remains free and accessible for residents to discover workforce programs and entrepreneurship pathways. This includes philanthropic grants and individual tax-deductible donations from readers like you.
  • Use our Preferred Partners. Our directory of vetted providers offers high-quality recommendations for services our readers need, and each referral supports our journalism.
  • Use our services. If you need entrepreneurs and tech leaders to buy your services, are seeking technologists to hire or want more professionals to know about your ecosystem, Technical.ly has the biggest and most engaged audience in the mid-Atlantic. We help companies tell their stories and answer big questions to meet and serve our community.
The journalism fund Preferred partners Our services
Engagement

Join our growing Slack community

Join 5,000 tech professionals and entrepreneurs in our community Slack today!

Trending

Penn dean is a startup founder and ‘engineer at heart’ who loves the connection between education and business

Every startup community wants ‘storytelling.’ Too few are doing anything about it.

A glimpse into Philly’s thriving greentech scene, a bright spot on a national tour

How one-click job listings overtook the process — and slowed down tech hiring

Technically Media