Architecting for Outages, an architect posts on surviving AWS

Everyone wants to survive a data center outage, but as AWS outage shows, not all do survive.  Here is a post that summarize best practices in SW architecture to survive an outage like AWS.

Retrospect on recent AWS outage and Resilient Cloud-Based Architecture

Thursday, June 9, 2011 at 8:19AM

A bit over a month ago Amazon experienced its infamous AWS outage in the US East Region. As a cloud evangelist, I was intrigued by the history of the outage as it occurred. There were great posts during and after the outage from those who went down. But more interestingly for me as architect were the detailed posts of those who managed to survive the outage relatively unharmed, such as SimpleGeo, Netflix,SmugMug, SmugMug’s CTO, Twilio, Bizo and others.

The list of best practices are:

The main principles, patterns and best practices are:

• Design for failure
• Stateless and autonomous services
• Redundant hot copies spread across zones
• Spread across several public cloud vendors and/or private cloud
• Automation and monitoring
• Avoiding ACID services and leveraging on NoSQL solutions
• Load balancing

If this seems daunting, there are new services coming to provide scalability and availability services.

The emerging solution to this complexity is a new class of application servers that offers to take care of the high availability and scalability concerns of your application, allowing you to focus on your business logic. Forrester calls these "Elastic Application Platforms", and defines them as:

An application platform that automates elasticity of application transactions, services, and data, delivering high availability and performance using elastic resources.

Amazon’s Data Center Container "Perdix" something we haven’t seen

Yesterday I went to Amazon’s Technology Open House.

Here is a 1/4 of the crowd getting food and drinks early before James Hamilton’s keynote.

In James’s presentation he has a section on Modular & Advanced Building Designs

Every day, Amazon Web Services adds enough new capacity to support all of Amazon.com’s global infrastructure through the company’s first 5 years, when it was $2.7 billion annual revenue.

James presents his latest observations on data center costs.

And waste in mechanical systems.

But, here is something I didn’t expect.  Amazon Perdix.  Amazon’s version of modular pre-fab data container data center.  The below picture has Microsoft’s design on the left and Amazon’s on the right.

James is a believer in low density, 30 servers per rack where the cost per server is $1,450 or less.

Being Faster creates more traffic than Quality, Uptime Institute one month after Symposium posts videos

Uptime Institute has posted more videos a month after their Symposium event. http://www.youtube.com/user/uptimeinstitute

After only one day there is single digit traffic on the most recent posts.

Videos that have been out for 5 days have 10 - 20 views.

One video that got up sooner and has more traffic is Matt Stansberry interviewing Gary Cook with 186 views, 187 now that I watched it.

If you are going to post content most of time speed beats quality.

Steve Jobs Keynote serious about data centers, compares Apple, Amazon, and Google

Steve Jobs gave his iCloud keynote http://events.apple.com.edgesuite.net/11piubpwiqubf06/event/

at minute 115:00 you can see Steve Jobs compare Apple, Amazon, and Google cost of music cloud services.

To make the point Apple is committed to iCloud he makes the point Apple is serious about data centers.  Steve discusses its 3rd data center in Maiden, NC at minute 116:00.

Steve says this data center is as eco friendly  as a data center can be with modern technology.

Steve is a great show man as usual and wows people showing the scale of the building.

and points to two dots on the roof that are actually people, getting laughs from the crowd.  When is the last time you heard someone laugh when they talk about the scale of their data center.

"Full of stuff.  expensive stuff."  More laughs.  Who would ever call millions of dollars of IT equipment stuff?  You won't see Jobs calling an iPhone, iPod, or iPad stuff.  Do you think he is making fun of the other stuff he doesn't make?

It's been over 20 years since I worked at WWDC as an Apple employee, and never would have thought Steve Jobs would be talking about data centers.  A lot has changed in 20 years.  Wow 20 years, and there are people I know that have been there the whole time.  This video was probably some of the first pictures they've seen of their mothership data center in Maiden, NC.

After 100 years a dam reaches its end of life, Elwha Dam, largest US dam removal

Many think of hydro-electric as cheap clean energy.  The Elwha dam after 100 years is about to start the largest dam removal project in the US on Sept 17, 2011. 

This PDF shows the dam removal process.

June 1, 2011 was the ceremony to turn off the power.

Two Elwha River dams unplugged on Wednesday

The end of an era arrived here Wednesday with the final shutdown of two dams on the Elwha River.

By Lynda V. Mapes

Seattle Times staff reporter

PREV 1 of 3 NEXT

STEVE RINGMAN / THE SEATTLE TIMES

Kevin Yancy, manager of the Elwha hydroelectric project for the Bureau of Reclamation, throws a switch Wednesday to shut down the Elwha Dam for good after nearly 100 years of generating electricity. The Elwha and a nearby dam will be torn down beginning this fall.

STEVE RINGMAN / THE SEATTLE TIMES

Members of the media and employees of Olympic National Park watch as the 97-year-old Elwha Dam's giant turbines spin down after being shut off from their flow of water during the shutdown of the dam Wednesday.

Related

PORT ANGELES,

Clallam County —

The end of an era arrived Wednesday with the final shutdown of two dams on the Elwha River.

The dams were unplugged from the Western power grid and their mighty turbines stilled, in preparation for taking both dams down beginning in September.

"We are cutting the heart out of these old girls," said Kevin Yancy, manager of the Elwha hydroelectric project for the U.S. Bureau of Reclamation. But their time, he acknowledged, has passed, after nearly a century of generating power that fueled development of the Olympic Peninsula.