Amazon Web Services Supercomputer configuration, 880 Servers

November 19, 2010 Dave Ohara

AWS announced their supercomputer configuration with Amazon’s James Hamilton posting on the configuration.

The cc1.4xlarge instance specification:

· 23GB of 1333MHz DDR3 Registered ECC

· 64GB/s main memory bandwidth

· 2 x Intel Xeon X5570 (quad-core Nehalem)

· 2 x 845GB 7200RPM HDDs

· 10Gbps Ethernet Network Interface

The AWS supercomputer configuration is 7040 cores. At 4 cores per processor and 2 processors per server you get 880 servers (nodes) in the compute environment.

If you assume assume about 350 watts/server you can get 300KW of power. 20 2U server per rack makes for 44 racks and 7KW per rack. Sounds abut right.

Amazon is one of 4 self-made configurations.

10Ge is rare in many of supercomputer clusters, but AWS chose 10G Ethernet which may explain their self-made configuration.

But AWS was after a specific scenario like Hadoop.

It’s this last point that I’m particularly excited about. The difference between just a bunch of servers in the cloud and a high performance cluster is the network. Bringing 10GigE direct to the host isn’t that common in the cloud but it’s not particularly remarkable. What is more noteworthy is it is a full bisection bandwidth network within the cluster. It is common industry practice tostatistically multiplex network traffic over an expensive network core with far less than full bisection bandwidth. Essentially, a gamble is made that not all servers in the cluster will transmit at full interface speed at the same time. For many workloads this actually is a good bet and one that can be safely made. For HPC workloads and other data intensive applications like Hadoop, it’s a poor assumption and leads to vast wasted compute resources waiting on a poor performing network.

Kindle momentum continues

October 25, 2010 Dave Ohara

Amazon’s made a few announcements on its e-Book reader solution demonstrating the momentum for a greener book reading solution.

If you are curious on the environmental impact of an e-book reader vs. print check this out.

After an analysis of a number of studies on the publishing and e-reader industries, we predict that, on average, the carbon emitted in the lifecycle of a Kindle is fully offset after the first year of use. Any additional years of use result in net carbon savings, equivalent to an average of 168 kg of CO2 per year (the emissions produced in the manufacture and distribution of 22.5 books). There are additional savings in toxic emissions from publishing and water usage that we haven’t quantified.

Multiplied by millions of units and increased sales of e-books, e-readers will have a staggering impact on improving the sustainability and environmental impact on one of the world’s most polluting industries: the publishing of books, newspapers and magazines.

Amazon says the Kindle 3 is the fastest selling Kindle.

Amazon Says Latest Kindle Is Fastest-Selling Version So Far

Amazon.com Inc. (AMZN) said the latest generation of its Kindle was "the fastest-selling" of its e-readers and the best-selling product on the company's websites in the U.S. and the U.K.

As of Monday, the e-commerce retailer said sales of the new, cheaper Kindle already surpassed total Kindle device sales from October through December 2009. Amazon began taking orders in late July and started shipping the models in late August.

Meanwhile, Amazon said it has sold three times as many Kindle books in the first nine months of the year from the year-earlier period. And in the past month, customers purchased more Kindle books than print books--hardcover and paperback combined--for the top 10, 25, 100, and 1,000 bestselling books on Amazon.com.

You can be suspicious of a press release as most of have learned.

An interesting piece of data I have is the traffic to my Kindle 3 frozen blog post.

Google Search is the overwhelming dominant way the post is found which would mean this is coming from users who are encountering a frozen Kindle 3. I am quite surprised how consistent over 2 months people are looking for an answer to their frozen kindle.

With the growth of the Kindle installed base, Amazon will be adding a lending feature.

later this year, we will be introducing lending for Kindle, a new feature that lets you loan your Kindle books to other Kindle device or Kindle app users. Each book can be lent once for a loan period of 14-days and the lender cannot read the book during the loan period. Additionally, not all e-books will be lendable - this is solely up to the publisher or rights holder, who determines which titles are enabled for lending.

One of these days we can hope Amazon will post how much carbon they have saved with electronic books. Maybe Amazon should add a counter to their Kindle page showing how many tons of carbon are saved as a kindle book, magazine or newspaper is purchased.

Imagine how few Kindle Servers Amazon needs. The Kindle solution could be one of the greenest data center solutions as paper, water, and environment are saved.

Amazon's James Hamliton shares Overall Data Center Costs, except taxes

October 10, 2010 Dave Ohara

James Hamilton has a nice post on the Overall Data Center Costs

Overall Data Center Costs

A couple of years ago, I did a detailed look at where the costs are in a modern , high-scale data center. The primary motivation behind bringing all the costs together was to understand where the problems are and find those easiest to address. Predictably, when I first brought these numbers together, a few data points just leapt off the page: 1) at scale, servers dominate overall costs, and 2) mechanical system cost and power consumption seems unreasonably high. Both of these areas have proven to be important technology areas to focus upon and there has been considerable industry-wide innovation particularly in cooling efficiency over the last couple of years.

I posted the original model at the Cost of Power in Large-Scale Data Centers. One of the reasons I posted it was to debunk the often repeated phrase “power is the dominate cost in a large-scale data center”. Servers dominate with mechanical systems and power distribution close behind. It turns out that power is incredibly important but it’s not the utility kWh charge that makes power important. It’s the cost of the power distribution equipment required to consume power and the cost of the mechanical systems that take the heat away once the power is consumed. I referred to this as fully burdened power.

This post supports one of the ideas I threw out there on Amazon's financial discipline.

At many companies the rigors of getting approval a data center construction project approved are from the business units, technical people, and the CFO. I would expect at Amazon.com, everyone wants to see the numbers, and they spend much more time on financial modeling.

Should we lease or build?

What is the best use of Amazon's capital and cash?

What is the overall operating expense of a leased vs. owned facility?

Latest decision in the East Coast. Lease.

The one variable that is difficult to put into an excel document is the taxes you pay. Believe it or not the tax incentives are one of the biggest things that gets the big builders to be in one state vs. another.

James's data is a good starting point, but don't forget to get your tax department involved in the data center project.

Amazon leases data center space while Google and Microsoft build

September 27, 2010 Dave Ohara

DataCenterKnowledge reports on Amazon.com leasing 125,000 sq ft of office/warehouse space and converting it to data center space.

Report: Amazon Leases Space in Virginia
September 28th, 2010 : Rich Miller
Amazon’s expanding cloud computing operation apparently needs room to grow. The commercial real estate news site Globe Street reports that Amazon has leased a building in northern Virginia that will be used to expand its data center operations.

Citing an unnamed source, Globe Street says Amazon has leased a 125,000 square foot facility in Sterling, Va. and will invest $60 million in converting the site into a data center. The building, which is owned by a joint venture between Altus Realty Partners and Perseus Realty Partners, features a 90,000 square foot warehouse with 18-foot ceiling clearance that can be converted to data center space. The facility also features about 35,000 square feet of space currently used for offices.

as referenced the Globe Street mentions the shortage of space.

As data center space becomes ever more scarce in the area, a growing number of firms are investing in building out the operations themselves. Microsoft, for example, is investing $499 million in the southern county of Mecklenburg to build a data center. It will be the largest investment here and create 50 jobs, according to Virginia Gov. Bob McDonnell.

Amazon has an estimated 150,000 plus servers which would have most companies building data centers not leasing them. Amazon is a different breed than other high tech companies thinking like a retailer and driven by a financial discipline. Amazon has only one built data center in Boardman, Oregon.

Will Amazon eventually build big data centers like the rest following Google, Microsoft, Yahoo, and Facebook?

At many companies the rigors of getting approval a data center construction project approved are from the business units, technical people, and the CFO. I would expect at Amazon.com, everyone wants to see the numbers, and they spend much more time on financial modeling.

Should we lease or build?

What is the best use of Amazon's capital and cash?

What is the overall operating expense of a leased vs. owned facility?

Latest decision in the East Coast. Lease.

Web Metrics on Frozen Kindle 3 post

September 5, 2010 Dave Ohara

One week ago, I posted on my experience contacting Amazon.com tech support regarding a frozen Kindle 3. The following are a bunch of metric analytics I'll use to describe my post 7 days old. Note: I am using this as an example of what I can figure out now on any post I put on this blog.

There are some interesting market research and intelligence I am figuring out about users around the world who hit my post. For example, understanding what keywords people are searching for helps develop better content. I can also get leads on companies looking for data center content and how much interest there is in content in different geographic locations.

First after 7 days, if you Google search "frozen Kindle 3" my blog post my post shows up #1.

Here is the traffic through Feedburner for the post. 311 feed readers and 106 clicks back. Nothing particularly big.

Going to Google Analytics, here is the traffic over the past week.

There were 897 views with average time on page of 4:49 which means a lot of people were taking time reading and referencing the post as they were trying to fix their Kindle 3. The bounce rate is 96.14% as the readers didn't have an in interest in green data centers, but it was since to know that 3.86% read something else on my blog.

Looking at the keywords typed in Google Search here are the top 15. The #1 entry is the 106 unique page views that came from the same 106 reported in Feedburner for clicks to content based on my RSS subscribers. #2 and on are the order in frequency of page views from keywords typed in that users eventually clicked on my post.

The list of keywords goes up to 241 down to one click entries.

There are a total 45 countries who have used Google Search to find the post. The following are top 10.

138 regions.

To get more specific here are the top 10 out of 500 cities.

There are 391 service providers listed including amazon, RIM,

What type of users are the Kindle 3 user base? Windows, Mac, Linux. Here are the top 13 which should work as a pretty good sample to figure out a user mix and there other devices. Note the # of Apple users - Mac, iPhone, iPad, and iPod.

And the browser mix illustrates the Apple loyalists with the high Safari hits.

And, what search engines did people use. My Microsoft friends ask why I don't use Bing more. well when more than 95% of my search traffic comes from Google, I keep to the same mindset of my users.

I'll write about analytics on a green data center technical topic. But, the main one I am studying is the 2,000 hits I have on the Top 5 data center construction companies. I am quite surprised at how much traffic I get to that post, and how it stays up there in traffic.