Microsoft Blog Post – Thinking Out of the Box – Chicago Container Data Center

October 20, 2008 Dave Ohara

DataCenterKnowldge has a post about Microsoft’s Chicago Data Center.

Microsoft: PUE of 1.22 for Data Center Containers
October 20th, 2008 : Rich Miller

Microsoft says its testing shows that the data center containers it is installing in its new Chicago data center are extraordinarily energy efficient. The 40-foot shipping containers packed with servers can deliver a Power Usage Effectiveness (PUE) energy efficiency rating of 1.22, which rivals recent PUEs reported by Google. Microsoft’s Mike Manos revealed the PUE numbers in a blog post about the Chicago data center, which has just completed its first phase of construction.

Mike Manos’ original post is here.

The specifics about the Green and PUE is below.

The other thing which is important is the energy efficiency of the containers. Now I want to be careful here as the reporting of efficiency numbers can be a dangerous exercise in the blogo-sphere. But our testing shows that our containers in Chicago can deliver an average PUE of 1.22 with an AVERAGE ANNUAL PEAK PUE of 1.36. I break these two numbers out separately because there is still some debate (at least in the circles I travel in) on which of these metrics is more meaningful. Regardless of your position on which is more meaningful, you have to admit those numbers are pretty darn compelling.

For the purists and math-heads out there, Microsoft includes house lighting and office loads in our PUE calculation. They are required to run the facility so we count them as overhead.

On the “Sustainability” side of containers it’s also interesting to note that shipping 2500 servers in one big container has a positive reduction on the CO2 related to transportation, let alone the amount of packaging material eliminated.

So in my mind, containers are driving huge cost and efficiency (read also as cost benefits in addition to “green” benefits) gains for the business. This is an extremely important point, as Microsoft expands its data center infrastructure, it is supremely important that we follow an established smart growth methodology for our facilities that is designed to prevent overbuilding—and thus avoid associated costs to the environment and to our shareholders. We are a business after all. We must do all of this while also meeting the rapidly growing demand for Microsoft’s Online and Live services.

HP’s PUE article, quotes HP Employee Chris Malone, Ahh Chris works for Google Now

October 5, 2008 Dave Ohara

Found this article on HP’s Enterprise Magazine, discussing PUE for data center efficiency. At the time it was written, Chris Malone was an HP employee, but now he works for Google. Transforming Your Enterprise Magazine

Winter 2008

Metrics drive efficiency in the data center

In an effort to understand where data center energy inefficiencies lie, companies are adopting standardized metrics of PUE and CPE to determine the initiatives required for optimization.

Energy consumption and inefficiency within data centers is a well publicized problem and a key concern for IT organizations. Gartner has predicted that, "By 2011, more than 70 percent of U.S. enterprise data centers will face tangible disruptions related to floor space, energy consumption and/or costs."*

Before data center inefficiency can be resolved, experts say it must be characterized and tracked in a consistent manner.

“Our investigations have shown that most data centers are inefficient, expending more power for cooling than for running the IT equipment,” notes Dr. Chris Malone, Thermal Technologies Architect for HP. “Until recently, there was no established method for benchmarking data center energy consumption, making it extremely difficult to characterize the full extent of the problem.”

Malone suggests a standardized evaluation of and rating for data center energy consumption to help companies determine:

How well is a data center running relative to others in the industry?

Are new data centers more efficient than the old ones?

Which data centers should be retired?

Do best practices really improve efficiency?

Do new technologies actually perform as suppliers promise?

HP didn’t quote Christian Belady who was a PUE innovator at HP.

For those of you who didn’t know Christian is now at Microsoft. Chris is at Google. Too bad HP couldn’t keep these guys at HP. Otherwise, we would be discussing HP’s innovations in data center efficiency.

Summary of Microsoft's Data Center Event - Microsoft Set the Bar

September 30, 2008 Dave Ohara

TechHermit has three different attendees submit comments about the Microsoft (NASDAQ: MSFT) Data Center Event. Mike Manos, Christian Belady, Daniel Costello, and the rest of the Microsoft Data Center team are changing the data center industry.

As good as this event was, the real one people want to get into is if Microsoft does this again for its Iowa Data Center, its next generation data center.

"The facility that we have planned here in Iowa will be something special and unique," said Manos, Microsoft's general manager of data center services. "It will be the first data center that what we consider our next-generation facility."
The new center will "drive significantly better energy efficiency than any center under construction today," Manos said. "Iowa, specifically West Des Moines, was the perfect location."

They could have an event for a whole week and people would gladly pay $10,000 each to attend.

Here are comments from TechHermit's blogs.

Jenny Rator

TH -
Well we just wrapped up the second day and I am now back in my hotel room relaxing after a very busy day.   The day began for me as I visited the presentation by Microsoft Security chief and General Manager Peter Boden. The program that Microsoft has around security, compliance, audits and the like is unlike anything I have ever seen before.   You never really think about the requirements for security and Microsoft’s program blends the logical and physical in a way that would astound most people.   Microsoft has an impressive list of certifications for their facilities, something you NEVER hear from anyone else advertising software as a service or operational accountability. More than that, Mr. Boden shared their approach, growth and evolution on how they got there. These were incredible lessons for those of us in the audience tasked with security requirements
I then attended a question and answer panel emceed by Michael Manos. The panel consisted of Microsoft’s top mechanical engineers, electrical engineers, architects, and Christian Belady. Frankly speaking this session was so information packed and the level of sharing of learnings so deep that its actually hard to describe. You could have done an entire day with just this team and not exhausted the questions and learnings. Everyone sitting around me agreed that you just dont get access to this type of information from ANYONE or you would have to pay an architecture and engineering company big bucks to get to talk to these kinds of people.   At lunch afterwards in one of the long hallways in the facility I talked about this session with several colleagues who were equally blown away by two key things - the forthrightness of answers and the level of sharing with the attendees. Microsoft has the most intelligent Data Center engineering organization I have ever seen.   If Microsoft ever does this again - This is a MUST ATTEND event.

Warhawk85:

TechHermit:
Like the person who posted yesterday I thought that I would send you some of my highlevel thoughts of this event.
IT TOTALLY ROCKED!
Microsoft did in one event what the last three years of attending Uptime Symposiums, DATA CENTER WORLDS, and Data Center Dynamics have completely failed to do! They shared real world examples, real data and engineering, and gave actual users exposure to real data that they could use. They did all this in a working data center that would completely blow your mind. The tour and the level of detail they went into on the facility design was just incredible. Microsoft knows how to build data centers. Dont let anyone ever tell you different. I have been in this business for 30 years and nothing even comes close.
The Microsoft event was vastly different than anything I have ever attended before in my career. It lacked the uppity air and pretentiousness of the Uptime events, and vastly more detailed than anything Dynamics or Data Center World would get into. As Michael Manos told me in the hall his goal was to have a conference “by operations people, for operations people, about operations and real world issues.” At that they totally succeeded.   Everyone I talked to went on about the quality of the sessions and sheer amazement at their open-ness.
In my opinion, Microsoft did more for Data Center end-users and the Data Center Community than a hundred conferences. I know you dont want to post about the Uptime controversy anymore but if there was ever a successor to that work, my vote would be for this group to manage it.

OilCanSam

Hermit,
I just wanted to send you my personal experience at the Microsoft Data Center Experience event in San Antonio as I see you have an entry about it. The event was wonderful. Microsoft executives and technical teams were frank, open, honest, and generally extremely generous with their time. While we have different design criteria and operations models it was great to see how someone else has approached the same problems. The tours of the facility, the technologies used, the engineering reasons they have done what they have done was clearly articulated to everyone. The opening night keynote had an interesting line : ” This event is not to show anyone the right way to do datacenters, the wrong way to do datacenters, just the Microsoft way of doing datacenters.” I have to say I was somewhat dubious that I would get a lot of sales marketing around their products but the event was all about operations and with the exception of some salesy stuff in the Virtualization track around Hyper-V it was incredibly down to earth. one of the best events around data centers and definitely the best Microsoft event I have ever attended. My personal kudo’s goes out to the Microsoft teams for an amazing experience.

I hope this creates pressure on Google (NASDAQ: GOOG) to open its doors or maybe people have given up on finding out anything about Google.. Has anyone else noticed how little news there is on Google's data centers?

Microsoft Shares Insight on its Data Center Chargeback System

August 28, 2008 Dave Ohara

The Power of Software blog has a new post by Amaya Souarez. There is so much information in this post, it is hard to summarize, so let’s start with a list, extracting words from the post.

Green is a priority.

I believe that implementing chargeback models based on power usage will encourage customers to consider power efficiency more seriously and reduce our overall impact on the environment.

Interesting analogy

I would like to illustrate this point by using the analogy of getting a first car for a teenager. By the way, if you currently have one of these creatures at home, then you have my commiserations.

Comparing Chargeback Models

Let’s say you are investigating the conditions under which you will allow your teenager to have a car. Consider the following two options:

The teenager buys the car, but you pay for the gas.

You buy the car and the teenager pays for the gas.

For each option, what sort of car do you think your teenager will want?

With option 1, you may find that they come back with a clapped out 351 cubic inch monster with the thirst of a Jentil, maybe something like the “striped tomato” from Starsky and Hutch. And who cares? You’re paying for the gas, right?

But there’s no point in having the coolest car in the neighborhood if you can’t even afford to take a date to a drive-in movie. If you select option 2, you may find your teenager develops a more healthy interest in how many miles per gallon (MPG) something more sensible can manage (like the new Prius hybrids at the Redmond Campus), rather than whether they have enough torque to leave tire slicks longer than a 747 landing at Princess Juliana International airport.

Statement of the problem.

With servers, the situation is more complex; the main issue that we struggle with is the fact that there is no direct equivalent to miles per gallon. How do you measure application output? It isn’t just about CPU utilization. What about an application that makes repeated calls to hard disk but doesn’t use much in the way of processor resources? If you fit more memory to the server, it can may be able to cache the disk access calls, but you’re then consuming more power in the additional memory module.

All the standard performance monitoring areas, such as processor, memory, hard disk, network, and cache make varying contributions to power consumption. What we needed at Microsoft was a chargeback model that is easy to understand, straightforward to administer, and allocates data center costs to customers proportionately.

Confession of the issues.

For the Microsoft data centers, the effort to change our chargeback model was not a simple conversion, as it took us one and a half years to move from our previous model of charging for floor space based upon rack utilization to the new model. Not surprisingly, it was not the tooling or process modifications that posed the biggest hurdle but the cultural and political changes that were required. Even today, I frequently have to remind customers that ‘DC space is power.’

Final action.

The final model that we now use has two basic components:

Floor Space. This component is billed per kilowatt (kW) of usage and includes all the floor space costs.

Power and Cooling. This component is billed per kilowatt hour (kWh) of usage and includes the cost of electricity as billed by our energy suppliers.

I’m not using real figures here, but you should be able to see the basis of how we implemented charging based on power consumption.

And there is plenty more in the blog entry.

Amaya Rocks!

Here are her final words.

Summary

Changing our chargeback model to one that uses power as the basis for floor space makes sense, both for us and for our customers. As older equipment is retired and replaced, we expect to see greater emphasis on power efficiency rather than raw output. Reducing power consumption on individual servers results in a reduction in the total power consumption for the data center, helping to conserve our power bandwidth and minimize our impact on the environment.

Microsoft Launches Next Generation Data Center in Des Moines, IA – Winning the Data Center PR War?

August 22, 2008 Dave Ohara

Microsoft officially announced its choice of Des Moines, IA to launch its next generation data center. News coverage is all over the web.

Here is the video from DesMoinesRegister

The one thing interesting about this announcement is a normal PR event would be the ground breaking where a handful of local officials and company executives smile while holding a shovel of dirt.

But, Microsoft has been able to escalate the status of a data center to a state press release. With logo’d cupcakes to honor the event.

There are tons of press coverage and the blogs have picked up on the Iowa data center as well. Contrast this with Microsoft’s Windows Vista PR team hiring Jerry Seinfeld to get visibility and credibility in the market. A little ironic Microsoft hires a comedian to get respect.

As DataCenterKnowledge wrote about the economic benefits of Google vs. Microsoft. Iowa is the new test area.

Many state and local governments have passed tax incentives to attract a Google data center, hoping it will convert struggling regions into high-tech hubs. It's part of a trend in which more economic development agencies are pursuing data center projects. There's no question that favorable environments can create data center clusters, a trend seen in many technology corridors. But here's the secret: If you're focused on developing a data center cluster in your area, you should be chasing Microsoft, not Google.

My vote is Microsoft will win the Iowa PR data center PR war because:

Microsoft is sharing its data center best practices and the data center industry recognizes this effort.
Microsoft’s Des Moines data center is the launch site for its latest generation of data center design.
Microsoft will hire within the state of Iowa. Google’s Council Bluffs location will have a large percentage of construction workers and employees coming from Omaha Nebraska. Why should the state Iowa give tax incentives to help the state of Nebraska? It is only a matter of time before the Iowa tax payers protest Google’s tax breaks.
When you look at the map below showing Chicago, Des Moines, and Council Bluffs. A question comes up why would Microsoft build data centers so close to each other? I have my own theories what Microsoft will do. These Microsoft guys are smart.

Will there be containers in Des Moines? ComputerWorld’s Eric Lai says yes.

A Microsoft spokeswoman confirmed that Microsoft plans to house the servers in shipping containers but declined to comment specifically on the size of the facility or the number of servers to be located there.

"We are still in the process of completing the design of the center. Once that is finalized, we will have an estimate for these questions," she wrote via e-mail.

However, Microsoft said its $500 million, 550,000-square-foot data center in Chicago will house up to 220 containers, each filled with as many as 2,000 servers, or 440,000 servers. The software maker said the server-filled containers are easier to transport, set up and maintain than servers on conventional racks, though not all observers agree.

It takes 5 hours to drive a container between Chicago and Des Moines. How’s that for a disaster recovery plan?

One funny thing about this picture

is Gov Chet Culver looks taller and bigger than Mike who is a big guy. Looking up the gov is an college football player so he is probably bigger than Mike. The gcod thing I found is the governor has an interest in Green.

Culver began his career as an environmental and consumer advocate in the Iowa Attorney General’s Office.

We can expect Microsoft’s Data Center to be as green as can be. There is no hydroelectric for carbon neutral power, but 20% of the power is reported to be coming from renewables.