Green Data Center Blog

HP Joins the Container Data Center Market

The Register writes on HP’s Container offering.

Hewlett-Packard has finally found its way into the data center trailer park.

It took a while, but the hardware vendor is introducing its own scheme for selling chunks of data centers in pre-packaged containers. HP joins the likes of Sun, IBM, Rackable, and Verari with similar White Trash Data Center programs.

HP calls its offering a "Performance Optimized Data Center" or POD. HP joins IBM by providing the option of filling containers not just with the vendor's own kit, but also a wide variety of third-party metal.

"We engineered our PODs to be the most flexible infrastructure in the industry," said Paul Miller, HP's marketing chief of enterprise gear. "If it can fit into a 19-inch rack, we can pretty much fit it into our POD."

Miller said customers are all about standardized hardware in the container arena. For example, the standards approach lets customers start with half a POD and then move their existing equipment into the container when they need the extra space.

Most other vendors have chosen a ground-up approach for the job — fitting their containers with specialized gear made for life in a 40-foot unit. HP, however, doesn't have new hardware for its container. But at least it's managed to cram a lot of what it currently has in there.

HP claims the shipping containers will support more than 3,500 compute nodes, or 12,000 large form factor hard drives. The company estimates that's equivalent to 4,000-plus square feet of typical data center capacity. It also promises delivery within six weeks of the customer's order.

Here is HP’s POD web site.

HP’s slick video is here.

How Strong is a Shipping Container? A Fireworks Test Demonstration

For the 4th of July it was appropriate to have fireworks.  For those of you looking at container based data centers, you probably are wondering how strong containers are.  Check out this video of a test by 321 Fireworks crew shown in a PBS show.

The folks at Pyro Works have a geeky version of the test, providing more details and repeat the 12” firework shell test.

If you search on youtube for" “container test” you’ll find a bunch of different tests like a container full of fireworks.  So, for those of you who concerned about the strength of a container data center you can compare it to data center construction.

If you want to launch large fireworks shells as a test.

Happy 4th of July!

Microsoft Presentation on Containers at GigaOM's Structure 08 - a PUE of 1.3!

PCWorld reports on a Container presentation by Microsoft's Daniel Costello, Director of data center research at GigaOm's Structure 08.

"The idea of modular, portable data centers is key to the industry's future," said Daniel Costello, Microsoft director of data center research, in a presentation at GigaOM's Structure 08 conference in San Francisco. "That's why I'm here to talk about data centers, not just for Microsoft but for our customers as well."

Buying these boxes from server vendors can be more energy-efficient and cost-effective than building a new, traditional data center, he said, and Microsoft sees them as more than just a way to add extra computing capacity at short notice. "We see them as a primary packaging unit," he said.

Using shipping containers is part of an effort by Microsoft to radically rethink its data centers, as it tries to add more computing capacity in a way that is cost effective and power efficient. "At Microsoft, we're questioning every component in the data center, up to and including the roof," Costello said. That includes "eliminating concrete from our data center bills."

"The definition of a datacenter has changed. It's not just bricks and mortar any more, and moving forward, we think it can be a lot more energy efficient," he said.

But vendors building portable data centers today are filling them with equipment that was designed for traditional data centers. "Moving forward, we need to design systems specifically for this form factor. If we look at the containers, that form factor will change over time as well."

Daniel covers the efficiencies in PUE.

They can offer a better "power unit efficiency" ratio than do traditional data centers, he said. PUE is a measure of a data center's power efficiency. If a server demands 500 watts and the PUE of a data center is 3.0, the power from the grid needed to run the server will be 1500 watts, according to a definition from the Green Grid industry consortium.

"We've seen PUE at a peak of 1.3" in modular data centers, Costello said, compared with between 1.6 and 2.0 for a traditional data center.

The containers can accommodate 1,000 watts per square foot, allowing companies to power a lot more servers in a given area, he said. Many companies are unable to add more equipment to their data centers because power supplies and cooling equipment are at maximum capacity. The portable data centers are an alternative to building new facilities or extending old ones.

And, the portable data center.

But he thinks portable data centers will be deployed widely to provide services to end users. "We used to talk about a PC on every desk," he said. "But how about a data center in every town?"

Which reminds me of a conversation when I was talking to Christian Belady at a Uptime Institute event.  Christian said he thinks power for data center is becoming scare. I told him I think what will happen in the future is Microsoft guys will drive around in a town looking for what buildings/businesses have power infrastructure in place. Buy those buildings with good power and networking infrastructure, level the building and put a container data center in the space in months.

More Details on Dell's XS23 server used in Cloud Computing, HPC, & Containers, designed for high energy efficiency

Dell has more details on its XS23 Cloud Server.  Seems like this is the kind of server's good for containers as well.

XS23 Cloud Server

Tue. May. 13, 2008

There has been some recent press around some of the equipment we’ve developed in our cloud computing group. The core of our business is essentially a consulting and design service and developing new products for customers is a big part of the fun. Because these aren’t mainstream PowerEdge systems, we don’t get the chance to show them off as much as we’d like. Our group has been talking for some time about “optimized designs” for cloud and hyperscale computing without showing what that can really mean, so it’s time to unveil something that’s come out of the lab.  Pictured here is one of our favorites: the XS23.

XS23 front – twelve 3.5” SAS or SATA drives; 3 per server

This product was designed for a customer that needed maximum compute density, a healthy amount of local disk and, of course, lowest power draw possible. Our architecture team threw all that in the blender and out came a 2U standard rack mount chassis that houses four dual-socket servers and twelve 3.5” hot plug drives.

The energy efficiency design is driven by customer requirements. Yippeee!!! Cloud Computing and HPC customers are driving the requirements for Greener HW designs.

This was expressly designed for an environment with high node failure tolerance - a cloud application. By designing out a lot of the capabilities that weren’t required (like redundant power) we were able to deliver the performance and power profile required. Efficiencies are gained by shared resources - as seen in a lot of general purpose designs available today. We think the key to designing the perfect cloud server is knowing where to stop and also what not to build in. This is a function of each customer’s unique design goals.

Mike Manos Provides Reasons for Microsoft's use of Containers, writes Response to ComputerWorld Article

For those of you who want to know more about why Microsoft (MSFT) is using Containers, Mike Manos just posted a response to the ComputerWorld article, "6 reasons why Microsoft's container-based approach to data centers won't work."

Mike's post, titled "Stirring Anthills ... A response to the recent Computerworld Article"

The stirring anthills is a good description of how fired up Mike is in his response to accusations Microsoft is not listening to the industry.

Again, I highly suggest you read his post, and here are some nuggets.

 

When one inserts the stick of challenge and change into the anthill of conventional and dogmatic thinking they are bound to stir up a commotion.

That is exactly what I thought when I read the recent Computerworld article by Eric Lai on containers as a data center technology.  The article found here, outlines six reasons why containers won't work and asks if Microsoft is listening.   Personally, it was an intensely humorous article, albeit not really unexpected.  My first response was "only six"?  You only found six reasons why it won't work?  Internally we thought of a whole lot more than that when the concept first appeared on our drawing boards. 

My Research and Engineering team is challenged with vetting technologies for applicability, efficiency, flexibility, longevity, and perhaps most importantly -- fiscal viability.   You see, as a business, we are not into investing in solutions that are going to have a net effect of adding cost for costs sake.    Every idea is painstakingly researched, prototyped, and piloted.  I can tell you one thing, the internal push-backs on the idea numbered much more than six and the biggest opponent (my team will tell you) was me!

...

Those who know me best know that I enjoy a good tussle and it probably has to do with growing up on the south side of Chicago.  My team calls me ornery, I prefer "critical thought combatant."   So I decided I would try and take on the "experts" and the points in the article myself with a small rebuttal posted here:

...

The economics of cost and use in containers (depending upon application, size, etc.) can be as high as 20% over conventional data centers.   These same metrics and savings have been discovered by others in the industry.  The larger question is if containers are a right-fit for you.  Some can answer yes, others no. After intensive research and investigation, the answer was yes for Microsoft.

...

However, I can say that regardless of the infrastructure technology the point made about thousands of machines going dark at one time could happen.  Although our facilities have been designed around our "Fail Small Design" created by my Research and Engineering group, outages can always happen.  As a result, and being a software company, we have been able to build our applications in such a way where the loss of server/compute capacity never takes the application completely offline.  It's called application geo-diversity.  Our applications live in and across our data center footprint. By putting redundancy in the applications, physical redundancy is not needed.  This is an important point, and one that scares many "experts."   Today, there is a huge need for experts who understand the interplay of electrical and mechanical systems.  Folks who make a good living by driving Business Continuity and Disaster Recovery efforts at the infrastructure level.   If your applications could survive whole facility outages would you invest in that kind of redundancy?  If your applications were naturally geo-diversified would you need a specific DR/BCP Plan?   Now not all of our properties are there yet, but you can rest assured we have achieved that across a majority of our footprint.  This kind of thing is bound to make some people nervous.   But fear not IT and DC warriors, these challenges are being tested and worked out in the cloud computing space, and it still has some time before it makes its way into the applications present in a traditional enterprise data center.

As a result we don't need to put many of our applications and infrastructure on generator backup. 

...

In my first address internally at Microsoft I put forth my own challenge to the team.   In effect, I outlined how data centers were the factories of the 21st century and that like it or not we were all modern day equivalents of those who experienced the industrial revolution.  Much like factories (bit factories I called them), our goal was to automate everything we do...in effect bring in the robots to continue the analogy.  If the assembled team felt their value was in wrench turning they would have a limited career growth within the group, if they up-leveled themselves and put an eye towards automating the tasks their value would be compounded.  In that time some people have left for precisely that reason.   Deploying tens of thousands of machines per month is not sustainable to do with humans in the traditional way.  Both in the front of the house (servers,network gear, etc) and the back of the house (facilities).   It's a tough message but one I won't shy away from.  I have one of the finest teams on the planet in running our facilities.   It's a fact, automation is key. 

...

The main point that everyone seems to overlook is the container is a scale unit for us.  Not a technology solution for incremental capacity, or providing capacity necessarily in remote regions.   If I deploy 10 containers in a data center, and each container holds 2000 servers, that's 20,000 servers.  When those servers are end of life, I remove 10 containers and replace them with 10 more.   Maybe those new models have 3000 servers per container due to continuing energy efficiency gains.   What's the alternative?  How people intensive do you think un-racking 20000 servers would be followed by racking 20000 more?   Bottom line here is that containers are our scale unit, not an end technology solution.

And, Mike closes with

I can assure you that outside of my metrics and reporting tool developers, I have absolutely no software developers working for me.   I own IT and facilities operations.   We understand the problems, we understand the physics, we understand quite a bit. Our staff has expertise with backgrounds as far ranging as running facilities on nuclear submarines to facilities systems for space going systems.  We have more than a bit of expertise here. With regards to the comment that we are unable to maintain a staff that is competent, the folks responsible for managing the facility have had a zero percent attrition rate over the last four years.  I would easily put my team up against anyone in the industry. 

I get quite touchy when people start talking negatively about my team and their skill-sets, especially when they make blind assumptions.  The fact of the matter is that due to the increasing visibility around data centers the IT and the Facilities sides of the house better start working together to solve the larger challenges in this space.  I see it and hear it at every industry event.  The us vs. them between IT and facilities; neither realizing that this approach spells doom for them both.  It’s about time somebody challenged something in this industry.  We have already seen that left to its own devices technological advancement in data centers has by and large stood still for the last two decades.  As Einstein said, "We can't solve problems by using the same kind of thinking we used when we created them."

Ultimately, containers are but the first step in a journey which we intend to shake the industry up with.  If the thought process around containers scares you then, the innovations, technology advances and challenges currently in various states of thought, pilot and implementation will be downright terrifying.  I guess in short, you should prepare for a vigorous stirring of the anthill.

Now if we can get someone to get Mike fired up like this once a month, we'll learn a lot more.  Enjoy Mike's post.

Obsolescence of Microsoft's Container Data Center, Nakagin Capsule Tower

Kisho Kurokawa was a leading Japanese Architect, Famous for the Nakagin Capsule Tower. There are many concepts Kurokawa used in the Capsule Tower that parallel Microsoft's Container Data Center.

The Nakagin Capsule Tower' (中銀カプセルタワー, Nakagin Kapuseru Tawā^?) is a mixed-use residential and office tower designed by architect Kisho Kurokawa and located in Shimbashi, Tokyo, Japan. Completed in 1972, it has thirteen floors which house prefabricated modules (or "capsules") which are each self-contained units.

Construction took place in two separate places: on-site and off-site. On-site construction included the two towers and their energy-supply systems and equipment, while the capsule parts were fabricated and the capsules assembled at a factory.

The capsules were prefabricated and fitted out with utilities and interior fittings before being shipped to the building site, where they were attached to the concrete towers. Each capsule is attached independently and cantilevered from the shaft so that any capsule may be easily removed without affecting the others.

Here is a video of Kurokawa where he talks about a recyclable, sustainable design, creating the first building of its kind in the world. The capsule's life cycle was designed to be 25 years. Kurokawa emphasizes everything is designed to be maintained.

Another perspective is from a Dwell Article by Tom Vanderbilt.

In his own writings, Kurokawa, a Buddhist, offered a fitting and, especially now, quite haunting encomium to the capsule tower: "We used to consider things that could live forever to be beautiful. But this way of thinking has been exposed as a lie. True beauty lies in things that die, things that change."

35 years after the Nakagin Tower, Microsoft's First Container Data Center are the latest efforts to apply Kurokawa's concepts of a recyclable, sustainable design using containers.

The Nakagin tower didn't change the way the designer intended. Which is why it reached its obsolescence, and will be demolished.

Can Microsoft's first container data center avoid the same fate?

Is the container a new unit of maintenance for Microsoft's data center?

Microsoft's Data Center Purchases Influencing Hardware Efficiencies, Dell joins Container Vendor List

Data Center Knowledge and The Register report on Dell joining the Container list of suppliers, Rackable, Sun, and Verari.

Dell Inc. (DELL) is building a data center in a shipping container for a customer, and will follow with a container product line. "We have (a container system) in the works for a customer," a Dell insider told The Register. "We are looking at that space very, very closely." The Register's Ashlee Vance said it appears Dell has "geared up a container for Microsoft's late April RFP."

The "data center in a box" concept has been embraced by Microsoft, which plans to pack between 150 and 220 40-foot containers into the first floor of its new Chicago data center. Microsoft executives say the new facility will house up to 300,000 servers.

Dell joins a growing herd of hardware vendors offering container solutions, including Sun Microsystems (JAVA), Rackable (RACK) and Verari Systems all report strong interest in their container products. Last month IBM said its new iDataPlex series of cloud computing servers is being offered in a 40-foot trailer, marking IBM's first foray into container-based systems.

Dell squeezes cloud into a shipping container

By Ashlee Vance in Santa Clara → More by this author

9 May 2008 19:22

Exclusive Sun Microsystems endured a lot of ribbing when it first popped out a data center in a shipping container. Now, however, it looks like all the majors are heading in that direction, including Dell, which The Register has learned has a containerized data center in development.

"We have (a container system) in the works for a customer," said a Dell insider. "We are looking at that space very, very closely."

All of the focus on containers is highlighting an interesting change Microsoft has made on the Data Center hardware industry. In the past, Microsoft has written hardware specification documents like this one.

This publication is the first in the Hardware Support and Directions for Windows Server series, which shares the Microsoft intention and investment direction for support of specific hardware technologies in current and future releases of the Windows Server operating system. This series focuses on Windows Server features that are relevant to the hardware capabilities of a server.

This is soft pitch to server OEMs and other hardware vendors to build good Windows Server boxes. This document will not drive big changes.

Given the purchasing by Microsoft's data center properties (search, hotmail, maps,etc.) are now driving Server OEMs with custom RFPs like the CBlox RFP, OEMs are building exactly what Microsoft wants to run a more efficient data center. And, versus Google's model of requiring exclusive designs no one else in the industry can purchase, the Microsoft skus spill into the rest of the market.

To confirm the idea, here is speculation on ask.com's sku being marketed by Dell.

One system, however, really caught our attention and is worth some ink now. It's the XS23, which regular folk cannot buy.

Dell refuses to comment on the server publicly, although we managed to work some information about the hardware out of source.

The XS23 squeezes 4 two-socket servers (in a 2X2 stack) in a 2U chassis along with twelve 3.5 inch SAS/SATA drives across the front of the system. It was designed for a search company, which we believe was Ask.com.

As we understand it, the disk to DIMM count was very important for this search customer, who wanted three drives for every server. This design was enough for the unnamed customer to buy tens of thousands of systems, according to our source.

The Dell system consumes 25 per cent less space than your general purpose blades, which do about 16 two-socket servers in 10U. Dell, of course, stripped out the redundant power supplies and fans to get that density, but these cloud folks have software that can deal with failures just fine.

We even managed to obtain a couple photos of the XS23. The big daddy shots are here and here.

One of the most valuable lists would be Microsoft's data center equipment RFPs and the equipment they chose, but don't hold your breath waiting for Microsoft to make this publicly available. It would be a PR disaster with almost every Server OEM screaming. 

Which Container is better Sun or Rackable?

I had an interesting call with an investment firm who wanted to discuss Data Center Containers.  He asked a question which one has a better container Sun or Rackable?

I told them it's too early in the container game to declare who the winner is.

Jokingly then I said Rackable, because it's bigger, and hey part of this is marketing and you see a 40 foot ice cube next to a Sun 20 foot project blackbox which one is more impressive.  He chuckled at the comment.

After a bit more thought. I do think Rackable picked a better initial build out with 40 foot vs. the 20 foot Sun Box.

If you are going to go with containers are you going to take small steps? Or go for it with a 40 footer.  Note Microsoft's references to CBlox being 40 feet.  There is no mention of 20 foot containers.

This flies in the face of the idea of over provisioning, and getting more than you need, but marketing means giving what people they want to buy, not what is logical for them.

ComputerWorld writes: 6 reasons why Microsoft's container-based approach to data centers won't work

I've been waiting for the ComputerWorld article on Containers part 2, and here it is.  After attending Uptime Institute and talking to vendors and customers, I do think containers will work, but there are lots of people who are going to disagree as it changes the game.

May 9, 2008 (Computerworld) Microsoft Corp.'s plan to fill its mammoth Chicago data center with servers housed in 40-foot shipping containers has experts wondering whether the strategy will succeed. In Microsoft's plan, each container in the data center, still being built, will be filled with several thousand servers.

Computerworld queried several outside experts — including the president of a data center construction firm, a data center engineer-turned-CIO, an operations executive for a data center operator and a "green" data center consultant — to get their assessments of the strategy. While they were individually impressed with some parts of Microsoft's plan, they also expressed skepticism that the idea will work in the long term.

Here are some of their objections, along with the responses of Mike Manos, Microsoft's senior director of data center services. Manos talked with Computerworld in an interview after the Data Center World show at which Microsoft's plan was announced

I am in there a few times. And unfortunately, was misquoted, but hey that is the tradeoff of talking to the press.

That's why some observers, such as Ohara, say the market is actually in smaller units. A former supply chain engineer for both Hewlett-Packard and Apple, Ohara has been developing his own prototypes for a "server cube" that would weigh about 1,000 pounds and measure 1 meter in each dimension — hence the name of his blog, GreenM3.

"It's taking what's in a server rack but putting it into a cube to make it more efficient to roll out," he said. "That potentially could apply to many more people."

I am not actually developing a server cube. I was trying to explain the method of how you look at compute in cubic meter, explaining my GreenM3 story.

I am also in the article for

"Say 25% of the servers have failed inside a container after a year. You may say you don't need that compute capacity — fine," said Dave Ohara, a data center consultant and blogger. "But what's potentially expensive is that 25% of the power committed to that container is doing nothing. Ideally, you want to use that power for something else.

"Electrical power is my scarce resource, not processing power," Ohara concluded.

But, I think anyone who is smart will figure this out, and design around it.

Microsoft's Debra Chrapaty Keynote Data Center Operations Video, presents Green concepts & CBlox Container

I mentioned Debra's Keynote in a previous post, and the good thing is her video is available for online viewing.

Microsoft's Debra Chrapaty presented at Microsoft Management Summit as a keynote speaker, The Reality of the Cloud.  The subject of her presentation is how Microsoft runs its data centers, the Global Foundation Services group.

Future trends she presents the Microsoft Container, CBlox as well.

The video is here.

She discusses a lot of concepts for Green Data Centers

• Power how to measure and manage.
• Building and running at scale
• Smart Growth
• Innovate for Efficiency and Sustainability
• Operational excellence
• Every kilowatt counts
• Environment Sustainability