Microsoft Research Predicting Problems in the Data Center

February 25, 2009 Dave Ohara

Microsoft posts on its Techfest event and Predicting Problems in the Data Center.

Moises Goldszmidt (above), principal researcher at Microsoft Research Silicon Valley, is showing a pair of demos, in conjunction with lab colleague Mihai Budiu, that examines performance in data centers.

"The challenge," Goldszmidt says, "is: How do I summarize thousands of machines and hundreds of metrics and find the key elements over that huge space that's giving us surprises, such that I can let it retrieve that fingerprint? How do I do that automatically?"

The demo is called Predicting Problems in the Data Center.

"We are using very sophisticated machine-learning techniques," Goldszmidt states, "that build automated models that are able to extract the main characteristics of each one of these crises."

The value of such work is readily apparent.

"Eighty percent of the time, we're predicting one hour in advance a set of actions we need to do to mitigate a problem," he says, resulting in "less downtime, less latency for our clients using our services. Our services are more efficient to run, because we don't have to have that many people look at the problem."

Microsoft Research Builds Intel Atom Servers

February 25, 2009 Dave Ohara

Intel has got to be hating this publicity. Microsoft using Intel Atom chips to build servers. I”ve been blogging on the idea of using Intel Atom’s for servers, and people laughed at the performance. But, if you think about where the future of Intel Atom chips the rapid growth of Netbooks, Intel is getting phenomenal pressure to increase performance per watt.

DataCenterKnowledge reports on Microsoft Research’s use of Intel Atom based servers at Techfest.

Microsoft’s Low-Power Server Prototype
February 24th, 2009 : Rich Miller
How low can your server power go? Microsoft is investigating that question in a project by its new Cloud Computing Futures (CCF) research unit, which aims to reduce data center costs by “four-fold or greater.” The new group was introduced today at the Microsoft TechFest in Redmond. One of CCF’s initial research projects is testing the viability of a small cloud computing server farm using low-power Intel Atom processors originally designed for use in netbooks and mobile applications.

“In addition to requiring far less energy - 5 watts versus 50 to 100 watts for a processor typically used in a data center- low-power processors also have quiescent states that consume little energy and can be awakened quickly,” explained Dan Reed, director of Scalable and Multicore Systems for Cloud Computing Futures. “These states are used in the sleep and hibernate features of laptops and netbooks. With our current Atom processor, its energy consumption when running is 28 to 34 watts, but in the sleep or hibernate state, it consumes 3 to 4 watts, a reduction of 10 times in the energy consumption of idle processors.”

In this brief video, CCF Director of Software Architecture Jim Larus demonstrates a prototype rack packed with these low-power processors:

That wasn’t the only data center project discussed at TechFest.

The article continues posting on the use of closed loop feedback for dynamically adjusting the servers available.

The Cloud Computing Futures team also discussed Marlowe, a system for selectively putting idle servers into a low-power state. Reed said Marlowe “highlights the power of an intelligent control system that can determine when to put a processor to sleep and when to awaken it to service the workload.

“This problem has two interesting challenges,” he said. “The first is to estimate how many processors are necessary to handle a given workload by responding to every request in a timely manner. (By analogy, how many checkout clerks should be at the cash registers?) The second is to anticipate the workload in the near future, since it takes 5 to 15 seconds to awaken a processor from sleep and 30 to 45 seconds for hibernate. The system needs to hold some processors in reserve and to anticipate the workload 5 to 45 seconds in the future to ensure that sufficient servers are available.”

The solution was a closed-loop control system. “It works by taking regular measurements of the system, such as CPU utilization, response time, and energy consumption; combining this data with the estimated future workload; then adjusting the number of servers in each power state,” Reed said.

But this is not a new idea. Cornell Medical school’s Biomedicine dept has been doing this for over 2 years. Here was my blog entry 1 1/2 years ago.

This facility is one of the only places I know of that turns off servers when they are not needed. For IT Pros they do the equivalent of turning off the lights when they leave the office this holiday weekend. Think about how many servers are running these next 4 days from Thurs – Sun with no load on them. Would anyone notice if they were turned off?

The amazing thing is the Biomedicine department has been turning off their servers in a high performance compute cluster for the past 6 months and the users don’t notice a change in service, because they turn off and on the compute nodes in response to the job queue. There aren’t going to be that many research scientist submitting jobs on Thanksgiving day. And, as each compute job is completed and sits idle, there is an automated system that turns off the servers. When new compute resources are required as new jobs are submitted on Monday, the machines are turned back on.

To put this in #’s there are 100 servers in the compute node which each consume as much power as six 60 watt light bulbs, and when idle drop to consuming three 60 watt light bulbs of electricity. So, if this weekend they can turn off half the machines, they’ll save one hundred fifty 60 watt light bulbs of electricity. This project is implemented by Jason Banfelder, Vanessa Borcherding, and Luis Gracia at Cornell Weill Medical University, and this team can tell their parents this holiday weekend that yes we did turn off the lights in the office when we left the office. Actually, when they left the servers were probably at 100% utilization, and as jobs completed idling servers, they were turned off.

Cornell built this in production using dell servers and OSIsoft’s PI system, so don’t think of the idea of turning off servers as a research project

Data Center opinions on Obama’s Effect

February 19, 2009 Dave Ohara

SearchDatacenter has a post with a variety of IT opinions.

The Obama effect on IT operations: Data center panel weighs in

Will President Obama's campaign promises bring change for data centers? How will increased oversight of financial systems affect IT operations? Continued digitization of health care? Increased likelihood of passing carbon cap-and-trade legislation that would affect energy prices? Incentives for green business?

W e asked our data center advisory board, made up of CIOs, facilities execs, admins and analysts, how the Obama administration will affect IT operations.

If I was going to place bets i agree with Christian Belady’s comment.

Prepare to report on carbon emissions for the data center
I think one of the things that we can expect is that companies will be required to report carbon emissions, and a significant part of the emissions will be from data centers. Many of you may know that I have advocated the importance of measurement over the years, and those who have embraced [a measurement strategy] will find carbon reporting trivial.

For Microsoft, tools like Scry can report carbon emissions already at the server, rack or data center level. A few other companies are doing the same. For the rest, adoption and investment of real time measurement techniques will need to happen fast. As a result, companies will have to spend much of their resources making this happen in an unforgiving economy -- not the best place to be.

-- Christian Belady, P.E., principal power and cooling architect, Microsoft

Who Will Compete Against Google’s PowerMeter?

February 10, 2009 Dave Ohara

Google has 227 news articles on their PowerMeter feature. Sounds like they have a monopoly on mindshare. In less than 24 hours they have 227 news articles.

Google introduces power meter software to reduce electricity use
Telegraph.co.uk, United Kingdom - 21 hours ago
By Claudine Beaumont The Google PowerMeter software will analyse the consumption information captured by "smart meters", and translate it into ...
Google announces PowerMeter Computerworld
Google Impacts Your Electric Bill Seattle Post Intelligencer
Google tool helps consumers reduce energy usage Reuters
The Tech Herald - National Business Review
all 227 news articles » GOOG

I was down for a quick chat with google yesterday, but their PR group was slammed, and I’ll have to catch them another time.

Someone asked me who the competition is, and I don’t think anyone has a chance. I don’t think Amazon would get into this service. Microsoft? Here is one blog post that is wrong.

The battle has begun. Software developers everywhere are quickly realizing that the market for energy management applications has enormous potential. But although it’s worthwhile to watch the smaller companies and see if any one of them comes out with the next breakthrough, when players like Google and Microsoft jump into the mix, you know that their product will get far more exposure in the market.
Google’s product, the Google Powermeter, is directed towards the home energy user. It is a web-based graph which shows you how much energy you have been using over the course of the day, and where that energy is being used. What it does not appear to do is give you the ability to remotely control your appliances. Still, knowing where you use the most energy can make a big difference, as a Google employee testifies in a promotional video – he saved $3,000 in one year without any significant lifestyle changes.
Microsoft, on the other hand, is working on a product that appears more robust, and feature-intensive, but not as simple and easy-to-use as the Powermeter (which isn’t surprising when you consider Outlook versus Gmail). The Environmental Dashboard application for Microsoft Dynamics AX is catered to businesses rather than home energy users. Like the Powermeter, the Environmental Dashboard helps businesses analyze and reduce their energy consumption, but it also helps them measure their greenhouse gas emissions.

There is no comparison between Google PowerMeter and Microsoft’s Environmental Dashboard. In fact, Microsoft’s dashboard could sit on top of Google’s PowerMeter service.

Microsoft’s Publishes Green Exchange Server Document – Virtualized to Save Energy & Hardware

January 27, 2009 Dave Ohara

Microsoft has a blog post on Virtualizing an Exchange Server Environment.

Should You Virtualize Your Exchange 2007 SP1 Environment?

Introduction

With the release of Microsoft Windows Server 2008 with Hyper-V and Microsoft Hyper-V Server 2008, a virtualized Exchange 2007 SP1 server is no longer restricted to the realm of the lab; it can be deployed in a production environment and receive full support from Microsoft. This past August, we published our support policies and recommendations for virtualizing Exchange, but many people have asked us to go beyond that guidance and weigh-in on the more philosophical question: is virtualization is a good idea when it comes to Exchange?

Due to the performance and business requirements of Exchange, most deployments would benefit from deployment on physical servers. However, there are some scenarios in which a virtualized Exchange 2007 infrastructure may allow you to realize real benefits in terms of space, power, and deployment flexibility. Presented here are sample scenarios in which virtualization may make sense, as well as checklists to help you evaluate whether the current load on your infrastructure makes it a good candidate for virtualization.

Microsoft’s Exchange team did their homework and gave examples where you would use a virtualized exchange environment.

Small Office with High Availability

Some organizations are small but they still require enterprise-class availability. For example, consider Contoso Ltd., a fictitious company that regards email as a critical service and has several small branch office site(s) consisting of 250 users. Contoso wants to keep their e-mail environment on-premises for legal reasons and they want to have a fully redundant email system. Contoso's users have average user profiles and the mailboxes are sized at 2 GB.

Remote or Branch Office with High Availability

In the early days of Exchange server, organizations needed to place local Exchange servers in remote and branch offices to provide sufficient performance. With improvements such as Cached Exchange Mode and Outlook Anywhere (RPC over HTTPS), consolidating those servers to a central datacenter became the recommended approach. However, in some situations, poor network connectivity to remote offices still requires some organizations to have a local Exchange server. Often the user populations at these locations are so small that it doesn't make sense to dedicate a whole physical server to the Exchange environment. The technical considerations in this scenario are the same as described in the "Small Office with High Availability" scenario above. For an example of how a company used Hyper-V in this scenario, refer to the case study on Caspian Pipeline Consortium.

Disaster Recovery

In order to provide redundancy for a remote site, some organizations may require a Warm Site that contains a duplicate of the primary production Exchange 2007 infrastructure. The intent of this standby site is to provide as near to the same level of functionality as possible in the event of the loss of the primary site. However, keeping a duplicate infrastructure for standby purposes, although useful for high SLA requirements, can be prohibitively expensive for some organizations. In that event, it is possible to provide a virtual duplicate of the entire primary site using Hyper-V. A typical Warm Site configuration utilizing physical Exchange 2007 servers would include one or more servers configured together as a standby cluster and one or more other servers configured as a CAS/Hub server. To achieve redundancy of just the messaging services within the Warm Site, a total of four physical servers would be needed. By contrast, a Hyper-V-based solution with only three physical servers can provide an organization with a Warm Site that includes two Mailbox servers in a CCR environment, as well as and redundant CAS, and Hub servers. Thus, by virtualizing Exchange in this scenario, you can provide a higher level of services to your users while also saving on hardware, power and cooling costs as well as space requirements when compared to a similarly configured physical solution. The following diagram illustrates one such configuration.

Mobile LAN

There are situations in which a company, agency, or governmental department may need a complete network infrastructure that can be deployed to specific locations at a moment's notice. This infrastructure is then connected to the organization's network via satellite or similar remote WAN technology. For example, a non-governmental organization (NGO) may need to react to a disaster and set up local servers to serve an affected community. This subset of servers would need to be completely self-contained and able to provide all necessary server services to the personnel located in the target location.

If you think virtualizing Exchange or other Mail Servers consider these scenarios where Virtualization can make you Greener.