Google's Focus on Performance improves Data Center PUE 8 - 25%, finding the hidden story in the data

Google announced its use of Machine Learning to improve its data center PUE in May 2014 and I posted on the release.  At 7x24 Exchange Fall 2014 event, 25 years of 7x24 Exchange were celebrated and Google’s Joe Kava, VP of Data Centers presented on “Google - beyond the PUE Plateau.”  The keynote is one of the more interesting and insightful presentations made as Google shared information on its experience deploying Machine Learning to its data center fleet.  One of the questions from the audience was “how was the first data center chosen to use Machine Learning?”  A special guest in the presentation was the data center mechanical engineer who spearheaded the project, Jim Gao.  His answer.  The data center that has most clean data to work with.

NewImage.png

Jim Gao and Joe Kava, 7x24 Exchange Fall Conference.

So what can this 25 year old mechanical engineer do with Machine Learning?  Below is data showing PUE, Wet Bulb, and Cooling Temperature across a range. The Blue areas are good, green not as good, yellow and red are bad.

NewImage.png

Some of you may be saying big deal.  I can figure out how to run the mechanical systems with a low PUE at a given wet bulb temperature to hit a given cooling temperature.  Well the above was a graph to illustrate what can be seen looking at performance data.  What is beyond our ability to see is working out the best way to run your mechanical systems with 19 Input Variables.  The below are the 19 inputs to the Predictive PUE Machine Learning system to figure out the lowest energy consumption.

NewImage.png

FYI, this predictive PUE system does not have autonomous control over mechanical systems.  It does provide information to the data center facility engineering teams on how they can improve PUE performance.  The predictive PUE model is 99.6% accurate.  Jim and Joe discussed how Google looked for a high degree of confidence in order to trust the numbers, and the human operators are an important part of the process like UPS drivers on their route.  UPS is famous for creating better routes for its drivers, but I bet they were not even close to the % savings Google achieved.

So how good are the results?  Google achieved from 8% to 25% reduction in its energy used to cool the data center with an average of 15%.  Who wouldn’t be excited to save an average of 15% on their cooling energy costs by providing new settings to run the mechanical plant?  Below is an example of what was historical PUE (blue) and New PUE (green) for a site.

NewImage.png

One of the risks Google took in this presentation is they let a 25 year old mechanical engineer get on stage.  Was the risk of the kid presenting?  No, Jim was as polished as many who have presented for years.  The risk was everyone at 7x24 Exchange knew who Jim was and they could try and see if he would consider leaving Google.  :-)

NewImage.png

The idea of using Machine Learning in data centers is new and have shown what can be discovered in the data.  It’s like there was a hidden story there waiting to be told.  Does you data center staff look for hidden stories in the data?  Shouldn’t you if you can save between 8-25% of the energy in systems.

Chicago Data Center Event, On Tap DC Reunion - Nov 20, 2014

Unfortunately I can’t make it out to Chicago for Steve Manos’s long over due Data Center Social.

If you find yourself in Chicago on Nov 20, 2014 here is information on the event.

If you can attend send your RSVP to stevemanos@canara.com or jason.okroy@saluteinc.com

Great Coaches for Development of Kids - Winning and Succeeding by trying your best

My son just finished his first year of pee wee tackle football where he was the amongst the smallest and youngest players on the team.  It was kind of scary seeing him go out in first practice with kids that were 20 lbs heavier, 6-8 inches taller and 2 years older, but he persevered and enjoyed the practices and games.  Here is a video where coaches were giving out awards at end of season.

Some parents are a little obsessed with winning and being competitive.  We’ve all seen what happens when the parents get out of hand being hyper competitive.

One of these days I hope to get my kids to absorb more life lessons like John Wooden articulates so well in this Ted Talk.

Some excellent points made.

Never try to be better than someone else, always learn from others. Never cease trying to be the best you can be -- that's under your control. If you get too engrossed and involved and concerned in regard to the things over which you have no control, it will adversely affect the things over which you have control.Then I ran across this simple verse that said, "At God's footstool to confess, a poor soul knelt, and bowed his head. 'I failed!' He cried. The Master said, 'Thou didst thy best, that is success.'"

3:01From those things, and one other perhaps, I coined my own definition of success, which is: peace of mind attained only through self-satisfaction in knowing you made the effort to do the best of which you're capable. I believe that's true. If you make the effort to do the best of which you're capable, to try and improve the situation that exists for you, I think that's success. And I don't think others can judge that.

DatacenterDynamics hires Uptime Institute's Bruce Taylor for EVP of North America and Consolidates from 9 to 3 events for 2015

As fast as the the data center industry changes it is ironic that the media/research companies move at a slower pace. 

What is one of the most interesting things to happen in the data center event/media/research industry is long time Uptime Institute executive Bruce Taylor is now Executive VP of DatacenterDynamics North America according to his LinkedIn profile

Also on Oct 31 DatacenterDynamics announced it is consolidating its US events from 9 to 3 with events in SF, Chicago, and NYC.

Starting in 2015, DCD Converged will shift from nine, one-day regional-market events to three larger-scale two-day conferences in the US. The revamped events program will focus on the three primary data center vertical classes: enterprise, internet and services.

The enterprise-focused event will take place March 17–18, 2015, at the New York Marriott Marquis in Manhattan. Meeting the needs of the enterprise data center community of end-users and operators, the New York event will place emphasis on the increasingly complex and heterogeneous mission-critical data center environment.

The San Francisco Marriott Marquis will play host to DCD’s internet-themed event, from July 30–31, 2015. The two-day conference will cater to the internet data center community of end-users and operators tracking the retail, wholesale and hyper scale continuum.

Services will be the main theme for DCD’s third and final US-based event next year, taking place Oct. 27–28, 2015, at the Chicago Hilton. The conference will focus on meeting the needs of the growing data center service provider community and how they provision next-generation services, from cloud to everything as a service.

2015 will be a competitive year for DatacenterDynamics vs. 451 Group/Uptime Institute vs. DatacenterKnowledge/DatacenterWorld vs. Gartner vs. 7x24 Exchange. 

For 2015 I’ll be at 7x24 Exchange and DatacenterDynamics.  I will not be at 451 Group/Uptime Institute given I am blacklisted from attending. 

Some media companies I have never talked to their executives or staff.  I regularly chat with 7x24 Exchange and DatacenterDynamics staff.  I guess that is part of the reason I attend their events is I feel like they listen to attendees and are open to change.

When will one of the top Data Center programs collapse? Google, Microsoft, Amazon, Facebook

Google, Microsoft, Amazon, and Facebook have the largest data center footprints with IT gear than anyone else in the industry.  Their programs are relatively young.  Google started 15 years ago.  Microsoft got going big 10 years ago.  Amazon launched AWS 8 years ago.  And Facebook followed after.

Throughout the history of technology developments a company collapses in the face of competition.  Each of these companies have different business models, but the data center programs are critical for the operation of these companies.  It is so important that billions of dollars are spent and there are thousands of people dedicated to run things.

It is an interesting thought exercise to ask which one of these data center programs could collapse and why.

Part of what inspired this post is this video on Richard Feynman “The World from another point of view.”  Asking questions that give you a different viewpoint can show you things that you hadn’t thought of, and then ask more questions.

Of these four - Google, Microsoft, Amazon, Facebook - how would split up the probability of a collapse of their data center programs and what would the recovery look like?  

Could a design decision be made that causes a cascading failure? Unthinkable.  And because it is unthinkable it might happen as no one thought how to mitigate the event.