Using Situation Awareness Principle to Green the Data Center, Google continues the march from 1.16 to 1.14 PUE

Google posts it's latest PUE achievement of 1.14.

Measuring to improve: comprehensive, real-world data center efficiency numbers

March 26, 2012 at 9:00 AM
To paraphrase Lord Kelvin, if you don’t measure you can’t improve. Our data center operations team lives by this credo, and we take every opportunity to measure the performance of our facilities. In the same way that you might examine your electricity bill and then tweak the thermostat, we constantly track our energy consumption and use that data to make improvements to our infrastructure. As a result, our data centers use 50 percent less energy than the typical data center.
...
NewImage
...
NewImage

Google's Joe Kava uses the Lord Kelvin principle of "if you don't measure you can't improve."  But, I think a more apt explanation for the complexity of greening a data center is situation awareness.

Situation awareness

From Wikipedia, the free encyclopedia

Situation awareness is the perception of environmental elements with respect to time and/or space, the comprehension of their meaning, and the

projection of their status after some variable has changed, such as time. It is also a field of study concerned with perception of the environment

critical to decision-makers in complex, dynamic areas from aviationair traffic control, power plant operations, military command and control, and

emergency services such as fire fighting and policing; to more ordinary but nevertheless complex tasks

such as driving an automobile or bicycle.

Situation awareness involves being aware of what is happening in the vicinity to understand how information, events, and one's own actions will

impact goals and objectives, both immediately and in the near future. Lacking or inadequate situation awareness has been identified as one of

the primary factors in accidents attributed to human error.[1] Thus, situation awareness is especially important in work domains where the information

flow can be quite high and poor decisions may lead to serious consequences (e.g., piloting an airplane, functioning as a soldier, or treating critically

ill or injured patients).

Having complete, accurate and up-to-the-minute SA is essential where technological and situational complexity on the human decision-maker are

a concern. Situation awareness has been recognized as a critical, yet often elusive, foundation for successful decision-making across a broad

range of complex and dynamic systems, including aviation and air traffic control,[2] emergency response and military command and controloperations,[3]

and offshore oil and nuclear power plant management.[4]

Situation awareness vs. Lord Kelvin's principle has you thinking in the bigger picture.  Thinking about knowledge.  Am I doing the right thing?  How did I get here and can I repeat it?

Situation assessment

Endsley (1995b, p. 36) argues that "it is important to distinguish the term situation awareness, as a state of knowledge, from the processes used to achieve that state. These processes, which may vary widely among individuals and contexts, will be referred to as situation assessment or the process of achieving, acquiring, or maintaining SA." Thus, in brief, situation awareness is viewed as "a state of knowledge," andsituation assessment as "the processes" used to achieve that knowledge. Note that SA is not only produced by the processes of situation assessment, it also drives those same processes in a recurrent fashion. For example, one's current awareness can determine what one pays attention to next and how one interprets the information perceived (Endsley, 2000).

Google has shared the high level concepts of achieving a lower PUE.

1. Measure PUE

You can't manage what you don’t measure, so characterize your data center's efficiency performance by measuring energy use. We use a ratio called PUE - Power Usage Effectiveness - to help us reduce energy used for non-computing, like cooling and power distribution. To effectively use PUE it's important to measure often - we sample at least once per second. It’s even more important to capture energy data over the entire year - seasonal weather variations have a notable affect on PUE.

2. Manage airflow

Good air flow management is fundamental to efficient data center operation. Start with minimizing hot and cold air mixing by using well-designed containment. Eliminate hot spots and be sure to use blanking plates for any unpopulated slots in your rack. We've found a little analysis can pay big dividends. For example, thermal modeling using computational fluid dynamics (CFD) can help you quickly characterize and optimize air flow for your facility without many disruptive reorganizations of your computing room. Also be sure to size your cooling load to your expected IT equipment, and if you are building extra capacity, be sure your cooling approach is energy proportional

...

 

 

 

 

 

 

 

 

 

 

 

 

What does Google do to determine where it should spend its resources?  At some point there is a marginal return or a negative return.  It will cost more than what can be saved.  On the other hand at Google's scale what may be small for most can be huge for them.

Our 2011 numbers and more are available for closer examination on our data center site. We’ve learned a lot through building and operating our data centers, so we’ve also shared our best practices. These include steps like raising the temperature on the server floor and using the natural environment to cool the data center, whether it’s outside air or recycled water.

The really interesting thing to know is what has Google tried and found not to work.  As any good engineer knows many times you learn more from failures than success.

Cover Image: November 2009 Scientific American MagazineSee Inside

How You Learn More from Success Than Failure

The brain may not learn from its mistakes after all

Have you ever bowled a string of strikes that seems like it came out of nowhere? There might be more to such streaks than pure luck, according to a study that offers new clues as to how the brain learns from positive and negative experiences.

I think good engineers have learned to rewire their brain vs. others.

“Success has a much greater influence on the brain than failure,” says Massachusetts Institute of Technology neuroscientist Earl Miller, who led the research. He believes the findings apply to many aspects of daily life in which failures are left unpunished but achieve­ments are rewarded in one way or another—such as when your teammates cheer your strikes at the bowling lane. The pleasurable feeling that comes with the successes is brought about by a surge in the neurotransmitter dopamine. By telling brain cells when they have struck gold, the chemical apparently signals them to keep doing whatever they did that led to success. As for failures, Miller says, we might do well to pay more attention to them, consciously encouraging our brain to learn a little more from failure than it would by default.

 

 

 

 

 

 

 

 

 

Water shortages coming, Wars and Financial impact

MSNBC had an AP article on the coming risk of water shortages causing wars.

US intel: Water a cause for war in coming decades

'Water as a weapon or to further terrorist objectives ... more likely beyond 10 years,' says report released on World Water Day

Image: Israeli soldier stands next to a manmade pool containing water from a spring located near Ramallah
Baz Ratner /  Reuters
An Israeli soldier stands next to a manmade pool containing water from a spring located near the West Bank village of Nabi Saleh on March 19. Jewish settlers have seized dozens of natural springs in the occupied West Bank, barring Palestinians or limiting their access to scarce water sources, a United Nations report said this week. In 2009 the spring was taken over by settlers from Halamish, forcing villagers to obtain their irrigation water from other sources, the report and residents said.
By MATTHEW LEE
updated 3/22/2012 2:18:08 PM ET

Drought, floods and a lack of fresh water may cause significant global instability and conflict in the coming decades, as developing countries scramble to meet demand from exploding populations while dealing with the effects of climate change, U.S. intelligence agencies said in a report released on World Water Day.

An assessment reflecting the joint judgment of federal intelligence agencies says the risk of water issues causing wars in the next 10 years is minimal even as they create tensions within and between states and threaten to disrupt national and global food markets. But beyond 2022, it says the use of water as a weapon of war or a tool of terrorism will become more likely, particularly in South Asia, the Middle East and North Africa.

Fidelity Investments has a video on World's Water with 16,845 views in one month.

If you don't think about water issues in your data center design and operations you are not alone, but the people who think about sustainability and green data centers know water will become scarcer and more expensive.

 

James Hamilton's post on Solar Panels at Data Centers gets referenced by Forbes,

Forbes goes into depth reusing James Hamilton's post.

Can Solar Reduce The Impact Of Two High-Profile Data Centers? Amazon Engineer Weighs In [Updated]

The sun sets on the horizon across 42nd street...

Solar arrays may not be able to provide the power density needed by data centers, one expert argues. (Image credit: AFP/Getty Images via @daylife)

Solar power may be not be best way to reduce the environmental impact of sprawling data centers built by companies such as Apple and Facebook, James Hamilton, an Amazon vice president and distinguished engineer argued on his personal blog last Saturday.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Don't expect Solar Panels at an Amazon Data Center without tax incentives

James Hamilton writes an analysis of Facebook's and Apple's solar panel deployments.

I love solar power, but in reflecting carefully on a couple of high profile datacenter deployments of solar power, I’m really developing serious reservations that this is the path to reducing data center environmental impact. I just can’t make the math work and find myself wondering if these large solar farms are really somewhere between a bad idea and pure marketing, where the environmental impact is purely optical.

And closes with this.

Looking more deeply at the Solar Array at Apple Maiden, the panels are built by SunPower. Sunpower is reportedly carrying $820m in debt and has received a $1.2B federal government loan guarantee. The panels are built on taxpayer guarantees and installed using tax payer funded tax incentives. It might possibly be a win for the overall economy but, as I work through the numbers, it seems less clear. And, after the spectacular failure of solar cell producer Solyndra which failed in bankruptcy with a $535 million dollar federal loan guarantee, it’s obvious there are large costs being carried by tax payers in these deployments. Generally, as much as I like data centers, I’m not convinced that tax payers should by paying to power them.

As I work through the numbers from two of the most widely reported upon datacenter solar array deployments, they just don’t seem to balance out positively without tax incentives. I’m not convinced that having the tax base fund datacenter deployments is a scalable solution. And, even if it could be shown that this will eventually become tax neutral, I’m not convinced we want to see datacenter deployments consuming 100s of acres of land on power generation. And, when trees are taken down to allow the solar deployment, it’s even harder to feel good about it.  From what I have seen so far, this is not heading in the right direction. If we had $x dollars to invest in lowering datacenter environmental impact and the marketing department was not involved in the decision, I’m not convinced the right next step will be solar.

Given this information, I wouldn't hold my breath for a solar panel at Amazon for a few reasons.

  1. The economics don't make sense for Amazon
  2. Amazon would not want the visibility for its data centers.  (Is Amazon the next Greenpeace target?)
  3. Taxes is what drives many Amazon decisions, and they look long term at tax incentives.
  4. There are probably a bunch more, but bottom line the numbers don't support solar panels

Amazon acquires automated material handling vendor Kiva Systems

Amazon has a press release on the acquisition of Kiva Systems.

Amazon.com to Acquire Kiva Systems, Inc.

SEATTLE--(BUSINESS WIRE)--Mar. 19, 2012-- Amazon.com, Inc. (NASDAQ:AMZN) today announced that it has reached an agreement to acquireKiva Systems, Inc., a leading innovator of material handling technology.

“Amazon has long used automation in its fulfillment centers, and Kiva’s technology is another way to improve productivity by bringing the products directly to employees to pick, pack and stow,” said Dave Clark, vice president, global customer fulfillment, Amazon.com. “Kiva shares our passion for invention, and we look forward to supporting their continued growth.”

“For the past ten years, the Kiva team has been focused on creating innovative material handling technologies,” said Mick Mountz, CEO and founder of Kiva Systems. “I’m delighted that Amazon is supporting our growth so that we can provide even more valuable solutions in the coming years.”

Most of you have probably never heard of Kiva Systems.  I have.  Why?  Because, I used to be a distribution logistics geek.  I studied the subject in college. Worked in Distribution Logistics and packaging engineering at HP.  Apple actually hired me for my distribution logistics expertise away from HP and I worked on Apple's distribution system for years before moving to hardware product development and software product development.

One of the things that distribution logistics teaches you is supply chain management principles.  When I worked on hardware development the connection is obvious. Working on software, I would think of bits and information as a supply chain issue as the bits move.  The data center is a factory to support the movement of bits in a supply chain.  It is interesting when you start to think of things as abstractions that are simply bits of information.

So, who is Kiva Systems and why are they so interesting.  Kiva moves material to the person on robotic carts.  The software is quite intelligent to keep the fast moving items closest to the material handlers, and move the slow moving items further away.  This is no different than storage and caching issues in an operating system.

Read more about Kiva implementations in these markets.

The WSJ does a pretty good job of giving background on Kiva Systems

Robots on the March

How Kiva's robots run a warehouse

  • To complete an order, Kiva's squat orange robots fetch tall movable shelves, or pods, that have the items needed, bringing them to the human "picker."
  • A laser pointer tells the human which item needs to be picked from each shelf. The worker, who stays in one place, scans a bar code to confirm it is the right item. It's placed in the order box, which sits on another one of the mobile pods.
  • New pods arrive steadily with additional items as needed. Items are grouped together to fulfill the orders.
  • Pods filled with completed orders are taken by the robots to the shipping door, where a human tapes them closed in preparation for final transport.

But, here is an easier way to understand what Kiva System is.

Here is a TED talk by the founder.

 

Here is a IEEE Spectrum video on YouTube.

 

This will make for fun conversations with some of friends who work on logistics.

Amazon.com most likely figured out how much more efficient it is to move material with robots.  The old way of thinking about robots is like this video.