Twitter Presents on Metrics and Monitoring

I am down in SJ on my now regular visit. Velocity 90 conference just occurred, but I was not able to attend.

clip_image002_thumb[1]

The great thing about a conference like Velocity you can get to much of the content on their web site. I am going through a bunch of them, and the Twitter presentation by John Adams triggered some thoughts.

After going through the presentation I found it interesting that data centers are not built to Twitter’s requirements which is a boon  for the collocation companies. Twitter uses AT&T hosting services.

Can data centers be built for companies like Twitter?  Or do collocation companies need to be building different type of facilities for this market?

It was quite refreshing to see an operations team who gets what it takes to green its  operations.  John says nothing about being “Green” in his presentation, but in his passion for performance he is being green.

Here is the full video of the twitter presentation.

Slides that caught my attention for the data center crowd are below.

It is typical that IT Operations doesn’t deal with the physical plant.

clip_image002[5]_thumb[1]

And, Twitter figured out clouds don’t work for them as performance and latency is a top issue.

clip_image002[7]_thumb[1]

They use closed loop feedbacks.

clip_image002[9]_thumb[1]

Prioritize the issues by finding weak points.

clip_image002[11]_thumb[1]

Have metrics as a priority.

clip_image002[13]_thumb[1]

clip_image002[15]_thumb[1]

Present the data with dashboards and admit to data porn.

clip_image002[17]_thumb[1]

Then turn the data into actionable information.

clip_image002[19]_thumb[1]

The Intel gang must be doing a little jig when they saw this slide about efficiency, and how to people can green their data center with Intel Nehalem.

clip_image002[21]_thumb[2]

And makes an excellent point on disk in their scenario.

clip_image002[23]_thumb[1]

And provide transparency on the status of the site.  This would be great to have a PUE Status for those who claim record PUE numbers.

clip_image002[25]_thumb[1]

Read more

Yahoo Joins PUE Disclosure with 1.21, but Under What Conditions?

Datacenterknowledge reports on Yahoo’s disclosure of its 1.21 PUE at O’Reilly’s Velocity 2009.

Yahoo Unstealths Its Data Center Efficiency

June 24th, 2009 : Rich Miller

The Yahoo data center in Quincy, Washington includes cooling-optimized "podules" with a PUE of 1.21 (photo by Yahoo Inc.) 

The Yahoo data center in Quincy, Washington includes cooling-optimized "podules" with a PUE of 1.21 (Photo: Yahoo Inc.)

When it comes to data center efficiency, Yahoo has maintained a lower profile than rivals Google and Microsoft. But the Yahoo team is building a compelling data center story of its own, with innovations in cooling design and energy efficiency ratings approaching the best that Google has achieved.

Yahoo’s Adam Bechtel began telling the story yesterday at the O’Reilly Velocity 2009 conference in San Jose, Calif. Bechtel, the chief architect of Yahoo’s data center operations, shared details of a patented cold-aisle containment system that integrates an overhead cooling module, building the air conditioning units into the top of a “podule” of cabinets packed with servers.

That design has helped Yahoo lower its Power Usage Effectiveness (PUE) to 1.21, according to Bechtel, just a hair shy of the best numbers disclosed by Google and a slightly better than the lowest PUE reported by Microsoft. The PUE metric (PDF) compares a facility’s total power usage to the amount of power used by the IT equipment, revealing how much is lost in distribution and conversion.

What is missing is under what conditions was 1.21 determined.

What is needed is transparency for PUE claims.  Wouldn’t it be great if you could connect to a web service at google, microsoft, and yahoo data centers to get their PUE at any time of the day with the current weather conditions?   And, make a request for any other time for the PUE number?

Once we see a data center do this, I’ll believe the PUE claims would stand up to a compliance audit.

I am waiting for a claim of 1.10 PUE.  Competition is good, and this all helps educate more people.  But, we need more transparency on how PUE is calculated.

Read more

Smart-Grid Momentum, Benefactors are Data Vendors

news.com’s greentech has a post on the potential for Smart-Grid to be a Bubble.

Is smart grid the next green-tech bubble?

by Martin LaMonica

WASHINGTON--Here at a conference on the utility of the future, the starring players are Google, IBM, Cisco Systems, Intel, and smart grid start-ups. The reason? Data.

Modernizing the grid isn't just about installing more transmissions lines and smart meters. It's a giant information challenge as well, said attendees of consulting firm Kema's Utility of the Future conference here on Thursday.

The heavyweight IT companies are seeking to capitalize on initiatives around the world to upgrade the power infrastructure. The U.S. Department of Energy is expected to soon announce how billions of dollars in stimulus money for smart grid will be allocated.

Smart grid has also become one of clean-tech venture capitalists' favorite areas, spawning dozens of start-ups with ways to make the grid run more efficiently and integrate more solar and wind power.

Altogether, it's a combination that could end up creating a bubble, said Diana Propper, a clean-tech venture capitalist at Expansion Capital Partners.

"I worry that there's so much money being sloshed around, whether it's venture capital or corporate or government money, that it will be spent inefficiently," she said during a panel. "The risk of a bubble is real."

And, the vendors see the opportunity as data increases.

Koch added that the stepped-up presence of IBM, Cisco, Intel, and Google in the utility industry could stiffen competition for smaller firms.

Data overload
To understand the interest of the major IT companies in the smart grid, consider Duke Energy's program.

It has 5 million meters installed in its territory and each customer has a few major appliances, such as water heaters and refrigerators. Each one of those devices--in addition to hundreds of thousands of sensors on the distribution grid--could be networked.

To collect and make sense of the mountains of data these devices produce requires a robust network and sophisticated IT systems.

"Just the number of devices to be connected and the volume of data that needs to be processed--it's enormous," said David Mohler, the chief technology officer of Duke Energy. "We realized early on that we needed to create an information architecture. That's not a utility's sweet spot."

Instead, the utility contracted with Cisco to build that data communications network. Duke is already testing smart-grid technology and plans larger-scale deployments in Ohio and Indiana starting at the fourth quarter of this year.

Home energy management is another important piece of smart-grid programs. Microsoft and Cisco as well as telecom companies, such as Verizon, have said they expect to make energy monitoring an extension of existing home networks.

Here in the US, I am waiting for the attorneys to jump on the privacy issues monitoring people’s power consumption.  Can you imagine the legal release required to give utilities permission to use your data, store it, and share it with others.

Read more

Evil Side of Benchmarks and Metrics – Why laptop batteries don’t last as long as claims

Newsweek has an article about why laptops batteries don’t last as long as manufacturer’s claims. The article starts with the problem of who defines the metrics for laptop batteries.

Daniel Lyons

Hurry Up and Type

Why your laptop runs out of juice so fast.

Published Jun 18, 2009

From the magazine issue dated Jun 29, 2009

Imagine if automakers got together and started measuring the gas mileage of new cars with a cool test of their own making—one in which the cars were rolling downhill with their engines idling. Suddenly you'd have some pretty amazing claims: Why, that three-ton SUV gets 300 miles per gallon! This subcompact gets 500! In tiny print at the bottom of the window sticker you'd find a disclaimer saying that, well, um, you know, your mileage may vary.

Crazy, right? Yet that's more or less what's happening with laptop computers and their battery lives. Right now, I'm looking at a Best Buy flier touting a $599 Dell laptop that gets "up to 5 hours and 40 minutes of battery life." Down in the fine print comes a disclaimer explaining that "battery life will vary" based on a bunch of factors. Translation: you ain't gonna get five hours and 40 minutes, bub. Not ever. Not even close.

The specific benchmark referenced is the MobileMark 2007.

So how can Dell and Best Buy make that claim? These battery-life numbers are based on a benchmark test called MobileMark 2007 (MM07). The test was created by a consortium called BAPCo (Business Application Performance Corp.), whose members are—you guessed it—computer makers and other tech companies. AMD, the No. 2 maker of microprocessors, is a member of BAPCo, but now has become a whistle-blower. AMD says PC makers know full well that the new tests produce misleading numbers, but they are touting them anyway.

Any experienced Technology person has learned to ignore most benchmarks as they are easy to be gamed, and there is requirement for the vendors to be transparent in how they achieve their results.

So, what about energy efficiency and power saving claims from server and data center vendors. Many are taking advantage of the lack of benchmarks for power saving claims.  Who has the capability to measure the range of vendors?

And, even if there is a benchmark how do users know how the tests were performed.  in this economy, it is easy for executives to make the decision and take the risk to overstate efficiency as customers look for ways to save money.

What do you do?  Ask for more information on how performance tests were run. Can you get a copy of the test results? What is the range of expected results? What factors most influence the results? How close do the test conditions match your conditions?

Benchmarks and metrics are good, but are too easy to be used to overstate claims without an auditing infrastructure.  Watch out for metrics without 3rd party validation.

Read more

Green Monitoring Solution from Hara – Sanskrit for Green

I checked out www.hara.com website.  Not a whole let technically and how their service works.

Here is a video that works as a business decision maker presentation.

And consider their advisory board.

Hara Advisory Board

The Hara Advisory Board includes distinguished academics from leading universities who are experts in environmental science and technology. The Advisory Board brings an extraordinary depth of understanding and perspective to Hara in areas directly relevant to sustainable business practices.

Sam Chiu
  • Professor of Environmental Engineering, Stanford University
  • Expert on green supply chain and environmental operations
Arpad Horvath
  • Professor of Civil and Environmental Engineering, UC Berkeley
  • Expert on carbon lifecycle analysis
Bob Pojasek
  • Professor of Environmental Science, Harvard University
  • Expert on pollution prevention and cleaner production
Ed Rubin
  • Professor of Environmental Engineering, Carnegie Mellon University
  • Winner of Nobel Peace Prize 2007

With John Doerr and the Kleiner Perkins backing the PR coverage is extensive.

New York Times

Expecting New Tax, Firm Prepares to Track Carbon

AMR Research

Hara: Cool Software For a Warm Planet

Triple Pundit

Hara: Resource Management for a Post-Carbon Economy

Los Angeles Times

'Clean-tech' start-ups are pushing the green button

IT Business Edge

Hara: Taking Green Efforts Mainstream

Business Week

U.S. Corporations Size Up Their Carbon Footprints

Fast Company

Hara Software Helps Large Organizations Make Small Footprints

CNET

Hara: Software for a carbon-constrained economy

The Register

Ex-SAP man joins Gore green crusade

earth2tech

Sustainable Software as a Service Hara Launches, Backed By Kleiner Perkins

AFP

Hara helps companies profit from being green

San Jose Mercury News

Hara unveils software to help with energy management

Forbes

The Business of Tracking Carbon

IDG

Green Tech Startup Pitches SaaS Energy Tool

Forbes

Energy Management Startup Hara Nabs Coke as Client, $6 Mil from Kleiner

Read more