Near miss science accounts for Human Factors, when will Data Centers do the same?

WSJ reports on the science of studying near misses and human factors that contribute to system failures.

Near Misses Are a Hit in Disaster Science

  • By CARL BIALIK

Columnist's name

While there never has been an oil spill in the Gulf of Mexico quite as large as the current disaster, there have been other terrible mishaps and, as in every industry, near misses.

These close calls are what Scott Shappell, professor of industrial engineering at Clemson University, looks for when he works with airlines on quantifying their risk from human errors.

"All you hear about are crashes, but it's the near misses that are telling," Prof. Shappell says. "If you only knew how many near misses there are in aviation, you would never fly again."

Near misses can be studied by statisticians to estimate the probability of an event that hasn't occurred before. Estimating the probability of unlikely disasters has become standard practice for nuclear and space regulators. Such an exercise, experts say, could help companies involved in deep-sea drilling evaluate risks and possibly prevent catastrophes like the Gulf oil spill.

Human Factors science is a common practice in Nuclear Power Plants, Aviation and complex programs like the Space Shuttle.

image

The following can just as be easy be stated about the data center industry.

There also is some danger in producing risk assessments that are too precise. "Most laypeople want a single number," says Todd Paulos, chief technology officer for Alejo Engineering Inc. in Huntington Beach, Calif., but "we can't predict anything to that accuracy."

The wild card in all these estimates is human error. Nuclear-power companies for years have sought to calculate and reduce risk by evaluating how people respond to simulations of potentially dangerous situations.

One of these days we'll see a data center consulting company offer Human Factors analysis.  Here is the company that has a bunch of Human Factors people who specialize in the.

Wiegmann, Shappell and Associates (WSA) are an Internationally recognized team of experts in system safety and human performance. As the authors of the Human Factors Analysis and Classification System (HFACS) and the Human Factors Intervention matrix (HFIX), WSA provides a variety of services to help incorporate these tools in to your organization to improve safety, quality and efficiency.

We're confident you'll find that Wiegmann, Shappell and Associates is not your typical safety consulting company. Renowned scientists with real world experience in the field, WSA bridges the gap between theory and practice to effectively integrate human performance and system safety principles and tools into your organization.

Read more

Oops, Biomass may not be Carbon Neutral

CNET reports on a Massachusetts study that Wood Biomass may not be carbon neutral.

Study finds biomass power not carbon neutral

by Martin LaMonica

Forested regions around the world are pursuing biomass as a renewable energy source but a study finds that the carbon footprint from burning biomass can be worse for global warming than coal.

The Manomet Center for Conservation Sciences on Thursday published the findings of a six-month study to measure the greenhouse gas impacts of using biomass, which, in many cases, does not meet claims of being "carbon neutral" over short periods of time.

The report was commissioned by the Massachusetts Department of Energy Resources, which said it will revise its regulations in response. "We can begin the process of refining our renewable energy regulations to provide incentives only for biomass energy that truly reduces our greenhouse gas emissions and protects our forests," said Massachusetts Department of Energy Resources Commissioner Phil Giudice, in a statement.

Note the study didn't focus on wood waste used for fuel generation.

In response, biomass industry people said that the study does not paint a completely accurate picture of biomass-to-power facilities because it assumes that they don't use residue wood products, such as branches and trees left from logging. "The study is not representative on how we plan to operate," Matt Wolfe of Madera Energy, which is proposing a wood-burning plant in western Massachusetts, told the Boston Globe.

The original study is here.

Manomet Study of Woody Biomass Energy Released

Manomet and its partners have released the results of a six-month study to better understand the implications of using wood for energy in Massachusetts, titled “Biomass Sustainability and Carbon Policy Study.”  The study was conducted for the Massachusetts Department of Energy Resources.  The full report, or its component chapters, can be downloaded below.

Read more

Going to visit my #10 traffic country, Italy

In one week I am going to take a two week break from blogging June 18 - July 2.  I am going on vacation to visit my #10 traffic country, Italy.

Here is my current traffic for last 5 months with about 62,000 visits.

image

Of that Italy is #10.

United States (US)
31,486

United Kingdom (GB)
3,516

India (IN)
2,679

Canada (CA)
2,489

France (FR)
1,695

Germany (DE)
1,256

Australia (AU)
1,202

Japan (JP)
829

Netherlands (NL)
811

Italy (IT)
665

Did I pick Italy because it was #10?  Of course not, but it is a bit of trivia that Italy is #10 in my traffic.

I am going to spend a week in cooking school, Tuscan Women Cook run by Texans.

Morning classes include both hands-on cooking and demonstrations at local country homes, plus visits to food artisans, olive oil producers, local food markets, and food festivals. We will stop and buy the day’s ricotta and Pecorino cheeses from local farmers. The classes are taught in Italian and are translated into English. Tuscan Women Cook is an insider’s view of true Tuscan seasonal cuisine.

I am the main cook in the family, and getting ready for my new cooking toy a WoodStone Oven.

Read more

World Cup Soccer, assessment of unrest and violence

Today is the start of World Cup Soccer and I was watching this screen, and it reminded of some work done by some really smart people.

Below is the setup for the South Africa first goal.

image

The movement that allows South Africa to get in position.

image 

And the goal

image

It is easy to do a post-analysis on what caused the goal.  But, it is much harder to do a pre-analysis on the factors that affect winning or losing.

Now let me show you what the smart people I know are able to do at the World Cup Soccer to understand the potential dynamics that can cause unrest and violence.  Here is the PDF I will be referring to.

The following is a demo showing the use of Thetus's Savanna product.

image

In the upper left is the assumption, "Demographics and migration patterns may influence stability during the world cup.  Immigration is a primary issue"

image

Which then supports, "What are the potential dynamics that can cause unrest or violence during the World Cup?"

image

How many systems have you seen that accept questions as input?  Almost all other systems are focused on data you have already set up monitoring, looking only within your system.  Can you ask questions that gets you looking for information beyond your data feeds?

You can browse the PDF to see more of the events and relationships.

Thetus Savanna allows an analysis process to see relationships and how events occur.  With this understanding you can mitigate undesirable action and invest in areas that are win conditions.

People actually do this activity all the time, but almost no one has a tool like Thetus Publisher.  (Disclosure: I am working with Thetus to figure out how their technology can be applied to data center scenarios, because they are some of the smartest people I've found to figure out how to model the complexity in the data center.)

There are only a handful of people at this time who can understand this technology and apply it.  And, luckily after two years figuring out who the right people are and what are the right scenarios, this method is closer to being deployed.

Writing this blog entry was the easiest way to tell the less than 10 people i know who will understand this approach "here is a cool graphic you can use to illustrate the potential use to your co-workers and team."

And, it is time to start sharing this approach.  I'll continue to post on this modeling method as it explains how I am using my blog posts.  In general, I am blogging about public facts that fit in modeling data centers.

As a few of my business associates and clients know, there is logic to my posts, and they can read the relationships between the posts.  This does not apply to all posts, but they know how to parse what I write.  One benefit of using this blogging approach is I can meet with people, and we immediately can drop into details as they have been reading what I have been blogging.

As my wife just told me last night, I wish you could tell me how to use technology as well as you write things.  I told her well, I spend more time thinking about what to write, than I do thinking about what to say.  :-)

In the same way it is sometimes hard to understand the exact movements in a soccer match that support a goal, once you recognize the patterns that you want to repeat, you start to score more.

Enjoy the World Cup Soccer. 

Read more

Electrostatic Discharge (ESD), Humidity and the Data Center

I was reading DataCenterKnowledge's post on Yahoo's Data Center revolution, and one area that I want to elaborate on is electrostatic discharge, humidity and IT equipment.

It’s been a long journey from third-party colocation space to the Yahoo Computing Coop. The journey started by addressing the accepted range of 45 to 55 percent humidification for data center space.  “Where did this belief come from?” Noteboom wondered.

Noteboom went searching for references, and says he found only a few academic scenarios warning of humidity issues. For practical arguments for tightly controlled humidity for computer rooms, Noteboom said he had to go back to the era of punch-card data entry, when concerns focused on whether excess humidity might wilt the punch cards and make them difficult to process properly.

“In 2005 we built a data center without humidification control,” said Noteboom. “It was filled with 25,000 servers. We were worried about static, and had all these protocols for handling equipment. We don’t have them anymore. We’ve operated through an entire lifecycle in this data center, and we’ve seen no impact.”

In 1980 as a summer intern I worked at HP on the idea that electrostatic discharge was causing component failures.  I was working with guys like Dick Moss who was an ESD expert.

Richard Y. Moss
Hewlett-Packard Co.,
Palo Alto, CA, USA

Two of the questions most asked by manufacturing managers are "How good must my ESD-control program be?" and "How much money can it save me?" These are reasonable questions, considering that managers must justify their decisions in terms of return-on-investment or payback period.

Unfortunately, the first question is difficult to answer and the second almost impossible. One rarely knows the extent of ESD damage in a process until after it has been eliminated.

There is a bunch of stuff I learned 30 years ago, zapping parts with ESD, testing parts for failure, digging holes in plastic IC components, poring fuming sulfuric acid to burn the plastic away, and looking at integrated circuits to see where failures were occurring.  We knew places like Fort Collins, CO had a higher electrostatic discharge risk due to low humidity, and you addressed the problem with wrist grounding straps, anti-static work surfaces, anti-static floor treatments, and air ionizers to remove the statically charged particles.

Quite frankly I've thought it dumb that the Data Center industry is obsessed about humidity which was most likely caused by a concern over electrostatic discharge, and potentially paper punch cards.  There are much more cost effective ways to remove ESD risk than humidity.  And, even a high humidity doesn't completely eliminate ESD.

If you want to address ESD then following guidelines like this.

Principles of electrostatic safe handling

There are two simple principles we can use to protect ESD sensitive components from ESD damage:

  • Only handle sensitive components in an ESD Protected Area (EPA) under protected and controlled conditions
  • Protect sensitive devices outside the EPA using ESD protective packaging

How many other dumb things are being done because of its been done in the past?

OK this is rant for the day.  I am calmer now.

Read more