Gigaom Research has a webinar on June 25, 2014 on Making Sense of Sensor data.
Maybe you’ve heard of the Internet of Things, and maybe you’re skeptical. But this isn’t just about thermostats and personal pedometers. It’s about fleet optimization, supply chain management, container shipping, manufacturing, sentiment analysis, and fraud prevention, too.
Analysis of streaming data focuses on determining not just the “what and why,” but also the “what’s next.” By combining sensor data with historical data, even deeper insights can be extracted, equipment breakdowns averted, money saved and efficiencies gained.
After spending the last few months intensely discussing a range of technologies in the data center industry something was bothering me. I understood what their technology did, but as I kept asking about performance and other operating issues I wasn’t getting answers I wanted. The simple think I want to know is “how well does this technology work.” If someone uses it what are the issues they will run into. By solving one set of problems, what new problems do they pick up?
Telling me what customers you have as references tells me you have done a good job selling your service, but that doesn’t mean it works well. Sometimes the people who make the purchasing decisions are far removed from the operating issues. Being able to have conversations with operations staff is one of the ways to get to the truth. Even if you have a nice looking report I’ll still be suspicious.
Hearing from someone who uses a technology in operations is one of the most credible sources. As an option push the vendor to answer, “how well does this work?” And when they tell you how it works. Repeat, I know how it works. I want to know how well it works, operates.
It was predictable that with Google sharing its use of Machine Learning in a mathematical model of a mechanical system that others would say they can do it too. DCK has a post on Romonet and Vigilent being other companies that use AI concepts in data centers.
Google made headlines when it revealed that it is using machine learning to optimize its data center performance. But the search giant isn’t the first company to harness artificial intelligence to fine-tune its server infrastructure. In fact, Google’s effort is only the latest in a series of initiatives to create an electronic “data center brain” that can analyze IT infrastructure.
...
One company that has welcomed the attention around Google’s announcement is Romonet, the UK-based maker of data center management tools.
...
Vigilent, which uses machine learning to provide real-time optimization of cooling within server rooms.
Google has been using Machine Learning for a long time and uses it for many other things like their Google Prediction API.
What is the Google Prediction API?
Google's cloud-based machine learning tools can help analyze your data to add the following features to your applications:
Customer sentiment analysis
Spam detection
Message routing decisions
Upsell opportunity analysis
Document and email classification
Diagnostics
Churn analysis
Suspicious activity identification
Recommendation systems
And much more...
Here is a Youtube video from 2011 where Google is telling developers how to use this API.
Learn how to recommend the unexpected, automate the repetitive, and distill the essential using machine learning. This session will show you how you can easily add smarts to your apps with the Prediction API, and how to create apps that rapidly adapt to new data.
So you are all pumped up to get AI in your data center. But, here are two things you need to be aware of that can make your projects harder to execute.
First the quality of your data. Everyone has heard garbage in - garbage out. But when you create machine learning systems the accuracy of data can be critical. Google’s Jim Gao, their data center “boy genius” discusses one example.
Catching Erroneous Meter Readings
In Q2 2011,Google announced that it would include natural gas as part of ongoing efforts to calculate PUE in a holistic and transparent manner [9]. This required installing automated natural gas meters at each of Google’s DCs. However, local variations in the type of gas meter used caused confusion regarding erroneous measurement units. For example, some meters reported 1 pulse per 1000 scf of natural gas, whereas others reported a 1:1 or 1:100 ratio. The local DC operations teams detected the anomalies when the realtime, actual PUE values exceeded the predicted PUE values by 0.02 - 0.1 during periods of natural gas usage.
Going through all your data inputs to make sure the data is clean is painful. Google used 70% of its data to train the model and 30% to validate the model. Are you that disciplined? Do you have a mechanical engineer on staff who can review the accuracy of your mathematical model?
Second, the culture in your company is an intangible to many. But, if you have been around enough data center operations staff, their habits and methods are not intangible. They are real and what makes so many things happen. Going back to Google’s Jim Gao. He had a wealth of subject matter expertise on machine learning and other AI methods in Google. He had help deploying the models from Google staff. And he had the support of the VP of data centers and the local data center operations teams.
I would like to thank Tal Shaked for his insights on neural network design and implementation. Alejandro
Lameda Lopez and Winnie Lam have been instrumental in model deployment on live Google data centers.
Finally, this project would not have been possible without the advice and technical support from Joe Kava,
as well as the local data center operations teams.
Think about these issues of data quality and the culture in your data center before you attempt an AI project. If you dig into automation projects it is rarely as easy as when people thought it would be.
May & June are busy months for the data center conferences. In two weeks, I’ll be at DCD SF, and there are many repeats of people I have seen at other conferences. You can find me in the Main Hall chairing the morning presentations. Unfortunately, I need to leave before the end of the conference for another event related to Gigaom Structure in SF which follows the DCD event.
One of the interesting differences between DatacenterDynamics and other conferences is it is peer-led.
I just came back from 7x24 Exchange which is a non-profit and I think you could say they have the idea of peer-led as well.
At the beginning of May was Data Center World, I went to LV to meet with friends, but didn’t have time for Data Center World or IBM’s conference. In May also was Uptime Symposium. On a plane flight to SJC, one of my data center friends asked if I was going to Uptime. I told him no, I am blacklisted for both the conference and exposition. He laughed and said I should get a badge and black out my name. To complete the idea, I have now modified my badge from the last time I attend Uptime Symposium. Some people will exchange conference badges to get into the conference. There is no way this one is going to get me into Uptime. :-) But it does get a bunch of laughs.
This is what the clouds look like now at 5:38a. See those little strands of clouds? Today is partially cloudy.
This morning I saw that Oracle is starting its Cloud Development Group in Seattle.
We're landing in Seattle.
Oracle is building the next great cloud computing environment, from the ground up. We’re committed to building the best high-scale, cost competitive, multi-tenant cloud where the Fortune 1000 will run their businesses.
If you’re a rock star engineer and want to build cutting edge, innovative new services, come join us.
We are building a team of the very best software engineers with expertise and passion for distributed systems, virtualized infrastructure and highly available services. Our aim? To provide our customers with best in class compute, storage, networking, database, security, and an ever expanding set of foundational cloud-based services.
Thanks to Amazon starting AWS, and Microsoft joining in there a large concentration of cloud talent to raid to start up a cloud development group. HP recruited Bill Hilf from Microsoft.
Bill Hilf — who also served as general manager of Microsoft’s Windows Azure cloud service — left Microsoft for HP this summer, and now serves as HP’s vice president of converged cloud products and services. That means he oversees strategy not only for the HP cloud service — a direct competitor toWindows Azure and the leader in the cloud game, Amazon Web Services — but also for the HP software and hardware tools that let businesses build private cloud-like services in their own data centers.
With Oracle and others looking to start-up cloud efforts, Seattle is one of the first choices to start development groups. It is bit ironic that one of the most famous cities being Cloudy is the center of Cloud Software Development.
