Is Elastra one of Amazon’s Cloud Computing infrastructure tools? An awesome PDF to understand a better approach to infrastructure management

January 19, 2010 Dave Ohara

I plan on having a meeting with Elastra next week when I am in the bay area. I wrote about their tools last week.

Elastra’s Cloud Computing Application Infrastructure = Green IT with a Model approach

Elastra connects the power use in the data center to the application architects and deployment decision makers.

Plan Composer function lets customers set their own policies based on application needs and specific power metrics (such as wattage, PUE, number of cores, etc.). Therefore, if an application requires 4GB of RAM and two cores for optimal performance, and if the customer is concerned with straight wattage, Elastra’s product will automatically route it to the lowest-power 4GB, dual-core virtual machine available.

Gigaom has a post on Elastra’s Cloud Computing infrastructure addressing greener services.

Elastra Makes Its Cloud Even Greener

By Derrick Harris Jan. 12, 2010, 2:51pm 1 Comment

0 0 33

Checking out the investors, look what I found.

Amazon.com, Inc.

(NASDAQ:AMZN), a Fortune 500 company based in Seattle, opened on the World Wide Web in July 1995 and today offers Earth's Biggest Selection. Amazon.com, Inc. seeks to be Earth's most customer-centric company, where customers can find and discover anything they might want to buy online, and endeavors to offer its customers the lowest possible prices. Amazon.com and other sellers offer millions of unique new, refurbished and used items in categories such as books, movies, music & games, digital downloads, electronics & computers, home & garden, toys, kids & baby, grocery, apparel, shoes & jewelry, health & beauty, sports & outdoors, and tools, auto & industrial.

Amazon Web Services provides Amazon's developer customers with access to in-the-cloud infrastructure services based on Amazon's own back-end technology platform, which developers can use to enable virtually any type of business. Examples of the services offered by Amazon Web Services are Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3), Amazon SimpleDB, Amazon Simple Queue Service (Amazon SQS), Amazon Flexible Payments Service (Amazon FPS), and Amazon Mechanical Turk. www.amazon.com

Then digging more I found this architecture PDF by Stuart Charlton.

I like this picture from his personal site.

better han the corporate one from Elastra.

Back to his PDF. Much of the information in the pdf is on the Elastra technology site, but I found it easier to read the PDF to understand Stu’s thinking.

The introduction targets the use of Elastra for the architect, stating the problems.

Reference Architecture Introduction

In today’s age of on-demand access to applications, compute, storage, and networks, modern IT applications and service management has many complications:

Applications can be deployed across organizationally & geographically distributed data centers. The technology in these data centers, from virtualization platforms, to host, storage, and network infrastructure, is typically heterogeneous, and not necessarily managed with uniform policies and interfaces.

The performance, scalability, and availability characteristics of an application are due to a complex combination of design and operational decisions. The greatest impacts on these factors are due to decisions in the architecture and development of the application, before configuring the data center infrastructure.

Application and infrastructure management is complex and inter-disciplinary. It’s unlikely a system can be diagnosed and maintained by one person to keeping the system design & configuration in their head. Application design, administration, and management typically is a collaborative activity across specialists; there is no “one-size fits all” design tool, management tool or application platform.

The design goals are right on.

Three design goals for an end-to-end cloud design approach include:

Separated Applications from Infrastructure, through modeling the application in terms of its architecture and infrastructure requirements, without tying the application to a specific set of underlying infrastructure

Enabling Computer-Assisted Modeling and Control Automation, provided by a set of control agents and user-guided by graphical design tools. This could help IT architects and operators determine design constraints on the application, match the design to the underlying infrastructure, and enable goal-driven automation to deploy, scale, or recover their IT systems on demand.

Explicit Collaboration To Enact Changes, through models that codify, relate and analyze the constraints and preferences that are appropriate to stakeholders across enterprise IT: from architects and developers, through operators, administrators, and managers.

The document has many great ideas including the use of models.

Declarative models are useful ways to drive complexity out of IT application design and configuration, in favor of more concise statements of intent. Given a declaration of preferences or constraints, an IT management system can compose multiple models together much more effectively than if the models were predominantly procedural, and also formally verify for conflicts or mistakes. On the other hand, not everything can be declarative; at some point, procedures are usually required to specify the “last mile” of provision, installation, or configuration.

Here is a diagram showing VMware Virtual Center (Private Cloud Inventory) and Amazon EC2/EBS (Public Cloud Inventory).

100, one hundred job openings in Amazon Web Services!

January 19, 2010 Dave Ohara

You can read many things into what it means that AWS has 100 open positions in its developer resources.

Job Openings in AWS Developer Resources

I'd like to highlight a few of the nearly one hundred open positions on the AWS team. If you are a software engineer, manager, technical writer, program manager, or product manager and you like to work on cutting-edge projects with world-scale impact, you owe it to yourself to look at these positions.

Is Amazon having a hard time finding the right people?

Is AWS growing so fast they can’t staff quick enough?

Does this have an effect on service levels?

The AWS posted some of their postings to get your attention.

If you want to make it even easier for our development community to build amazing applications, I'd urge you to check out these positions:

Senior Program Manager – You are passionate about great customer user experiences and understand how to organize and drive distributed development projects running under tight deadlines. You will own the process, schedule, and success of several AWS teams building rich internet applications which enable customers to use our cloud computing services through point-and-click interfaces (position 92278).

Senior Software Developer – You will fundamentally change the way people build, deploy, and manage cloud applications in a small team running fast to offer AWS customers a brand new service. You will apply your experience building reliable, scalable, data driven distributed applications, knowledge of web protocols, and in-depth knowledge of several application servers, Linux tools and Java EE architectures. (position 109479).

PHP SDK Developer – You are an experienced PHP developer with a passion for PHP APIs and a desire to help thousands of PHP developers use the AWS cloud to build PHP-based web and Facebook applications quickly (position 110634).

Software Developer – You are a self-starting, self-directing Java developer who is creative and passionate about developing web-based applications which will be used by thousands of developers (position 110635).

Senior Development Manager – You are a dynamic, innovative, and hands-on software development manager who will lead the production of AJAX web apps that enable customers to use our cloud computing services through point-and-click, web-based interfaces. You will be responsible for leading a central platform team of talented and nimble engineers that work with several teams across AWS to produce great developer tools and websites for our customers (position 110632).

Senior Technical Writer – You are passionate about making it easy for developers to get started with AWS services and to gain a deeper understanding through technical documentation. You believe strongly in providing world-class, error-free documentation and will get your hands dirty with code samples and command line tools to validate your understanding (positions 111118, 108280).

Software Development Engineer / Test - You are a customer advocate and passionate about ensuring the AWS management console provides an error-free user experience which delights customers by automating testing and deployment processes (positions 111171, 110638)

But the posting that got my attention was this Sr. Business Development Manager – AWS Federal Government.

As a business development leader for Amazon Web Services (AWS) focusing on the US Federal Government Sector, you will have the exciting opportunity to help shape and deliver on a strategy to build mind share and adoption of Amazon's platform of infrastructure web services (Amazon S3, Amazon EC2, Amazon CloudFront, Amazon SimpleDB, and Amazon SQS). Your broad responsibilities will include helping to further expand AWS to key federal government agencies and partners by establishing both business and technical relationships, and managing the day-to-day interactions with these organizations. The ideal candidate will posses both a business background (with a focus on the federal market) that enables them to drive an engagement and interact at the CXO/VP level, as well as a technical background that enables them to easily interact with software developers and architects. He/she should also have a demonstrated ability to think strategically about business, product, and technical challenges for both AWS customers as well as AWS.

It will be interesting to see the battle for cloud computing solutions in the federal gov’t sector. This is where IBM, HP, Cisco, Oracle, and Microsoft will be fighting hard head to head.

Amazon Web Services network issues, transparency of data spreads from cloudkick

January 15, 2010 Dave Ohara

Amazon has a phenomenal amount of data on AWS and much of that data is shared with the partner community,. One partner is cloudkick.

graphs

visualize important metrics like http latency and ping - from multiple data centers.

monitoring

set up monitoring in a just a few clicks
get alerts when services go critical
monitor ping, http, https & ssh

Cloudkick wrote this post on Amazon’s network performance issues.

Visual evidence of Amazon EC2 network issues

Update: After seeing this story picked up, we ran our numbers again to demonstrate a broader picture of the issue at hand. We ran a sample of ping latency across several hundred EC2 instances managed by Cloudkick located in the US-East availability zone. Below you will see that the issues started around Christmas, and have been on-going since.

An average ping latency of 50ms (as seen in the period between 11-30 and 12-14) is relatively low and normal. The spikes in latencies up to 1000ms are definitely abnormal, and should never be encountered on healthy private network.

Amazon has a great track record in performance and reliability, so this is why we are so surprised by this data. As Amazon spokesperson Kay Kinton said, “When customers report a problem they are having, we take it very seriously. Sometimes this means working with customers to tweak their configurations or it could mean making modifications in our services to assure maximum performance.”

Original post:

A couple of weeks ago we noticed that our ping latency graphs on Cloudkick looked very odd.

This post was picked by DataCenterKnowledge

Amazon: We Don’t Have Cloud Capacity Issues
January 14th, 2010 : Rich Miller

A chart from CloudKick looking at latency for resources running on Amazon EC2.

One of the key selling points for cloud computing is scalability: the ability to handle traffic spikes smoothly without the expense and hassle of adding more dedicated servers. But this week some users of Amazon EC2 are reporting that their apps on the cloud computing service are having problems scaling efficiently, and suggesting that this uneven performance could be due to capacity problems in Amazon’s data center

And Register as news, and refers to DataCenterKnowledge and cloudkick.

"Amazon has a great track record in performance and reliability, so this is why we are so surprised by this data," reads Cloudkick's blog post on the matter.

Cloudkick's numbers are limited to Amazon's "US-East" availability zone. EC2 serves up processing power from two separate geographic locations - the US and Europe - and each geographic region is split into multiple zones designed never to vanish at the same time.

enStratus, an outfit similar to Cloudkick, confirms the latency increase, but it says the spike is significantly smaller. Response time from the company's network into "all regions" of the Amazon cloud increased by 10 per cent on January 9, enStratus CTO George Reese tellsThe Reg, and it has remained roughly that high ever since. Reese's sample size is around 300 server instances.

Cloudkick and enStratus released their data in the wake of a blog post from Alan Williamson, co-head of the UK-based cloud consultancy AW2.0, who asked whether Amazon was experiencing capacity issues after one of his customers experienced a serious slowdown beginning at the end of last year. "We began noticing [the problem] around the end of November," Williamson tells The Reg. "We had been running with Amazon for approximately 20 months with absolutely no problems whatsoever. We could throw almost anything at them and it wouldn't even hiccup."

Echoing what Cloudkick and enStratrus have seen, Williamson says he eventually traced the problem back to network latency. On the application in question, the average time needed to turn around a web request jumped from about 2 to 3 milliseconds to about between 50 and 100 milliseconds.

Responding to an inquiry about the post from Data Center Knowledge, Amazon said that their infrastructure does not have capacity issues. And this afternoon, the company sent a similar statement to The Reg.

"We do not have over-capacity issues. When customers report a problem they are having, we take it very seriously," a company spokeswoman said. "Sometimes this means working with customers to tweak their configurations or it could mean making modifications in our services to assure maximum performance."

Marketing Cloud Computing, why Amazon and Google have the advantage

January 15, 2010 Dave Ohara

i have been resisting the cloud computing hype as I am sure many of you have, but I’ve started to look at Cloud Computing as an agent to change the behavior of IT to be greener in the data center.

There are skeptics to the hype.

Merrill Lynch: Cloud Computing Market Will Reach $160 Billion...Really?

Written by Alex Williams / November 25, 2009 11:40 AM / 8 Comments

« Prior Post Next Post »

The estimates for cloud computing can make you wonder sometimes about what to believe. Analyst firms and it looks like investment houses, can be notorious for wild estimates about market sizes.

So we have to wonder about the estimates from Merrill Lynch, which is estimating the cloud computing market to reach $160 billion by 2011.The estimate includes $95 billion in business and productivity applications.

Whoa! That makes cloud computing one of the fastest growing markets in the world.

But Merrill Lynch is not alone in its lofty estimates. Earlier this year, Gartner pegged the market at $150 billion by 2013.

…

In other words, cloud computing will be huge but to call it a $160 billion market seems like a form of hype that can lead to all kinds of issues. It's almost reminiscent of the dot-com bubble.

And look what happened there.

The same report discusses the business and and technical reasons of why Cloud Computing is good.

But then you need to look at the dynamics in play. IT is built on legacy systems, custom, built to order environments. Cloud computing provides a level of automation.

From the PriceWaterhouseCoopers summer Technology Forecast:

"Legacy IT soaks up much of the available IT budget and is a primary barrier to IT responsiveness and overall business agility."

The report goes on to say that cloud will be necesssary for automating the world of IT:

"...IT must adopt an architecture that creates loose coupling between the IT infrastructure and application workloads. It also must modernize and automate IT's own internal business processes for provisioning, managing, and orchestrating infrastructure resources."

With all this speculation it is hard to know what works in cloud computing.

Which is why I think Amazon and Google have the advantage. They have the data that shows where there is the adoption of cloud computing.

Marketing the Cloud Computing covers an interesting point here.

May 12, 2009

Marketing Cloud Computing: Uncharted Territories

One of the aspects of cloud computing that receives too little attention is the massive change it brings to how software and IT infrastructure are marketed, sold, purchased and serviced. Through my work at GigaSpaces, and now advising start-ups and large companies with various cloud offerings, I have come to realize how much marketing cloud computing is still uncharted territory -- and especially when it comes to the enterprise.

Many of the value propositions cloud brings to the table have been commonplace in the consumer Internet for more than a decade: self-service, ease-of-use, pay-by-the-drink pricing and so on. The same is true from the vendor's point-of-view: a low-touch, low-value, high-volume and short sales cycle. It's no surprise then that consumer-oriented companies, such as Amazon and Google, are the ones leading the charge in what is essentially a B2B market.