Monday, August 28, 2006

Amazon Elastic Compute Cloud (EC2) - killer apps

I must admit that I am pretty excited about the recently announced service from Amazon (still in Beta). The Elastic Compute Cloud, or "EC2". It has nothing to do with Books or ISBN numbers. And, it's pretty darn cool. EC2 in a nutshell:
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers. [...] Amazon EC2 presents a true virtual computing environment, allowing you to use web service interfaces to requisition machines for use, load them with your custom application environment, manage your network's access permissions, and run your image using as many or few systems as you desire. [...] Amazon EC2 enables you to increase or decrease capacity within minutes, not hours or days. You can commission one, hundreds or even thousands of server instances simultaneously. Of course, because this is all controlled with web service APIs, your application can automatically scale itself up and down depending on its needs.

Ok, that's a bit of marketing speak. Bottom line: EC2 provides a highly scalable pool of compute servers that are hosted by Amazon. You pay by the drink (only for what you use), so it is truly a Utility Computing model. And, best of all, it is all programmable and managed by web services requests, which means that you can scale your app up and down very easily and completely on-demand. For more, read the FAQ. This is gonna be big.

And, to be perfectly clear, you never have to order a server, receive it, unbox it, install it, diagnose it when it's broken, fix it, upgrade it when it's old, add more A/C when the server room gets too hot, buy a ton of bandwidth into your site to account for the expected "peak load", etc. Lots of headaches go away through a hosted model. But, this is so much better than the usual dumb hosting model, which provides you with either a physical server or a virtual private server - those solutions have no real and easy way of scaling your application on-demand.

Now, the guys on the Amazon Web Services team are seriously good. I kid you not. They have been on the forefront of opening up the core set of infrastructure and data that Amazon has built to find ways of increasing Amazon revenues. Did I say that this is gonna be big? Their blurb:
Amazon has spent ten years and over $1 billion developing a world-class technology and content platform that powers the Amazon web sites for millions of customers every day. Using Amazon Web Services, developers can build software applications leveraging the same robust, scalable, and reliable technology.

But, EC2 takes a very big step forward. Especially now. This kind of capability allows startups to have a world-class and scalable infrastructure from day one. No need to buy massive racks of servers and bandwidth pipes. If the startup's service takes off, you can spin it in-house later. I'm sure Amazon will end up pricing this to entice you to never want to bring these capabilities back in-house (current pricing feels a bit high, but many details are not yet fully known).

Jon Udell has posted a screencast showing the EC2 service running his application, just as his local servers do. No big deal, but you can tell that he's pretty excited. Maluke has a nice write-up here, including this snippet:
Please note that EC2 is not limited to web hosting applications, far from it. It makes even more sense to use it for virtual render farms, to run simulations and other tasks that require a lot of computing power but are usually executed only once in a while. So if you need for ex. 100 instance-hours to complete the computation, you can make your own cluster of 20 machines of similar power (will cost about $10000 for hardware alone) and complete the task in 5 hours, you can use EC2 to create this virtual cluster, compute and then shut it down when done and pay much less — $10 per job. Or you could use EC2 to create a 200 machine virtual cluster, complete the job in half an hour and pay the same $10 for it. Think about that.

Of course, I'm a bit biased. The service will need some killer virtualization management tools, like what we built at Cassatt. The Cassatt system (many patents pending) is a goal-oriented system designed specifically to scale application tiers on-demand, based on your specifications. Of course, nothing works with EC2 today, but having easy management tools allows this to be very easy to deploy and manage for production-type services. That's what Cassatt does. Disclosure: I am founder and was the founding CTO of Cassatt.

Another somewhat biased example of a killer application for EC2. Using it as your dev/test server farm. My buddies over at Gauntlet Systems (recently acquired by Borland) had a hosted build/test system which just makes a lot of sense to me. Plugging Gauntlet Systems into EC2 would make for a very scalable software test farm with very little need to build the infrastructure yourself (just pass the charges back to your customers, since you would be charging them by the drink anyway; basically, let Amazon do the nasty accounting for you!).

Final somewhat biased example here is the use of Akimbi Systems plugged into EC2. Akimbi was recently acquired by VMware. They have a great tool for moving your dev/test environment to a virtualized and very dynamic environment. Ground-breaking stuff. Having a module for Akimbi that plugs into EC2 would allow you to fully outsource your test environment. Disclosure: I am an advisor to Akimbi.

OK, I'm excited too. Nice work Amazon!

Tags: , , , , , , ,


  1. [...] love the Amazon Web Services guys (and have written many articles about what they are up to). I’m a big fan. The fact is that Google App [...]

  2. good post, thanks for the descriptive article about cloud computing

  3. I've just come across your site regarding Cloud Computing and Virtualization. There is some good information and we may be interested in including you in our blog. Please feel free to contact. Cheers

  4. [...] September 8, 2006 Startups , Venture Capital 2 Comments Earlier I wrote about Amazon’s new Elastic Compute Cloud (EC2) and some of the Killer Apps needed from a virtualization management perspective to really get it to take off. Let me add one to the [...]

  5. Bluehost is ultimately the best hosting company with plans for all of your hosting needs.