After the Cloud: The Power of the Little Guy

Non-Standard Applications

However, what about the numerous applications out there which cannot be shoe-horned into the CGI model that figures so prominently into many cloud offerings? What about the applications that need multiple instances running? Or ones that need access to file systems? Or network communications?

There are certainly many ways in which one could re-architect an existing solution to fit into various cloud models, but that's often extremely expensive and, as such, a risky business proposition. Indeed, the power of the cloud has really been that it offers exceptional reliability and scalability without the need to rewrite software.

One potential solution for non-standard applications could be provided by the likes of Amazon EC2 which has exposed the virtualization aspects of cloud implementations to their end customers: you tell them how many instances you need, and they'll start them up for you and run them only as long as you need.

This is fantastic. And for the near term, probably sufficient, especially when one considers the flexibility provided by such open source projects as Eucalyptus, allowing organizations to do like Amazon EC2 does, but with their own infrastructure.

As a developer though, my interests are not well-suited to the Amazon model, and this leaves me with a problem... one that I've had for quite a while. Simply put, it's a problem of power. I have many applications that don't need the power supplied by servers. They don't need the RAM, the CPU, the storage, or the bandwidth. Though they need those things in a limited sense, more than anything, they need the network. I think that others also have this power problem, but it's one that we've simply come to accept as the nature of the beast: we get more than we need, and we just have to keep paying for it.

There are trends that indicate it might not be a good idea to continue ignoring this aspect of our infrastructure. This old-school approach to hosting applications is likely exacting a pretty heavy toll on bottom lines. Even more importantly, though, there are many business opportunities to be explored with the elimination of this over-provisioning of power.

By looking at where we've come, we can extrapolate potential destinations. There's one in particular that promises to be quite interesting...

Distributed Services

The modern internet consumer has come to depend very deeply upon distributed services. One provider will host a user's email, another will provide space for that user's web pages. Users manage their domain names and DNS with registrars. We also make use of such things as VoIP solutions, online banking, and shopping – many of which provide APIs to allow programmers access to the data these various organizations provide for consumption by third-part software.
Distributed services has allowed for an evolution of information consumption through the natural selection of the best of breed. In many ways it has increased the freedom of the consumer with regard to their information. But from a business perspective, the most interesting benefit has probably been the countless ways such distribution has allowed for unique combinations of services with applications and markets that no one foresaw when the individual services were first published.

With all of this goodness and innovation, we are left with a disturbing trend: more and more of our data is being managed managed in facilities and by companies that are paying a great deal of money to maintain the infrastructure we now implicitly depend upon. The power consumption of data centers is monstrous, but there are other cost considerations as well. Land leases. Telecommunications. Physical security. Sales. Customer support. The list of expenses is frightening to mere mortals, and seemingly endless. When viewed from a distance, such a "structure" seems exceedingly baroque, ornate not in the workings of physical beauty, but rather in the almost fractal nature of its extensive underpinnings. Due to the operating expenses these guys incur, they need to charge their end users for this as well. Everyone pays, no matter if they need to use all the power they get.

Arguably, some cloud providers need the hardware, policies, and support in place for their own organizations. Providing a cloud solution allows them to mitigate the costs they would otherwise be absorbing completely on their own.

Regardless, is all of this really necessary to deliver the next generation in distributed services? Is there absolutely no other way to do it?