Monday, February 23, 2009

AppEngine

I have been playing a bit with AppEngine lately. I is a cloud computing service geared towards running Python based web applications on Google's data-center infrastructure. Applications can only run short-lived http request handlers, which will be terminated if they exceed their CPU quota. On the other hand, many parallel instance of the same application can be spawned transparently if the load requires it.

AppEngine comes with its own proprietary non-relational data-store, which maps well to Django's object persistence model for example. And since Django is such a popular framework for web development in Python, it comes pre-packaged in the AppEngine API.

I think this is a very compelling solution for many new web applications since it takes a lot of pain out of building and scaling infrastructure. For those who don't believe that this is a significant problem, here is a very interesting presentation from the MySQL 04 conference about how LifeJournal scaled up from a hobby project on a single machine in a dorm room to high traffic website running on a whole cluster of machines.

One of the biggest strengths of web-apps seems to be the ability to innovate very quickly - throw some things out very quickly and see what sticks. Having a platform with a very low barrier to entry - no cost, no system administration headaches - might help to start some services which otherwise would never have made it off the ground. And should the new service become a runaway success - or even just being slashdotted, the web equivalent of a one-hit-wonder - then having an infrastructure which can scale seamlessly may just prove to be invaluable.

My biggest gripes with AppEngine is the lack of support for batch jobs and reliable database administration. Not everything can be easily implemented as short-lived web-requests - in particularly not backing up, restoring or upgrading and entire large data-store in an atomic or at least consistent fashion. The good news is that at least according to the roadmap, some of these problems are know and are being addressed.

And there is always the problem with lock-in to a single-vendor proprietary platform, which not everybody is comfortable with. I guess a Django based application, written to run on AppEngine, could be ported to run on a traditional server hosting, but any such transition would bring back all the infrastructure problems AppEngine was abstracting away in the first place.