Creating a Web based Game with Google App Engine?

Google App Engine is a service provided by Google, free of charge for small to medium-sized projects. Google App Engine is a framework for Python or Java for writing web applications; the remarkable thing about it is that applications written in Google App Engine (GAE) can be uploaded to and run on Google's servers, using their (almost) infinitely scalable database engine. Theoretically, you can publish an application there which grows from 1000 to 1.000.000 users within a week without ever having performance problems or similar, making it a promising platform for online games.

Sounds great? Yeah, it does.

So, what's the problem?
Well, there's some restrictions on what you can do with the datastore; Google doesn't allow a too large amount of queries per page view, and also limits the number of results you may fetch. This, however, usually won't be a problem for a well-written program. Furthermore, you can only do key queries; so you cannot, like in MySQL, find all, whatever, space ships within a bounding box of coordinates. However, after you rearranged your design patterns a bit, this problem is also solvable.

What really makes GAE unsuitable for game programming is the performance issues. In MySQL, there's a socket connection between the client script and the MySQL server. Each query usually has a network latency of less than 1ms, allowing a quite large amount of queries per page view (if you do 30 SELECTs and 30 UPDATEs, you'll be totally fine). However, due to the distributed nature of GAE's Datastore (data is splattered among several servers), every GET or PUT call establishes a new connection, resulting in a network latency of more than 50ms per call. Thus, it's totally impossible to GET and PUT everything that requires to be updated regularly in an average online game; you cannot GET all entities in the beginning to avoid establishing multiple connections (as it's probably more than 1000), and you probably also won't know which ones are affected in advance (so you also cannot only fetch those). Therefore, even with severe optimizations, you'll at the very least need a dozen GETs per page view. This already takes quite a while, at 50ms per query.

So, whatever. Fact is, it's not possible to GET and PUT all required data every page view from and to the datastore, at least not if you use one datastore entity per python object. The next idea is, caching: use GAE's memcache API, store the data in memcache and synchronize it with the datastore every few seconds for persistence. This isn't good, either: you can't fill the cache easily due to the 1000-entity-per-script limitation, and reading and writing still carries a network latency penalty (as a memcache call is a remote procedure call like a datastore call is, too; however, the latency is somewhat smaller). Additionally, what is much worse, you lose all the database advantages: you cannot select "all ships that belong to user A". So, either, you write a query engine yourself, which I really don't want to do (isn't that what the framework should be for?), or you do a for-loop over all the stuff in the cache for every object you are looking for, which will take vast amounts of CPU time.
In my limited imagination, Google could quite easily solve this issue by, for example, implementing a MySQL-like query cache in memcache: exactly like I described above, just in a fast language and with sophisticated query algorithms. Probably it won't work like that, but Google surely could think about something similar that would. Like they currently are, anyway, the data storage engines provided by GAE are not usable for game programming at the moment.

There's also some other things that I disliked about GAE; for example, missing essential python libraries like numpy, and thus not having access to the fast pre-defined vector classes and similar stuff. They also have a so-called "ReferenceProperty" for the datastore which helps referring to other entities; a good thing, in general, but when used as intended, it causes a crapload of problems with python's module structure (circular dependencies, to be exact).

Maybe, it's just me not being used to the way GAE is intended to work. More probably, however, GAE just isn't designed to be a framework to build games on; it's probably well-suited for homepages, online shops, forums or blogs, which do not need to modify large amounts of entities, and also do not implement much logic stuff.