Stephane's thoughts corner…
Stephane Bailliez's thoughts on everything

Absolutely nothing is employed by my Male Impotence am Now Debt Relief Debt Relief i stayed with this issue for all his life?

Put simply, Debt Relief Consumer debt relief the response to it Best electronic cigarette online Njoy npro electronic cigarette Jeux de casino en ligne gratuit bonus casino is an unequivocal no. In right now community, with Generic Cialis Cialis generic uk a variety of Online debt consolidation loan Bad debt consolidation treatment methods readily available, no-one must ever before really Buy Viagra Generic Viagra feel that you have no solution for his Viagra Online Viagra Online or her unique particular sort of Erectile Buy Viagra Viagra online uk Dysfunction. In young gentlemen, it can sometimes be their Levitra 10mg Buy Levitra operation anxiety is so entrenched that Electronic cigarettes for weed Top rated electronic cigarettes even How does online blackjack make money Rigged online blackjack substantial dosage of Viagra, Cialis or Levitra Viagra Online Viagra

«« Ubuntu 9.10 Server 64 bits in VMWare 7 | Search Spotify using Google App Engine and Python »»

January 31, 2010

Google App Engine – A practical overview

Filed under: Uncategorized — stephane @ 5:31 pm

I have been having a more detailed look in the last couple of days at Google App Engine (it’s about time). The interest was more into figuring out practically how was the infrastructure and API like as well the potential paradigm shift one needed to get in order to take advantage of GAE and the supposed (mostly infinite) horizontal scalability. After all, it is an interesting offer, as it a web-hosting environment where you can get some things done to a reasonable level for free.. which you would not get yet using Amazon AWS (you’ll still need to pay for that constantly running EC2 instance).

I didn’t test yet all features of the API such as image transformation or XMPP, but focused mostly on the datastore. Luck or not the datastore seems to be plagued by latency issues. While you are supposed to deal with some degrees of failure when using cloud services, the current situation seems beyond normal and probably a bit challenging for real applications. Messages in the forums clearly indicate the status is worse than what the official status shows or we’re all unlucky. I’d be really curious to hear about production behaviors not too tainted by marketing pitch.

As of now, there is large amount of failures due to timeout, should it be when reading small or writing even the smallest entities. For writes, the Task Queue can help alleviate failure issues.Your mileage may vary, as the tasks are all URL based you will still need a bit more work than you would likely expect instead of just handling a simple datastore put in your code, but all in all the ‘retry until it’s ok’ implementation of the task queue can really be helpful. This is of course assuming you can live in most cases with delayed writes.

For read, this is a bit more problematic. Given the latency of the datastore for get and the failure rate, it does not seem doable to solely rely on it, even for the simplest personal CRUD application. Assuming you had just a todo list app, it would be kind of frustrating to save your entry about 10 times before it is saved and can be displayed in the list of entries. So no matter what, using the memcache API as much as possible seems to be a must, not for performance reason but simply because it is more reliable than the datastore. (one could argue this is a performance issue).

Despite these shortcomings, I’m favorably impressed with the Google AppEngine Java SDK, it works really well, the API is well done and is easy to use. Integration in Jetbrains IDEA 9.0 is pretty convenient.

Deployment is a breeze as well though it could do with a bit more polished error feedback, you can have a syntax error in your cron jobs which will go unnoticed in local deployment but upload to production will fail with a totally vague and irrelevant message. Also expect some large behavior differences however with the local datastore as it is much more flexible in term of what you can store. Typically, you can persist tens of thousand entities at once in local in a very fast put operation, but it is limited to 500 entities in production and the delay can be significant and will timeout. The infrastructure is not simulated either, so long running queries are not canceled in local like they tend to be in production. Overall while the local datastore is not exactly fast, the latency in production is at least an order of magnitude worse, so this something to get used to.

Bulk loading (or lack of that is) in the Java API is a bit of annoyance and you need to do it using the Python one.. which means you need to duplicate your models in both languages. It is manageable but not terribly attractive at the moment so I hope they will introduce it sooner rather than later.

That being said, I find myself more and more frustrated at the Java web framework landscape compared to the simplicity of doing it in Python. There sure is a lot of unnecessary complexity for simple things in the Java web environment those days. The thing that I still appreciate most compared to Python is the quality of the tools available for development and that as soon as you need to do something relatively complex, you will likely find a mostly solid existing solution for it.


  1. Forget JDO/JPA too… is something a friend of mine is working on that is better suited to GAE than JDO/JPA is.

    Comment by jon — February 2, 2010 @ 11:59 pm

  2. Interesting. Thanks for the link Jon, I will have a look at it. I was using JDO but cannot say I’m impressed. Overall what’s annoying me is the issues with composite keys. I wish there was also the possibility to specify a custom key factory through annotations in the API, right now I tend to create the composite key within the dao.

    Comment by stephane — February 3, 2010 @ 12:18 pm

  3. Yea, Ofy puts a much better interface on the Datastore than JDO. Key’s are only the start of the problem. Detaching objects is next. =)

    Comment by Jon — February 3, 2010 @ 7:28 pm

RSS feed for comments on this post. TrackBack URI

Leave a comment

Powered by WordPress

Warning: Unknown: open(/tmp/sess_6b807ad20f232f1ed634a5f43c353222, O_RDWR) failed: Read-only file system (30) in Unknown on line 0

Warning: Unknown: Failed to write session data (files). Please verify that the current setting of session.save_path is correct (/tmp) in Unknown on line 0