Introduction

In this article we explain how to port applications that were originally designed for App Engine with Django to any environment that supports Django. The helper application provides a Django based implementation of the App Engine APIs and its aim is to minimize the effort of porting your App Engine application to run anywhere using Django.

After a short instruction on how to obtain the gae2django helper application, we demonstrate using the helper so that Rietveld , the code review tool written by Guido van Rossum, runs on any environment that supports Django.

Finally, we talk through how to use the helper with your own App Engine Django application, and give some notes about required changes and potential pitfalls.

To follow the examples in this article, we assume you are using the command line and have a Mercurial client installed on your system.

Obtaining the Helper

The project is hosted on Google project hosting, which uses Mercurial. To obtain the gae2django helper run hg clone http://code.google.com/p/django-gae2django django-gae2django from the command line to fetch the latest version from the project's repository.

The django-gae2django directory contains a Django project structure along with the helper. You should use this same directory structure when including the helper with your existing Google App Engine applications.

The Rietveld Example

In addition to the helper, there's an examples directory with everything prepared to convert Rietveld to pure Django. To learn more about Rietveld visit its project page at http://code.google.com/p/rietveld/ or see it in action at http://codereview.appspot.com .

The core of Rietveld is a Django application called codereview . Rietveld was built originally for App Engine and uses Django's URL resolving and template rendering functionality.

Since Rietveld makes heavy use of the Google App Engine APIs, it will not run by default on pure Django. And maintaining two separate versions of the application would also require the added problem of keeping the features of both versions in sync. Instead, we'll use gae2django to allow the code originally written for App Engine to run in a pure Django environment.

To get started with the Rietveld example, change into the examples/rietveld directory and make all (refer to the README file in that directory for manual setup instructions). make all does three things:

1. Fetches the entire Rietveld application from its repository
2. Applies the patches required to get it running on Django
3. Runs ./manage.py syncdb to setup the database used in this example

During the final step, Django will ask if you would like to create a superuser account. Answer "yes" here and give the required information. You can create additional users later by following the steps outlined in User authentication in Django .

After you have installed Rietveld, run ./manage.py runserver localhost:8000 to start the server. Point your browser to http://localhost:8000 to use Rietveld running on pure Django.

In the next sections, we step through all of the changes the helper made to the original codereview application, but if you want to have a look at the changes first, why not use the code review tool?

To upload the changes made by the helper, run the following command in the codereview directory:

../static/upload.py -s 127.0.0.1:8000 -m "codereview on Django"
Upload server: 127.0.0.1:8000 (change with -s/--server)
Login URL: 'http://127.0.0.1:8000/accounts/login/'
Username: USERNAME_AS_PROVIDED_ABOVE
Password:
Issue created. URL: http://127.0.0.1:8000/1
Uploading views.py
Uploading library.py

Use the username and password of the superuser created when setting up Rietveld. The above command uploads the original and modified files to your local instance of Rietveld. Loading the "Recent Issues" page will show the change as the first issue.

Next, we explain how the helper works and how to use it for your own application.

Installing the Helper in Your Own Project

To install the gae2django helper in your own project, open your manage.py script and add the following at the top before any Django imports:

import gae2django
gae2django.install()

In settings.py add gae2django under the INSTALLED_APPS setting. Add gae2django.middleware.FixRequestUserMiddleware to MIDDLEWARE_CLASSES directly after Django's authentication middleware.

The App Engine API replacement is exposed as the google module to Python. You can see how the API replacement behaves by running:

./manage.py shell
>>> from google.appengine.api import users
>>> users.create_login_url("foo")
      '/accounts/login/?next=foo'

Now that gae2django is installed, calls to App Engine's Users API are mapped to Django's authentication system, App Engine models and properties are mapped to Django models and fields, GQL queries are translated to model filters, and calls to the Mail API are handled by Django's mail module.

Changes to Your Application

If you are following along, you've probably noticed the patches directory and the rietveld_helper application. These contain modifications to the original application, and are required to run Rietveld using pure Django. Not all differences between App Engine and pure Django's are automatically covered by using the helper.

Depending on your use the App Engine APIs, a few additional changes to your original App Engine application might be required to run on pure Django. The following sections give you some hints about the differences between both frameworks and how these issues are resolved for Rietveld. This should allow you to implement similar changes for your own application.

The User Class

Semantically, the most significant difference between pure Django and App Engine involve the User's API. With Django's User model, email is a field, but in the App Engine Users API the email is retrieved using a method. Also, Django doesn't have the nickname method which is available in App Engine. When gae2django translates Rietveld, every call to user.email() in Rietveld is replaced by user.email and the user.nickname() is set from the profile class.

Anonymous Users

Another difference between the App Engine Users API and Django's behavior is how anonymous users are handled. If the current user isn't logged in on App Engine users.get_current_user() returns None . Django has an AnonymousUser instance. With Rietveld we solve this by adding another middleware class in rietveld_helper . This class removes the user that is added to the request instance when the user is anonymous. Since the Rietveld only checks request.user and constructs a User object from scratch, this is all we need to do.

You may need to implement additional solutions for this issue with your own application, such as replacing conditional statements like user is None with user.is_anonymous() .

Models and Properties

The Datastore API implementation of the helper is fairly robust. For the Rietveld example no changes to the original models or GQL queries were made.

The gae2django implementation of google.appengine.ext.db.Model creates additional model fields (prefixed with gae_* ) to store a key for an instance and, if necessary, to store an ancestry path to resolve parent relationships.

GQL queries are analyzed and translated into Django field lookups . The returned QuerySets work almost exactly the same as result sets in App Engine.

Application Specific Changes

The changes to upload.py are specific to the code review application. upload.py is a command line script to upload changes in a version controlled working tree to the review server and it requires authentication on the server from the command line. The modified version of this script in the static directory uses Django's login form to retrieve an authentication cookie from the Django server. Have a look at patches/upload.diff if you are interested in this more advanced topic.

In addition to the middleware class for the Users API, rietveld_helper provides the ROOT_URLCONF for Django. The URLs for Rietveld are imported in rietveld_helper/urls.py and are extended by additional, Django specific, URLs for sign in, sign out, and for serving static files.

Upcoming Development

The gae2django helper application is still in an early stage of development and not all parts of Google's App Engine APIs are currently covered by this project. At the moment the project focuses on Rietveld as an reference implementation. Upcoming development will include greater coverage of the App Engine APIs, unit tests, and other example applications. If you know an App Engine based application suitable for demonstrating and testing the helper's functionality, read the section about contributing to this project below.

Contributing

Please file bug reports and feature requests on the project site at http://code.google.com/p/django-gae2django/issues/entry or if you have code to contribute upload it to http://codereview.appspot.com and add [email protected] as reviewer.

Conclusion

The gae2django helper application can be used to simplify the porting of applications designed for Google App Engine to pure Django projects. It simplifies the reuse of the application code as it provides Django adapters for most of App Engine's methods and classes. It reduces the effort porting an application to Django by freeing the developer to deal only with cases when the frameworks differ and application specific issues.