Learning objectives
- Learn how to build and deploy an App Engine app, a simple guestbook
Prerequisites
- Basic familiarity with Python
- PC, Mac, or Linux computer with Python 2.7 installed
- The Introduction to App Engine class
- App Engine 101 in Python : the predecessor to this class
Related
Amy Unruh, Dan Sanderson, Oct 2012
Google Developer Relations
Introduction
Storing data in a scalable web application can be tricky. A user could be interacting with any of dozens of web servers at a given time, and the user's next request could go to a different web server than the previous request. All web servers need to be interacting with data that is also spread out across dozens of machines, possibly in different locations around the world.
With Google App Engine, you don't have to worry about any of that. The App Engine Datastore is a schemaless object datastore providing robust, scalable storage for your web application, with the following features:
- No planned downtime
- Atomic transactions
- High availability of reads and writes
- Strong consistency for reads and ancestor queries
- Eventual consistency for all other queries
The Python Datastore interface includes rich data modeling APIs and a SQL-like query language called GQL.
As part of App Engine's infrastructure, the Datastore takes care of all distribution, replication, and load balancing of data—and provides a powerful query engine and transactions as well. App Engine includes two data modeling APIs for Python, which use the Datastore behind the scenes. In this course, we'll use the newer of these, the NDB API ; this lesson shows you how to use it to persist and manage the guestbook application's data.
The High Replication Datastore and Entities
Before we look at the code, let's introduce some Datastore concepts.
App Engine's High Replication Datastore (HRD) uses the Paxos algorithm to replicate data across multiple data centers.
Data is written to the Datastore in objects known as entities . Each Datastore entity is of a particular kind , which categorizes the entity for the purpose of queries. Each entity has a key that uniquely identifies it. The key consists of the kind of the entity; an identifier for the individual entity; and an optional ancestor path locating the entity within the Datastore hierarchy. More specifically, an entity can optionally designate another entity as its parent; the first entity is a child of the parent entity. The entities in the Datastore thus form a hierarchically structured space similar to the directory structure of a file system. An entity's parent, parent's parent, and so on recursively, are its ancestors; its children, children's children, and so on, are its descendants. An entity without a parent is a root entity.
The Datastore is extremely resilient in the face of catastrophic failure, but its consistency guarantees may differ from what you're familiar with. An entity and its descendants are said to belong to the same entity group; the common ancestor's key is the group's parent key, which serves to identify the entire group. Queries over a single entity group, called ancestor queries , refer to the parent key. Entity groups are a unit of both consistency and transactionality: whereas queries over multiple entity groups may return stale, eventually consistent results, those limited to a single entity group always return up-to-date, strongly consistent results.
The code in this lesson organizes related entities into entity groups, and uses ancestor queries on those entity groups to return strongly consistent results. We’ll briefly discuss some application design considerations related to this organization. For more detailed information, see Structuring Data for Strong Consistency in the App Engine documentation.
A Complete Example Using the Datastore
Here is a new version of
helloworld/main.py
that stores greetings in the Datastore. The rest of this lesson and the next will walk through the new pieces.
import cgi
import urllib
import webapp2
from google.appengine.ext import ndb
from google.appengine.api import users
class Greeting(ndb.Model):
"""Models an individual guestbook entry with author, content, and date."""
author = ndb.UserProperty()
content = ndb.StringProperty()
date = ndb.DateTimeProperty(auto_now_add=True)
@classmethod
def query_book(cls, ancestor_key):
return cls.query(ancestor=ancestor_key).order(-cls.date)
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.out.write('<html><body>')
guestbook_name = self.request.get('guestbook_name')
# There is no need to actually create the parent Book entity; we can
# set it to be the parent of another entity without explicitly creating it
ancestor_key = ndb.Key("Book", guestbook_name or "*notitle*")
greetings = Greeting.query_book(ancestor_key).fetch(20)
for greeting in greetings:
if greeting.author:
self.response.out.write(
'<b>%s</b> wrote:' % greeting.author.nickname())
else:
self.response.out.write('An anonymous person wrote:')
self.response.out.write('<blockquote>%s</blockquote>' %
cgi.escape(greeting.content))
self.response.out.write("""
<form action="/sign" method="post">
<input type="hidden" value="%s" name="guestbook_name">
<div><textarea name="content" rows="3" cols="60"></textarea></div>
<div><input type="submit" value="Sign Guestbook"></div>
</form>
<hr>
<form>Guestbook name: <input value="%s" name="guestbook_name">
<input type="submit" value="switch"></form>
</body>
</html>""" % (cgi.escape(guestbook_name), cgi.escape(guestbook_name)))
class Guestbook(webapp2.RequestHandler):
def post(self):
# Set parent key on each greeting to ensure that each
# guestbook's greetings are in the same entity group.
guestbook_name = self.request.get('guestbook_name')
# There is no need to actually create the parent Book entity; we can
# set it to be the parent of another entity without explicitly creating it
greeting = Greeting(parent=ndb.Key("Book", guestbook_name or "*notitle*"),
content = self.request.get('content'))
if users.get_current_user():
greeting.author = users.get_current_user()
greeting.put()
self.redirect('/?' + urllib.urlencode({'guestbook_name': guestbook_name}))
app = webapp2.WSGIApplication([
('/', MainPage),
('/sign', Guestbook)
])
Replace
helloworld/main.py
with this file, then reload
http://localhost:8080/
in your browser. Post a few messages to verify that messages get stored and displayed correctly.
Now, let's look at what's going on with this new code.
Storing the Submitted Greetings
For the guestbook application, we want to store greetings posted by users. Each greeting includes the author's name, the message content, and the date and time the message was posted, so we can display messages in chronological order.
To use
NDB
, import the
google.appengine.ext.ndb
module:
from google.appengine.ext import ndb
class Greeting(ndb.Model):
"""Models an individual guestbook entry with author, content, and date."""
author = ndb.UserProperty()
content = ndb.StringProperty()
date = ndb.DateTimeProperty(auto_now_add=True)
This code defines an NDB
Greeting
model with three properties:
author
, whose value is a
User
object;
content
, whose value is a string; and
date
, whose value is a
datetime.datetime
.
Some property constructors take parameters to further configure their behavior. Giving the
ndb.DateTimeProperty
constructor an
auto_now_add=True
parameter configures the model to automatically stamp new objects with the date and time the object is created, if the application doesn't otherwise provide a value. For a complete list of property types and their options, see the
NDB Properties
page.
Now that we have a data model for greetings, the application can use the model to create new
Greeting
objects and put them into the Datastore. The following new version of the
Guestbook
handler creates new greetings and saves them to the Datastore:
class Guestbook(webapp2.RequestHandler):
def post(self):
# Set the parent key on each greeting to ensure that each
# guestbook's greetings are in the same entity group.
guestbook_name = self.request.get('guestbook_name')
# There is no need to actually create the parent Book entity; we can
# set it to be the parent of another entity without explicitly creating it
greeting = Greeting(parent=ndb.Key("Book", guestbook_name or "*notitle*"),
content = self.request.get('content'))
if users.get_current_user():
greeting.author = users.get_current_user()
greeting.put()
self.redirect('/?' + urllib.urlencode({'guestbook_name': guestbook_name}))
This new
Guestbook
handler creates a new
Greeting
object, then sets its author and content properties with the data posted by the user. The parent of the new object is a Datastore entity of kind
Book
, with ID generated from the guestbook name. (There is no need to actually create the parent
Book
entity; we can set it to be the parent of another entity without explicitly creating it). This parent
Book
entity is being used as a placeholder for transaction and consistency purposes. See the
Transactions
page in the documentation for more information. Objects that share a common
ancestor
form an
entity group,
so setting the same parent for each
Greeting
object assigns all greetings to the same entity group. We do this because querying in the High Replication Datastore is strongly consistent only within entity groups. This ensures that a user will always see a greeting immediately after it was written. We'll discuss consistency a bit further below.
The example code doesn't set the
date
property, so
date
is automatically set to the time the entity is saved, as we configured the model to do.
Finally,
greeting.put()
saves our new object to the Datastore. If we instead had been accessing an existing object,
put()
would have updated the existing object. Here, since we created this object with the model constructor,
put()
adds the new object to the Datastore.
Review and Summary
In this lesson, we've shown how to use the App Engine Datastore's NDB interface to persist
Greeting
objects to the Datastore.
In the next lesson , we'll show how to query for Greetings, and introduce Datastore indexes .