Sign in

1. Products
2. Google Cloud Platform
3. Google App Engine
4. Python

Python

Feedback on this document

NDB Properties

An NDB entity model can define properties . Entity properties are a little like data members of Python classes, a structured way to hold data; they are a little like fields in a database schema.

1. Introduction
2. Types
3. Property Options
4. Repeated Properties
5. Date and Time Properties
6. Structured Properties
7. Computed Properties
8. Google Protocol RPC Message Properties

Introduction

A typical application defines a data model by defining a class that inherits from Model with some property class attributes. For example,

from google.appengine.ext import ndb

class Account(ndb.Model):
  username = ndb.StringProperty()
  userid = ndb.IntegerProperty()
  email = ndb.StringProperty()

Here, username , userid , and email are properties of Account .

There are several other property types. Some are handy for representing dates and times and have convenient auto-update features.

An application can adjust a property's behavior by specifying options on the property; these can ease validation, set defaults, or change query indexing.

A model can have more complex properties. Repeated properties are list-like. Structured properties are object-like. Read-only computed properties are defined via functions; this makes it easy to define a property in terms of one or more other properties. Expando models can define properties dynamically.

Types

NDB supports the following property types:

Property type	Description
IntegerProperty	64-bit signed integer
FloatProperty	Double-precision floating-point number
BooleanProperty	Boolean
StringProperty	Unicode string; up to 500 characters, indexed
TextProperty	Unicode string; unlimited length, not indexed
BlobProperty	Uninterpreted byte string: if you set indexed=True , up to 500 characters, indexed; if indexed is False (the default), unlimited length, not indexed. Optional keyword argument: compressed .
DateTimeProperty	Date and time (see Date and Time Properties )
DateProperty	Date (see Date and Time Properties )
TimeProperty	Time (see Date and Time Properties )
GeoPtProperty	Geographical location. This is a ndb.GeoPt object. The object has attributes lat and lon , both floats. You can construct one with two floats like ndb.GeoPt(52.37, 4.88) or with a string ndb.GeoPt("52.37, 4.88") . (This is actually the same class as db.GeoPt )
KeyProperty	Datastore key Optional keyword argument: kind= kind , to require that keys assigned to this property always have the indicated kind. May be a string or a Model subclass.
BlobKeyProperty	Blobstore key Corresponds to BlobReferenceProperty in the old db API, but the property value is a BlobKey instead of a BlobInfo ; you can construct a BlobInfo from it using BlobInfo( blobkey )
UserProperty	User object.
StructuredProperty	Includes one kind of model inside another, by value (see Structured Properties )
LocalStructuredProperty	Like StructuredProperty , but on-disk representation is an opaque blob and is not indexed (see Structured Properties ). Optional keyword argument: compressed .
JsonProperty	Value is a Python object (such as a list or a dict or a string) that is serializable using Python's json module; the Datastore stores the JSON serialization as a blob. Unindexed by default. Optional keyword argument: compressed .
PickleProperty	Value is a Python object (such as a list or a dict or a string) that is serializable using Python's pickle protocol; the Datastore stores the pickle serialization as a blob. Unindexed by default. Optional keyword argument: compressed .
GenericProperty	Generic value Used mostly by the Expando class, but also usable explicitly. Its type may be any of int , long , float , bool , str , unicode , datetime , Key , BlobKey , GeoPt , User , None .
ComputedProperty	Value computed from other properties by a user-defined function. (See Computed Properties .)

Some of these properties have an optional keyword argument, compressed . If the property has compressed=True , then its data is compressed with gzip on disk. It takes up less space but needs CPU to encode/decode on write and read operations.

Both compression and decompression are "lazy"; a compressed property value will only be decompressed the first time you access it. If you read an entity containing a compressed property value and write it back without accessing the compressed property, it won't be decompressed and compressed at all. The in-context cache participates in this lazy scheme as well, but memcache always stores the compressed value for compressed properties.

Because of the extra CPU time needed for compression, it is usually best to use compressed properties only if the data would be too big to fit without it. Remember that gzip-based compression is typically not effective for images and other media data, since those formats already are compressed using a media-specific compression algorithm (e.g. JPEG for images).

Property Options

Most property types support some standard arguments. The first is an optional positional argument specifying the property's Datastore name. You can use this to give the property a different name in the Datastore than from the application's viewpoint. A common use for this is to reduce space in the Datastore, letting the Datastore use abbreviated property names while your code uses longer, more meaningful ones. For example,

class Employee(ndb.Model):
  full_name = ndb.StringProperty('n')
  retirement_age = ndb.IntegerProperty('r')

This is particularly useful for repeated properties for which you expect many values per entity.

In addition, most property types support the following keyword arguments:

Argument	Type	Default	Description
indexed	bool	Usually True	Include property in Datastore's indexes; if False , values cannot be queried but writes are faster. Not all property types support indexing; setting indexed to True fails for these. Unindexed properties cost fewer write ops than indexed properties.
repeated	bool	False	Property value is a Python list containing values of the underlying type (see Repeated Properties ).
required	bool	False	Property must have a value specified. Cannot be combined with repeated=True but can be combined with default=True .
default	Property's underlying type	None	Default value of property if none explicitly specified. Cannot be combined with repeated=True but can be combined with required=True .
choices	List of values of underlying type	None	Optional list of allowable values.
validator	Function	None	Optional function to validate and possibly coerce the value. Will be called with arguments ( prop , value )

Optional HTML label to use in web form frameworks like jinja2.

Repeated Properties

Any property with repeated=True becomes a repeated property . The property takes a list of values of the underlying type, rather than a single value. For example, the value of a property defined with IntegerProperty(repeated=True) is a list of integers.

The Datastore may see multiple values for such a property. A separate index record is created for each value. This affects query semantics; see Querying for Repeated Properties for an example.

This example uses a repeated property:

class Article(ndb.Model):
  title = ndb.StringProperty()
  stars = ndb.IntegerProperty()
  tags = ndb.StringProperty(repeated=True)

art = Article(title='Python versus Ruby',
              stars=3,
              tags=['python', 'ruby'])
art.put()

This creates a Datastore entity with the following contents:

name = 'Python versus Ruby'
stars = 3
tags = 'python' # repeated = True
tags = 'ruby' # repeated = True

When querying for the tags property, this entity will satisfy a query for either 'python' or 'ruby' .

When updating a repeated property, you can either assign it a new list or mutate the existing list in place. When you assign a new list, the types of the list items are validated immediately. Invalid item types (for example, assigning [1, 2] to art.tags above) raise an exception. When you mutate the list, the change is not validated immediately. Instead, the value will be validated when you write the entity to the Datastore.

The datastore preserves the order of the list items in a repeated property, so you can assign some meaning to their ordering.

Date and Time Properties

Three property types are available for storing date- and time-related values:

• DateProperty
• TimeProperty
• DateTimeProperty

These take values belonging to the corresponding classes ( date , time , datetime ) of the standard Python datetime module. The most general of the three is DateTimeProperty , which denotes both a calendar date and a time of day; the others are occasionally useful for special purposes requiring just a date (such as a date of birth) or just a time (such as a meeting time). For technical reasons, DateProperty and TimeProperty are subclasses of DateTimeProperty , but you shouldn't depend on this inheritance relationship (and note that it differs from the inheritance relationships between the underlying classes defined by the datetime module itself).

Note: App Engine clock times are always expressed in coordinated universal time (UTC). This becomes relevant if you use the current date or time ( datetime.datetime.now() ) as a value or convert between datetime objects and POSIX timestamps or time tuples. However, no explicit time zone information is stored in the Datastore, so if you are careful you can use these to represent local times in any timezone—if you use the current time or the conversions.

Each of these properties has two extra Boolean keyword options:

These options cannot be combined with repeated=True . Both of them default to False ; if both are set to True , auto_now takes precedence. It is possible to override the value for a property with auto_now_add=True , but not for one with auto_now=True . The automatic value is not generated until the entity is written; that is, these options don't provide dynamic defaults. (These details differ from the old db API.)

Note: When a transaction writing a property with auto_now_add=True fails and is later retried, it will reuse the same time value as on the original try rather than update it to the time of the retry. If the transaction fails permanently, the property's value will still be set in the in-memory copy of the entity.

Structured Properties

You can structure a model's properties. For example, you can define a model class Contact containing a list of addresses, each with internal structure. Structured properties (type StructuredProperty ) let you do this; for example:

class Address(ndb.Model):
  type = ndb.StringProperty() # E.g., 'home', 'work'
  street = ndb.StringProperty()
  city = ndb.StringProperty()

class Contact(ndb.Model):
  name = ndb.StringProperty()
  addresses = ndb.StructuredProperty(Address, repeated=True)

guido = Contact(name='Guido',
                addresses=[Address(type='home',
                                   city='Amsterdam'),
                           Address(type='work',
                                   street='Spear St',
                                   city='SF')])

guido.put()

This creates a single Datastore entity with the following properties:

name = 'Guido'
addresses.type = 'home'
addresses.type = 'work'
addresses.street = None
addresses.street = 'Spear St'
addresses.city = 'Amsterdam'
addresses.city = 'SF'

Reading back such an entity reconstructs the original Contact entity exactly. Although the Address instances are defined using the same syntax as for model classes, they are not full-fledged entities. They don't have their own keys in the Datastore. They cannot be retrieved independently of the Contact entity to which they belong. An application can, however, query for the values of their individual fields; see Filtering for Structured Property Values . Note that address.type , address.street , and address.city are viewed as parallel arrays from the Datastore's viewpoint, but the NDB library hides this aspect and constructs the corresponding list of Address instances.

You can specify the usual property options for structured properties (except indexed ). The Datastore name is the second positional argument in this case (the first being the model class used to define the substructure).

When you don't need to query for a substructure's internal properties, you can use a local structured property ( LocalStructuredProperty ) instead. If you replace StructuredProperty with LocalStructuredProperty in the example above, the behavior of the Python code is the same, but the Datastore sees just an opaque blob for each address. The guido entity created in the example would be stored as follows:

name = 'Guido'
address = <opaque blob for {'type': 'home', 'city': 'Amsterdam'}>
address = <opaque blob for {'type': 'work', 'city': 'SF',
                            'street': 'Spear St'}>

The entity will be read back correctly. Since properties of this type are always unindexed, you cannot query for address values.

Although a StructuredProperty can be repeated and a StructuredProperty can contain another StructuredProperty , beware: if one structured property contains another, only one of them can be repeated. A work-around is to use LocalStructuredProperty , which does not have this constraint (but does not allow queries on its property values).

Computed Properties

Computed properties ( ComputedProperty ) are read-only properties whose value is computed from other property values by an application-supplied function. The computed value is written to the Datastore so that it can be queried and displayed in the Datastore viewer, but the stored value is ignored when the entity is read back from the Datastore; rather, the value is recomputed by calling the function whenever the value is requested. For example:

class SomeEntity(ndb.Model):
  name = ndb.StringProperty()
  name_lower = ndb.ComputedProperty(lambda self: self.name.lower())

x = SomeEntity(name='Nick')

This stores an entity with the following property values:

name = 'Nick'
name_lower = 'nick'

If we change name to 'Nickie' and ask for the value of name_lower , it returns 'nickie':

x.name = 'Nick'
assert x.name_lower == 'nick'
x.name = 'Nickie'
assert x.name_lower == 'nickie'

Note: Use ComputedProperty if the application queries for the computed value. If you just want to use the derived version in Python code, define a regular method or use Python's @property built-in.

Google Protocol RPC Message Properties

The Google Protocol RPC library uses Message objects for structured data; they can represent RPC requests, responses, or other things. NDB provides an API for storing Google Protocol RPC Message objects as entity properties. Suppose you define a Message subclass:

from protorpc import messages

class Note(messages.Message):
  text = messages.StringField(1, required=True)
  when = messages.IntegerField(2)

You can store Note objects in the datastore as entity property values by using NDB's msgprop API.

from google.appengine.ext import ndb
from google.appengine.ext.ndb import msgprop

class NoteStore(ndb.Model):
  note = msgprop.MessageProperty(Note, indexed_fields=['when'])
  name = ndb.StringProperty()

my_note = Note(text='Excellent note', when=50)

ns = NoteStore(note=my_note, name='excellent')
key = ns.put()

new_notes = NoteStore.query(NoteStore.note.when >= 123).fetch()

If you want to query for field names, they must be indexed. You can specify a list of field names that will be indexed with the indexed_fields parameter to MessageProperty .

MessageProperty supports many, but not all, Property options . It supports:

• name
• repeated
• required
• default
• choices
• validator
• verbose_name

Message properties do not support the indexed property option; you can't index Message values. (You can index fields of a message as described above.)

Nested messages (using MessageField ) also work:

class NoteBook(messages.Message):
  notes = messages.MessageField(Note, 1, repeated=True)

class SignedStoreableNotebook(ndb.Model):
  author = ndb.StringProperty()
  nb = msgprop.MessageProperty(NoteBook,
                               indexed_fields=['notes.text', 'notes.when'])

MessageProperty has a special property option, protocol , which specifies how the message object is serialized to the datastore. The values are protocol names as used by protorpc.remote.Protocols class. Supported protocol names are protobuf and protojson ; the default is protobuf .

msgprop also defines EnumProperty , a property type which can be used to store a protorpc.messages.Enum value in an entity. Example:

class Color(messages.Enum):
  RED = 620
  GREEN = 495
  BLUE = 450

class Part(ndb.Model):
  name = ndb.StringProperty()
  color = msgprop.EnumProperty(Color, required=True)

p1 = Part(name='foo', color=Color.RED)
print p1.color  # prints "RED"

EnumProperty stores the value as an integer; in fact, EnumProperty is a subclass of IntegerProperty . This implies that you can rename your enum values without having to modify already-stored entities, but you cannot renumber them.

The EnumProperty supports the following property options :

• name
• indexed
• repeated
• required
• default
• choices
• validator
• verbose_name