Please note that the contents of this offline web site may be out of date. To access the most recent documentation visit the online version .
Note that links that point to online resources are green in color and will open in a new window.
We would love it if you could give us feedback about this material by filling this form (You have to be online to fill it)



Getting Information about Your Search Indexes

Learning objectives

Prerequisites

Related

Amy Unruh, Oct 2012
Google Developer Relations

Introduction

Once you’ve created some documents and added them to some index(es), you’ll often find it useful to inspect the structure and contents of the indexes. This lesson teaches you how.

Index Schemas

Each index has an associated schema that is used to interpret queries on that index. The schema is created dynamically from the field names and types present in all of the documents in the index; so adding documents to an index can cause updates to the index’s schema as well.

It can be useful to view an index’s schema, since it allows you to see the structure of the data in that index. A simple schema for an index might look like the following:


{'comment': ['TEXT'], 'date': ['DATE'], 'author': ['TEXT'], 'count': ['NUMBER']}

This shows that the documents in the index use the four fields comment , date , author , and count , with the indicated field types. (A given individual document need not have used all of these fields.) The schema takes the form of a dictionary in which each key is the name of a document field that has been added to the index, and the corresponding value is a list of the field types used with that field name. If you use the same field name with different field types (in different documents, for instance), the schema will list more than one field type for that name, like this:

{'field1': ['TEXT', 'NUMBER']}

As a more complex example, here is the schema for the product document index in the example application, once the sample data has been added to it. (This output shows the raw field names, not the class variables we’ve been using in our code examples).


{u'category': ['ATOM'], u'publisher': ['TEXT'], u'isbn': ['TEXT'],
 u'description': ['TEXT'], u'name': ['TEXT'], u'author': ['TEXT'],
 u'price': ['NUMBER'], u'title': ['TEXT'], u'pid': ['TEXT'], u'modified': ['DATE'],
 u'tv_type': ['TEXT'], u'ar': ['NUMBER'],
 u'brand': ['TEXT'], u'pages': ['NUMBER'], u'size': ['NUMBER']}

In the sample app, this index is called productsearch1 by default. The schema reflects both categories of product documents used by the application ( books and hd_televisions ), showing not only the core product fields shared by both categories, but also those specific to one category or the other, such as author (used only by ‘books’), tv_type (used only by hd_televisions ), and so on.

Viewing Indexes and Documents

You can view information about your application’s indexes, and the documents they contain, by clicking the application’s name in the App Engine Administration Console :

Viewing indexes in the Administration Console

Figure 1 : Viewing indexes in the Administration Console.

Click the Text Search link in the left sidebar, under Data, to see a list of the application’s defined indexes. Clicking an index name displays the documents that index contains. You’ll see all the defined schema fields for the index; for each document with a field of that name, you’ll see the field’s value. You can also issue queries on the index data directly from the Administration Console, using the Search API’s query language .

Accessing Index Schemas Programatically

It’s also possible to access all of your application’s indexes programmatically, along with the schemas for each:


from google.appengine.api import search
import logging

for index in search.list_indexes(fetch_schema=True):
    logging.info("index %s", index.name)
    logging.info("schema: %s", index.schema)

The listed schema information will look like the dictionaries shown above.

There is currently no way to delete a schema or remove fields from it.

Summary and Review

In this lesson, we’ve looked at ways to get information about an application’s indexes and their schemas.

Try this yourself with the example application: Examine the application’s indexes in the Administration Console, then try adding the code of the previous section (which lists your indexes and schemas) to the get() method of the IndexHandler class in handlers.py :


class IndexHandler(BaseHandler):
  """Displays the 'home' page."""

  def get(self):
    cat_info = models.Category.getCategoryInfo()
    sort_info = docs.Product.getSortMenu()
    # add this code temporarily to look at your indexes
    for index in search.list_indexes(fetch_schema=True):
      logging.info("index %s", index.name)
      logging.info("schema: %s", index.schema)
    template_values = {
        'cat_info': cat_info,
        'sort_info': sort_info,
        }
    self.render_template('index.html', template_values)

(You should only add this code temporarily, since it adds expense to the request.) When you redeploy the application and look at its logs when you load the index page, you’ll see your indexes and schemas listed out for you.

Now that we’ve worked though the basics of indexing and querying content, you already have the tools to add full-text search to your App Engine applications. However, the Search API supports much more sophisticated querying than we’ve shown so far. In the follow-on class, A Deeper Look at the Python Search API , we’ll delve deeper into search query construction and learn how to control and sort the results that are returned.

Authentication required

You need to be signed in with Google+ to do that.

Signing you in...

Google Developers needs your permission to do that.