Module: Google::Cloud::Datastore

Defined in:
lib/google/cloud/datastore.rb,
lib/google/cloud/datastore/key.rb,
lib/google/cloud/datastore/query.rb,
lib/google/cloud/datastore/commit.rb,
lib/google/cloud/datastore/cursor.rb,
lib/google/cloud/datastore/entity.rb,
lib/google/cloud/datastore/errors.rb,
lib/google/cloud/datastore/dataset.rb,
lib/google/cloud/datastore/service.rb,
lib/google/cloud/datastore/version.rb,
lib/google/cloud/datastore/gql_query.rb,
lib/google/cloud/datastore/properties.rb,
lib/google/cloud/datastore/credentials.rb,
lib/google/cloud/datastore/transaction.rb,
lib/google/cloud/datastore/v1/datastore_client.rb,
lib/google/cloud/datastore/dataset/query_results.rb,
lib/google/cloud/datastore/dataset/lookup_results.rb

Overview

Google Cloud Datastore

Google Cloud Datastore is a fully managed, schemaless database for storing non-relational data. You should feel at home if you are familiar with relational databases, but there are some key differences to be aware of to make the most of using Datastore.

The goal of google-cloud is to provide a API that is comfortable to Rubyists. Authentication is handled by #datastore. You can provide the project and credential information to connect to the Datastore service, or if you are running on Google Compute Engine this configuration is taken care of for you.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new(
  project: "my-todo-project",
  keyfile: "/path/to/keyfile.json"
)

task = datastore.find "Task", "sampleTask"
task["priority"] = 5
datastore.save task

You can learn more about various options for connection on the Authentication Guide.

To learn more about Datastore, read the Google Cloud Datastore Concepts Overview .

Retrieving records

Records, called "entities" in Datastore, are retrieved by using a key. The key is more than a numeric identifier, it is a complex data structure that can be used to model relationships. The simplest key has a string kind value, and either a numeric id value, or a string name value. A single record can be retrieved by calling Dataset#find and passing the parts of the key:

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task = datastore.find "Task", "sampleTask"

Optionally, Dataset#find can be given a key object:

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task_key = datastore.key "Task", 123456
task = datastore.find task_key

See Dataset#find

Querying records

Multiple records can be found that match criteria. (See Query#where)

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

query = datastore.query("Task").
  where("done", "=", false)

tasks = datastore.run query

Records can also be ordered. (See Query#order)

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

query = datastore.query("Task").
  order("created")

tasks = datastore.run query

The number of records returned can be specified. (See Query#limit)

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

query = datastore.query("Task").
  limit(5)

tasks = datastore.run query

Records' key structures can also be queried. (See Query#ancestor)

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task_list_key = datastore.key "TaskList", "default"

query = datastore.query("Task").
  ancestor(task_list_key)

tasks = datastore.run query

See Query and Dataset#run

Paginating records

All records may not return at once, but multiple calls can be made to Datastore to return them all.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

query = datastore.query("Task")
tasks = datastore.run query
tasks.all do |task|
  puts t["description"]
end

See Dataset::LookupResults and Dataset::QueryResults

Creating records

New entities can be created and persisted buy calling Dataset#save. The entity must have a key to be saved. If the key is incomplete then it will be completed when saved.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task = datastore.entity "Task" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 4
  t["description"] = "Learn Cloud Datastore"
end
task.key.id #=> nil
datastore.save task
task.key.id #=> 123456

Multiple new entities may be created in a batch.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task1 = datastore.entity "Task" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 4
  t["description"] = "Learn Cloud Datastore"
end

task2 = datastore.entity "Task" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 5
  t["description"] = "Integrate Cloud Datastore"
end

tasks = datastore.save(task1, task2)
task_key1 = tasks[0].key
task_key2 = tasks[1].key

Entities in Datastore form a hierarchically structured space similar to the directory structure of a file system. When you create an entity, you can optionally designate another entity as its parent; the new entity is a child of the parent entity.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task_key = datastore.key "Task", "sampleTask"
task_key.parent = datastore.key "TaskList", "default"

task = datastore.entity task_key do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 5
  t["description"] = "Integrate Cloud Datastore"
end

Setting properties

Entities hold properties. A property has a name that is a string or symbol, and a value that is an object. Most value objects are supported, including String, Integer, Date, Time, and even other entity or key objects. Changes to the entity's properties are persisted by calling Dataset#save.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task = datastore.find "Task", "sampleTask"
# Read the priority property
task["priority"] #=> 4
# Write the priority property
task["priority"] = 5
# Persist the changes
datastore.save task

Array properties can be used to store more than one value.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task = datastore.entity "Task", "sampleTask" do |t|
  t["tags"] = ["fun", "programming"]
  t["collaborators"] = ["alice", "bob"]
end

Deleting records

Entities can be removed from Datastore by calling Dataset#delete and passing the entity object or the entity's key object.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task = datastore.find "Task", "sampleTask"
datastore.delete task

Multiple entities may be deleted in a batch.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task_key1 = datastore.key "Task", "sampleTask1"
task_key2 = datastore.key "Task", "sampleTask2"
datastore.delete task_key1, task_key2

Transactions

Complex logic can be wrapped in a Transaction. All queries and updates within the Dataset#transaction block are run within the transaction scope, and will be automatically committed when the block completes.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task_key = datastore.key "Task", "sampleTask"

datastore.transaction do |tx|
  if tx.find(task_key).nil?
    task = datastore.entity task_key do |t|
      t["type"] = "Personal"
      t["done"] = false
      t["priority"] = 4
      t["description"] = "Learn Cloud Datastore"
    end
    tx.save task
  end
end

Alternatively, if no block is given the transaction object is returned allowing you to commit or rollback manually.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task_key = datastore.key "Task", "sampleTask"

tx = datastore.transaction
begin
  if tx.find(task_key).nil?
    task = datastore.entity task_key do |t|
      t["type"] = "Personal"
      t["done"] = false
      t["priority"] = 4
      t["description"] = "Learn Cloud Datastore"
    end
    tx.save task
  end
  tx.commit
rescue
  tx.rollback
end

See Transaction and Dataset#transaction

Querying metadata

Datastore provides programmatic access to some of its metadata to support meta-programming, implementing backend administrative functions, simplify consistent caching, and similar purposes. The metadata available includes information about the entity groups, namespaces, entity kinds, and properties your application uses, as well as the property representations for each property.

The special entity kind __namespace__ can be used to find all the namespaces used in your application entities.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

query = datastore.query("__namespace__").
  select("__key__").
  where("__key__", ">=", datastore.key("__namespace__", "g")).
  where("__key__", "<", datastore.key("__namespace__", "h"))

namespaces = datastore.run(query).map do |entity|
  entity.key.name
end

The special entity kind __kind__ can be used to return all the kinds used in your application.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

query = datastore.query("__kind__").
  select("__key__")

kinds = datastore.run(query).map do |entity|
  entity.key.name
end

Property queries return entities of kind __property__ denoting the indexed properties associated with an entity kind. (Unindexed properties are not included.)

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

query = datastore.query("__property__").
  select("__key__")

entities = datastore.run(query)
properties_by_kind = entities.each_with_object({}) do |entity, memo|
  kind = entity.key.parent.name
  prop = entity.key.name
  memo[kind] ||= []
  memo[kind] << prop
end

Property queries support ancestor filtering on a __kind__ or __property__ key, to limit the query results to a single kind or property. The property_representation property in the entity representing property p of kind k is an array containing all representations of p's value in any entity of kind k.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

ancestor_key = datastore.key "__kind__", "Task"
query = datastore.query("__property__").
  ancestor(ancestor_key)

entities = datastore.run(query)
representations = entities.each_with_object({}) do |entity, memo|
  property_name = entity.key.name
  property_types = entity["property_representation"]
  memo[property_name] = property_types
end

Property queries can also be filtered with a range over the pseudo-property __key__, where the keys denote either __kind__ or __property__ entities.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

start_key = datastore.key "__property__", "priority"
start_key.parent = datastore.key "__kind__", "Task"
query = datastore.query("__property__").
  select("__key__").
  where("__key__", ">=", start_key)

entities = datastore.run(query)
properties_by_kind = entities.each_with_object({}) do |entity, memo|
  kind = entity.key.parent.name
  prop = entity.key.name
  memo[kind] ||= []
  memo[kind] << prop
end

Configuring timeout

You can configure the request timeout value in seconds.

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new timeout: 120

The Cloud Datastore Emulator

As of this release, the Cloud Datastore emulator that is part of the gcloud SDK is no longer compatible with google-cloud. This is because the gcloud SDK's Cloud Datastore emulator does not yet support gRPC as a transport layer.

A gRPC-compatible emulator is available until the gcloud SDK Cloud Datastore emulator supports gRPC. To use it you must download the gRPC emulator and use the cloud_datastore_emulator script.

When you run the Cloud Datastore emulator you will see a message similar to the following printed:

If you are using a library that supports the DATASTORE_EMULATOR_HOST
environment variable, run:

export DATASTORE_EMULATOR_HOST=localhost:8978

Now you can connect to the emulator using the DATASTORE_EMULATOR_HOST environment variable:

require "google/cloud/datastore"

# Make Datastore use the emulator
ENV["DATASTORE_EMULATOR_HOST"] = "localhost:8978"

datastore = Google::Cloud::Datastore.new project: "emulator-project-id"

task = datastore.entity "Task", "emulatorTask" do |t|
  t["type"] = "Testing"
  t["done"] = false
  t["priority"] = 5
  t["description"] = "Use Datastore Emulator"
end

datastore.save task

Defined Under Namespace

Modules: V1 Classes: Commit, Cursor, Dataset, Entity, GqlQuery, Key, KeyError, Properties, PropertyError, Query, Transaction, TransactionError

Constant Summary collapse

VERSION = 
"0.23.0"

Class Method Summary collapse

Class Method Details

.new(project: nil, keyfile: nil, scope: nil, timeout: nil, client_config: nil) ⇒ Google::Cloud::Datastore::Dataset

Creates a new object for connecting to the Datastore service. Each call creates a new connection.

For more information on connecting to Google Cloud see the Authentication Guide.

Examples:

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new(
  project: "my-todo-project",
  keyfile: "/path/to/keyfile.json"
)

task = datastore.entity "Task", "sampleTask" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 4
  t["description"] = "Learn Cloud Datastore"
end

datastore.save task

Parameters:

  • project (String) —

    Dataset identifier for the Datastore you are connecting to.

  • keyfile (String, Hash) —

    Keyfile downloaded from Google Cloud. If file path the file must be readable.

  • scope (String, Array<String>) —

    The OAuth 2.0 scopes controlling the set of resources and operations that the connection can access. See Using OAuth 2.0 to Access Google APIs.

    The default scope is:

    • https://www.googleapis.com/auth/datastore
  • timeout (Integer) —

    Default timeout to use in requests. Optional.

  • client_config (Hash) —

    A hash of values to override the default behavior of the API client. See Google::Gax::CallSettings. Optional.

Returns:



568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
 
# File 'lib/google/cloud/datastore.rb', line 568

def self.new project: nil, keyfile: nil, scope: nil, timeout: nil,
             client_config: nil
  project ||= Google::Cloud::Datastore::Dataset.default_project
  project = project.to_s # Always cast to a string
  fail ArgumentError, "project is missing" if project.empty?

  if ENV["DATASTORE_EMULATOR_HOST"]
    return Google::Cloud::Datastore::Dataset.new(
      Google::Cloud::Datastore::Service.new(
        project, :this_channel_is_insecure,
        host: ENV["DATASTORE_EMULATOR_HOST"],
        client_config: client_config))
  end
  credentials = credentials_with_scope keyfile, scope
  Google::Cloud::Datastore::Dataset.new(
    Google::Cloud::Datastore::Service.new(
      project, credentials, timeout: timeout,
                            client_config: client_config))
end