Class: Google::Cloud::Bigquery::Data

Inherits:
Array
  • Object
show all
Defined in:
lib/google/cloud/bigquery/data.rb

Overview

Data

Represents a page of results (rows) as an array of hashes. Because Data delegates to Array, methods such as Array#count represent the number of rows in the page. In addition, methods of this class include result set metadata such as total and provide access to the schema of the query or table. See Project#query, Google::Cloud::Bigquery::Dataset#query and Table#data.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

sql = "SELECT word FROM `bigquery-public-data.samples.shakespeare`"
job = bigquery.query_job sql

job.wait_until_done!
data = job.data

data.count # 100000
data.total # 164656
data.each do |row|
  puts row[:word]
end
data = data.next if data.next?

Instance Method Summary collapse

Instance Method Details

#all(request_limit: nil) {|row| ... } ⇒ Enumerator

Retrieves all rows by repeatedly loading #next until #next? returns false. Calls the given block once for each row, which is passed as the parameter.

An enumerator is returned if no block is given.

This method may make several API calls until all rows are retrieved. Be sure to use as narrow a search criteria as possible. Please use with caution.

Examples:

Iterating each rows by passing a block:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.data.all do |row|
  puts row[:word]
end

Using the enumerator by not passing a block:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

words = table.data.all.map do |row|
  row[:word]
end

Limit the number of API calls made:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.data.all(request_limit: 10) do |row|
  puts row[:word]
end

Parameters:

  • request_limit (Integer) —

    The upper limit of API requests to make to load all data. Default is no limit.

Yields:

  • (row) —

    The block for accessing each row of data.

Yield Parameters:

  • row (Hash) —

    The row object.

Returns:

  • (Enumerator) —

    An enumerator providing access to all of the data.



311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
 
# File 'lib/google/cloud/bigquery/data.rb', line 311

def all request_limit: nil
  request_limit = request_limit.to_i if request_limit
  unless block_given?
    return enum_for(:all, request_limit: request_limit)
  end
  results = self
  loop do
    results.each { |r| yield r }
    if request_limit
      request_limit -= 1
      break if request_limit < 0
    end
    break unless results.next?
    results = results.next
  end
end

#etag ⇒ String

An ETag hash for the page of results represented by the data instance.

Returns:

  • (String) —

    The ETag hash.



84
85
86
 
# File 'lib/google/cloud/bigquery/data.rb', line 84

def etag
  @gapi_json[:etag]
end

#fields ⇒ Array<Schema::Field>

The fields of the data, obtained from the schema of the table from which the data was read.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

data.fields.each do |field|
  puts field.name
end

Returns:



171
172
173
 
# File 'lib/google/cloud/bigquery/data.rb', line 171

def fields
  schema.fields
end

#headers ⇒ Array<Symbol>

The names of the columns in the data, obtained from the schema of the table from which the data was read.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

data.headers.each do |header|
  puts header
end

Returns:

  • (Array<Symbol>) —

    An array of column names.



194
195
196
 
# File 'lib/google/cloud/bigquery/data.rb', line 194

def headers
  schema.headers
end

#kind ⇒ String

The resource type of the API response.

Returns:

  • (String) —

    The resource type.



75
76
77
 
# File 'lib/google/cloud/bigquery/data.rb', line 75

def kind
  @gapi_json[:kind]
end

#next ⇒ Data

Retrieves the next page of data.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

sql = "SELECT word FROM `bigquery-public-data.samples.shakespeare`"
job = bigquery.query_job sql

job.wait_until_done!
data = job.data

data.count # 100000
data.total # 164656
data.each do |row|
  puts row[:word]
end
data = data.next if data.next?

Returns:

  • (Data) —

    A new instance providing the next page of data.



248
249
250
251
252
253
254
255
256
 
# File 'lib/google/cloud/bigquery/data.rb', line 248

def next
  return nil unless next?
  ensure_service!
  data_json = service.list_tabledata \
    @table_gapi.table_reference.dataset_id,
    @table_gapi.table_reference.table_id,
    token: token
  self.class.from_gapi_json data_json, @table_gapi, @service
end

#next? ⇒ Boolean

Whether there is a next page of data.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

sql = "SELECT word FROM `bigquery-public-data.samples.shakespeare`"
job = bigquery.query_job sql

job.wait_until_done!
data = job.data

data.count # 100000
data.total # 164656
data.each do |row|
  puts row[:word]
end
data = data.next if data.next?

Returns:

  • (Boolean) —

    true when there is a next page, false otherwise.



221
222
223
 
# File 'lib/google/cloud/bigquery/data.rb', line 221

def next?
  !token.nil?
end

#schema ⇒ Schema

The schema of the table from which the data was read.

The returned object is frozen and changes are not allowed. Use Table#schema to update the schema.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

schema = data.schema
field = schema.field "name"
field.required? #=> true

Returns:

  • (Schema) —

    A schema object.



148
149
150
 
# File 'lib/google/cloud/bigquery/data.rb', line 148

def schema
  Schema.from_gapi(@table_gapi.schema).freeze
end

#token ⇒ String

A token used for paging results. Used by the data instance to retrieve subsequent pages. See #next.

Returns:

  • (String) —

    The pagination token.



94
95
96
 
# File 'lib/google/cloud/bigquery/data.rb', line 94

def token
  @gapi_json[:pageToken]
end

#total ⇒ Integer

The total number of rows in the complete table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

sql = "SELECT word FROM `bigquery-public-data.samples.shakespeare`"
job = bigquery.query_job sql

job.wait_until_done!
data = job.data

data.count # 100000
data.total # 164656
data.each do |row|
  puts row[:word]
end
data = data.next if data.next?

Returns:

  • (Integer) —

    The number of rows.



121
122
123
124
125
 
# File 'lib/google/cloud/bigquery/data.rb', line 121

def total
  Integer @gapi_json[:totalRows]
rescue StandardError
  nil
end