Class: Google::Cloud::Bigquery::Data

Inherits:
Array
  • Object
show all
Defined in:
lib/google/cloud/bigquery/data.rb

Overview

Data

Represents Table Data as a list of name/value pairs (hashes.) Also contains metadata such as etag and total, and provides access to the schema of the table from which the data was read.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data
puts "#{data.count} of #{data.total}"
if data.next?
  next_data = data.next
end

Instance Method Summary collapse

Instance Method Details

#all(request_limit: nil) {|row| ... } ⇒ Enumerator

Retrieves all rows by repeatedly loading #next until #next? returns false. Calls the given block once for each row, which is passed as the parameter.

An enumerator is returned if no block is given.

This method may make several API calls until all rows are retrieved. Be sure to use as narrow a search criteria as possible. Please use with caution.

Examples:

Iterating each rows by passing a block:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.data.all do |row|
  puts row[:word]
end

Using the enumerator by not passing a block:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

words = table.data.all.map do |row|
  row[:word]
end

Limit the number of API calls made:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.data.all(request_limit: 10) do |row|
  puts row[:word]
end

Parameters:

  • request_limit (Integer) —

    The upper limit of API requests to make to load all data. Default is no limit.

Yields:

  • (row) —

    The block for accessing each row of data.

Yield Parameters:

  • row (Hash) —

    The row object.

Returns:

  • (Enumerator) —

    An enumerator providing access to all of the data.



286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
 
# File 'lib/google/cloud/bigquery/data.rb', line 286

def all request_limit: nil
  request_limit = request_limit.to_i if request_limit
  unless block_given?
    return enum_for(:all, request_limit: request_limit)
  end
  results = self
  loop do
    results.each { |r| yield r }
    if request_limit
      request_limit -= 1
      break if request_limit < 0
    end
    break unless results.next?
    results = results.next
  end
end

#etag ⇒ String

An ETag hash for the page of results represented by the data instance.

Returns:

  • (String) —

    The ETag hash.



77
78
79
 
# File 'lib/google/cloud/bigquery/data.rb', line 77

def etag
  @gapi_json[:etag]
end

#fields ⇒ Array<Schema::Field>

The fields of the data, obtained from the schema of the table from which the data was read.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

data.fields.each do |field|
  puts field.name
end

Returns:



159
160
161
 
# File 'lib/google/cloud/bigquery/data.rb', line 159

def fields
  schema.fields
end

#headers ⇒ Array<Symbol>

The names of the columns in the data, obtained from the schema of the table from which the data was read.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

data.headers.each do |header|
  puts header
end

Returns:

  • (Array<Symbol>) —

    An array of column names.



182
183
184
 
# File 'lib/google/cloud/bigquery/data.rb', line 182

def headers
  schema.headers
end

#kind ⇒ String

The resource type of the API response.

Returns:

  • (String) —

    The resource type.



68
69
70
 
# File 'lib/google/cloud/bigquery/data.rb', line 68

def kind
  @gapi_json[:kind]
end

#next ⇒ Data

Retrieves the next page of data.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data
if data.next?
  next_data = data.next
end

Returns:

  • (Data) —

    A new instance providing the next page of data.



224
225
226
227
228
229
230
231
232
 
# File 'lib/google/cloud/bigquery/data.rb', line 224

def next
  return nil unless next?
  ensure_service!
  data_json = service.list_tabledata \
    @table_gapi.table_reference.dataset_id,
    @table_gapi.table_reference.table_id,
    token: token
  self.class.from_gapi_json data_json, @table_gapi, @service
end

#next? ⇒ Boolean

Whether there is a next page of data.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data
if data.next?
  next_data = data.next
end

Returns:

  • (Boolean) —

    true when there is a next page, false otherwise.



203
204
205
 
# File 'lib/google/cloud/bigquery/data.rb', line 203

def next?
  !token.nil?
end

#schema ⇒ Schema

The schema of the table from which the data was read.

The returned object is frozen and changes are not allowed. Use Table#schema to update the schema.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

schema = data.schema
field = schema.field "name"
field.required? #=> true

Returns:

  • (Schema) —

    A schema object.



136
137
138
 
# File 'lib/google/cloud/bigquery/data.rb', line 136

def schema
  Schema.from_gapi(@table_gapi.schema).freeze
end

#token ⇒ String

A token used for paging results. Used by the data instance to retrieve subsequent pages. See #next.

Returns:

  • (String) —

    The pagination token.



87
88
89
 
# File 'lib/google/cloud/bigquery/data.rb', line 87

def token
  @gapi_json[:pageToken]
end

#total ⇒ Integer

The total number of rows in the complete table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data
puts "#{data.count} of #{data.total}"
if data.next?
  next_data = data.next
end

Returns:

  • (Integer) —

    The number of rows.



109
110
111
112
113
 
# File 'lib/google/cloud/bigquery/data.rb', line 109

def total
  Integer @gapi_json[:totalRows]
rescue
  nil
end