Class: Google::Cloud::Bigquery::QueryJob

Inherits:
Job
  • Object
show all
Defined in:
lib/google/cloud/bigquery/query_job.rb

Overview

QueryJob

A Job subclass representing a query operation that may be performed on a Table. A QueryJob instance is created when you call Project#query_job, Dataset#query_job.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

job = bigquery.query_job "SELECT COUNT(word) as count FROM " \
                         "publicdata.samples.shakespeare"

job.wait_until_done!

if job.failed?
  puts job.error
else
  puts job.data.first
end

See Also:

Defined Under Namespace

Classes: Stage, Step

Instance Method Summary collapse

Methods inherited from Job

#cancel, #configuration, #created_at, #done?, #ended_at, #error, #errors, #failed?, #job_id, #labels, #pending?, #project_id, #reload!, #rerun!, #running?, #started_at, #state, #statistics, #status, #user_email

Instance Method Details

#batch? ⇒ Boolean

Checks if the priority for the query is BATCH.

Returns:

  • (Boolean) —

    true when the priority is BATCH, false otherwise.



56
57
58
59
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 56

def batch?
  val = @gapi.configuration.query.priority
  val == "BATCH"
end

#bytes_processed ⇒ Integer

The number of bytes processed by the query.

Returns:

  • (Integer) —

    Total bytes processed for the job.



157
158
159
160
161
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 157

def bytes_processed
  Integer @gapi.statistics.query.total_bytes_processed
rescue
  nil
end

#cache? ⇒ Boolean

Checks if the query job looks for an existing result in the query cache. For more information, see Query Caching.

Returns:

  • (Boolean) —

    true when the query cache will be used, false otherwise.



94
95
96
97
98
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 94

def cache?
  val = @gapi.configuration.query.use_query_cache
  return false if val.nil?
  val
end

#cache_hit? ⇒ Boolean

Checks if the query results are from the query cache.

Returns:

  • (Boolean) —

    true when the job statistics indicate a cache hit, false otherwise.



148
149
150
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 148

def cache_hit?
  @gapi.statistics.query.cache_hit
end

#data(token: nil, max: nil, start: nil) ⇒ Google::Cloud::Bigquery::Data Also known as: query_results

Retrieves the query results for the job.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

sql = "SELECT word FROM publicdata.samples.shakespeare"
job = bigquery.query_job sql

job.wait_until_done!
data = job.data
data.each do |row|
  puts row[:word]
end
data = data.next if data.next?

Parameters:

  • token (String) —

    Page token, returned by a previous call, identifying the result set.

  • max (Integer) —

    Maximum number of results to return.

  • start (Integer) —

    Zero-based index of the starting row to read.

Returns:



302
303
304
305
306
307
308
309
310
311
312
313
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 302

def data token: nil, max: nil, start: nil
  return nil unless done?

  ensure_schema!

  options = { token: token, max: max, start: start }
  data_hash = service.list_tabledata \
    destination_table_dataset_id,
    destination_table_table_id,
    options
  Data.from_gapi_json data_hash, destination_table_gapi, service
end

#destination ⇒ Table

The table in which the query results are stored.

Returns:

  • (Table) —

    A table instance.



200
201
202
203
204
205
206
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 200

def destination
  table = @gapi.configuration.query.destination_table
  return nil unless table
  retrieve_table table.project_id,
                 table.dataset_id,
                 table.table_id
end

#flatten? ⇒ Boolean

Checks if the query job flattens nested and repeated fields in the query results. The default is true. If the value is false,

large_results? should return true.

Returns:

  • (Boolean) —

    true when the job flattens results, false otherwise.



108
109
110
111
112
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 108

def flatten?
  val = @gapi.configuration.query.flatten_results
  return true if val.nil?
  val
end

#interactive? ⇒ Boolean

Checks if the priority for the query is INTERACTIVE.

Returns:

  • (Boolean) —

    true when the priority is INTERACTIVE, false otherwise.



67
68
69
70
71
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 67

def interactive?
  val = @gapi.configuration.query.priority
  return true if val.nil?
  val == "INTERACTIVE"
end

#large_results? ⇒ Boolean

Checks if the the query job allows arbitrarily large results at a slight cost to performance.

Returns:

  • (Boolean) —

    true when large results are allowed, false otherwise.



80
81
82
83
84
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 80

def large_results?
  val = @gapi.configuration.query.allow_large_results
  return false if val.nil?
  val
end

#legacy_sql? ⇒ Boolean

Checks if the query job is using legacy sql.

Returns:

  • (Boolean) —

    true when legacy sql is used, false otherwise.



213
214
215
216
217
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 213

def legacy_sql?
  val = @gapi.configuration.query.use_legacy_sql
  return true if val.nil?
  val
end

#maximum_billing_tier ⇒ Integer?

Limits the billing tier for this job. Queries that have resource usage beyond this tier will fail (without incurring a charge). If unspecified, this will be set to your project default. For more information, see High-Compute queries.

Returns:

  • (Integer, nil) —

    The tier number, or nil for the project default.



124
125
126
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 124

def maximum_billing_tier
  @gapi.configuration.query.maximum_billing_tier
end

#maximum_bytes_billed ⇒ Integer?

Limits the bytes billed for this job. Queries that will have bytes billed beyond this limit will fail (without incurring a charge). If nil, this will be set to your project default.

Returns:

  • (Integer, nil) —

    The number of bytes, or nil for the project default.



136
137
138
139
140
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 136

def maximum_bytes_billed
  Integer @gapi.configuration.query.maximum_bytes_billed
rescue
  nil
end

#query_plan ⇒ Array<Google::Cloud::Bigquery::QueryJob::Stage>

Describes the execution plan for the query.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

sql = "SELECT word FROM publicdata.samples.shakespeare"
job = bigquery.query_job sql

job.wait_until_done!

stages = job.query_plan
stages.each do |stage|
  puts stage.name
  stage.steps.each do |step|
    puts step.kind
    puts step.substeps.inspect
  end
end

Returns:



188
189
190
191
192
193
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 188

def query_plan
  return nil unless @gapi.statistics.query.query_plan
  Array(@gapi.statistics.query.query_plan).map do |stage|
    Stage.from_gapi stage
  end
end

#standard_sql? ⇒ Boolean

Checks if the query job is using standard sql.

Returns:

  • (Boolean) —

    true when standard sql is used, false otherwise.



224
225
226
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 224

def standard_sql?
  !legacy_sql?
end

#udfs ⇒ Array<String>

The user-defined function resources used in the query. May be either a code resource to load from a Google Cloud Storage URI (gs://bucket/path), or an inline resource that contains code for a user-defined function (UDF). Providing an inline code resource is equivalent to providing a URI for a file containing the same code. See User-Defined Functions.

Returns:

  • (Array<String>) —

    An array containing Google Cloud Storage URIs and/or inline source code.



239
240
241
242
243
244
245
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 239

def udfs
  udfs_gapi = @gapi.configuration.query.user_defined_function_resources
  return nil unless udfs_gapi
  Array(udfs_gapi).map do |udf|
    udf.inline_code || udf.resource_uri
  end
end

#wait_until_done! ⇒ Object

Refreshes the job until the job is DONE. The delay between refreshes will incrementally increase.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

sql = "SELECT word FROM publicdata.samples.shakespeare"
job = bigquery.query_job sql

job.wait_until_done!
job.done? #=> true


262
263
264
265
266
267
268
269
270
271
272
273
274
 
# File 'lib/google/cloud/bigquery/query_job.rb', line 262

def wait_until_done!
  return if done?

  ensure_service!
  loop do
    query_results_gapi = service.job_query_results job_id, max: 0
    if query_results_gapi.job_complete
      @destination_schema_gapi = query_results_gapi.schema
      break
    end
  end
  reload!
end