Class: Google::Cloud::Bigquery::Job

Inherits:
Object
  • Object
show all
Defined in:
lib/google/cloud/bigquery/job.rb,
lib/google/cloud/bigquery/job/list.rb

Overview

Job

Represents a generic Job that may be performed on a Table.

The subclasses of Job represent the specific BigQuery job types: CopyJob, ExtractJob, LoadJob, and QueryJob.

A job instance is created when you call Project#query_job, Dataset#query_job, Table#copy_job, Table#extract_job, Table#load_job.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

job = bigquery.query_job "SELECT COUNT(word) as count FROM " \
                         "`bigquery-public-data.samples.shakespeare`"

job.wait_until_done!

if job.failed?
  puts job.error
else
  puts job.data.first
end

See Also:

Direct Known Subclasses

CopyJob, ExtractJob, LoadJob, QueryJob

Defined Under Namespace

Classes: List

Attributes collapse

Instance Method Summary collapse

Instance Method Details

#cancel ⇒ Object

Cancels the job.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

query = "SELECT COUNT(word) as count FROM " \
        "`bigquery-public-data.samples.shakespeare`"

job = bigquery.query_job query

job.cancel


305
306
307
308
309
310
 
# File 'lib/google/cloud/bigquery/job.rb', line 305

def cancel
  ensure_service!
  resp = service.cancel_job job_id, location: location
  @gapi = resp.job
  true
end

#configuration ⇒ Object Also known as: config

The configuration for the job. Returns a hash.

See Also:



210
211
212
 
# File 'lib/google/cloud/bigquery/job.rb', line 210

def configuration
  JSON.parse @gapi.configuration.to_json
end

#created_at ⇒ Time?

The time when the job was created.

Returns:

  • (Time, nil) —

    The creation time from the job statistics.



174
175
176
177
178
 
# File 'lib/google/cloud/bigquery/job.rb', line 174

def created_at
  ::Time.at(Integer(@gapi.statistics.creation_time) / 1000.0)
rescue StandardError
  nil
end

#done? ⇒ Boolean

Checks if the job's state is DONE. When true, the job has stopped running. However, a DONE state does not mean that the job completed successfully. Use #failed? to detect if an error occurred or if the job was successful.

Returns:

  • (Boolean) —

    true when DONE, false otherwise.



154
155
156
157
 
# File 'lib/google/cloud/bigquery/job.rb', line 154

def done?
  return false if state.nil?
  "done".casecmp(state).zero?
end

#ended_at ⇒ Time?

The time when the job ended. This field is present when the job's state is DONE.

Returns:

  • (Time, nil) —

    The end time from the job statistics.



199
200
201
202
203
 
# File 'lib/google/cloud/bigquery/job.rb', line 199

def ended_at
  ::Time.at(Integer(@gapi.statistics.end_time) / 1000.0)
rescue StandardError
  nil
end

#error ⇒ Hash?

The last error for the job, if any errors have occurred. Returns a hash.

Returns:

  • (Hash, nil) —

    Returns a hash containing reason and message keys:

    { "reason"=>"notFound", "message"=>"Not found: Table bigquery-public-data:samples.BAD_ID" }

See Also:



253
254
255
 
# File 'lib/google/cloud/bigquery/job.rb', line 253

def error
  status["errorResult"]
end

#errors ⇒ Array<Hash>?

The errors for the job, if any errors have occurred. Returns an array of hash objects. See #error.

Returns:

  • (Array<Hash>, nil) —

    Returns an array of hashes containing reason and message keys:

    { "reason"=>"notFound", "message"=>"Not found: Table bigquery-public-data:samples.BAD_ID" }



269
270
271
 
# File 'lib/google/cloud/bigquery/job.rb', line 269

def errors
  Array status["errors"]
end

#failed? ⇒ Boolean

Checks if an error is present. Use #error to access the error object.

Returns:

  • (Boolean) —

    true when there is an error, false otherwise.



165
166
167
 
# File 'lib/google/cloud/bigquery/job.rb', line 165

def failed?
  !error.nil?
end

#job_id ⇒ String

The ID of the job.

Returns:

  • (String) —

    The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or dashes (-). The maximum length is 1,024 characters.



80
81
82
 
# File 'lib/google/cloud/bigquery/job.rb', line 80

def job_id
  @gapi.job_reference.job_id
end

#labels ⇒ Hash

A hash of user-provided labels associated with this job. Labels can be provided when the job is created, and used to organize and group jobs.

The returned hash is frozen and changes are not allowed. Use #labels= to replace the entire hash.

Returns:

  • (Hash) —

    The job labels.



284
285
286
287
288
 
# File 'lib/google/cloud/bigquery/job.rb', line 284

def labels
  m = @gapi.configuration.labels
  m = m.to_h if m.respond_to? :to_h
  m.dup.freeze
end

#location ⇒ String

The geographic location where the job runs.

Returns:

  • (String) —

    A geographic location, such as "US", "EU" or "asia-northeast1".



100
101
102
 
# File 'lib/google/cloud/bigquery/job.rb', line 100

def location
  @gapi.job_reference.location
end

#pending? ⇒ Boolean

Checks if the job's state is PENDING.

Returns:

  • (Boolean) —

    true when PENDING, false otherwise.



141
142
143
144
 
# File 'lib/google/cloud/bigquery/job.rb', line 141

def pending?
  return false if state.nil?
  "pending".casecmp(state).zero?
end

#project_id ⇒ String

The ID of the project containing the job.

Returns:

  • (String) —

    The project ID.



89
90
91
 
# File 'lib/google/cloud/bigquery/job.rb', line 89

def project_id
  @gapi.job_reference.project_id
end

#reload! ⇒ Object Also known as: refresh!

Reloads the job with current data from the BigQuery service.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

query = "SELECT COUNT(word) as count FROM " \
        "`bigquery-public-data.samples.shakespeare`"

job = bigquery.query_job query

job.done?
job.reload!
job.done? #=> true


351
352
353
354
355
 
# File 'lib/google/cloud/bigquery/job.rb', line 351

def reload!
  ensure_service!
  gapi = service.get_job job_id, location: location
  @gapi = gapi
end

#rerun! ⇒ Object

Created a new job with the current configuration.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

query = "SELECT COUNT(word) as count FROM " \
        "`bigquery-public-data.samples.shakespeare`"

job = bigquery.query_job query

job.wait_until_done!
job.rerun!


328
329
330
331
332
 
# File 'lib/google/cloud/bigquery/job.rb', line 328

def rerun!
  ensure_service!
  gapi = service.insert_job @gapi.configuration, location: location
  Job.from_gapi gapi, service
end

#running? ⇒ Boolean

Checks if the job's state is RUNNING.

Returns:

  • (Boolean) —

    true when RUNNING, false otherwise.



131
132
133
134
 
# File 'lib/google/cloud/bigquery/job.rb', line 131

def running?
  return false if state.nil?
  "running".casecmp(state).zero?
end

#started_at ⇒ Time?

The time when the job was started. This field is present after the job's state changes from PENDING to either RUNNING or DONE.

Returns:

  • (Time, nil) —

    The start time from the job statistics.



187
188
189
190
191
 
# File 'lib/google/cloud/bigquery/job.rb', line 187

def started_at
  ::Time.at(Integer(@gapi.statistics.start_time) / 1000.0)
rescue StandardError
  nil
end

#state ⇒ String

The current state of the job. A DONE state does not mean that the job completed successfully. Use #failed? to discover if an error occurred or if the job was successful.

Returns:

  • (String) —

    The state code. The possible values are PENDING, RUNNING, and DONE.



121
122
123
124
 
# File 'lib/google/cloud/bigquery/job.rb', line 121

def state
  return nil if @gapi.status.nil?
  @gapi.status.state
end

#statistics ⇒ Hash Also known as: stats

The statistics for the job. Returns a hash.

Returns:

  • (Hash) —

    The job statistics.

See Also:



223
224
225
 
# File 'lib/google/cloud/bigquery/job.rb', line 223

def statistics
  JSON.parse @gapi.statistics.to_json
end

#status ⇒ Hash

The job's status. Returns a hash. The values contained in the hash are also exposed by #state, #error, and #errors.

Returns:

  • (Hash) —

    The job status.



234
235
236
 
# File 'lib/google/cloud/bigquery/job.rb', line 234

def status
  JSON.parse @gapi.status.to_json
end

#user_email ⇒ String

The email address of the user who ran the job.

Returns:

  • (String) —

    The email address.



109
110
111
 
# File 'lib/google/cloud/bigquery/job.rb', line 109

def user_email
  @gapi.user_email
end

#wait_until_done! ⇒ Object

Refreshes the job until the job is DONE. The delay between refreshes starts at 5 seconds and increases exponentially to a maximum of 60 seconds.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

extract_job = table.extract_job "gs://my-bucket/file-name.json",
                                format: "json"
extract_job.wait_until_done!
extract_job.done? #=> true


375
376
377
378
379
380
381
382
383
384
385
386
 
# File 'lib/google/cloud/bigquery/job.rb', line 375

def wait_until_done!
  backoff = lambda do |retries|
    delay = [retries**2 + 5, 60].min # Maximum delay is 60
    sleep delay
  end
  retries = 0
  until done?
    backoff.call retries
    retries += 1
    reload!
  end
end