Class: Google::Cloud::Bigquery::Job

Inherits:
Object
  • Object
show all
Defined in:
lib/google/cloud/bigquery/job.rb,
lib/google/cloud/bigquery/job/list.rb

Overview

Job

Represents a generic Job that may be performed on a Table.

The subclasses of Job represent the specific BigQuery job types: CopyJob, ExtractJob, LoadJob, and QueryJob.

A job instance is created when you call Project#query_job, Dataset#query_job, Table#copy_job, Table#extract_job, Table#load_job.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

job = bigquery.query_job "SELECT COUNT(word) as count FROM " \
                         "publicdata.samples.shakespeare"

job.wait_until_done!

if job.failed?
  puts job.error
else
  puts job.data.first
end

See Also:

Direct Known Subclasses

CopyJob, ExtractJob, LoadJob, QueryJob

Defined Under Namespace

Classes: List

Attributes collapse

Instance Method Summary collapse

Instance Method Details

#cancel ⇒ Object

Cancels the job.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

job = bigquery.query_job "SELECT COUNT(word) as count FROM " \
                         "publicdata.samples.shakespeare"

job.cancel


292
293
294
295
296
297
 
# File 'lib/google/cloud/bigquery/job.rb', line 292

def cancel
  ensure_service!
  resp = service.cancel_job job_id
  @gapi = resp.job
  true
end

#configuration ⇒ Object Also known as: config

The configuration for the job. Returns a hash.

See Also:



199
200
201
 
# File 'lib/google/cloud/bigquery/job.rb', line 199

def configuration
  JSON.parse @gapi.configuration.to_json
end

#created_at ⇒ Time?

The time when the job was created.

Returns:

  • (Time, nil) —

    The creation time from the job statistics.



163
164
165
166
167
 
# File 'lib/google/cloud/bigquery/job.rb', line 163

def created_at
  ::Time.at(Integer(@gapi.statistics.creation_time) / 1000.0)
rescue
  nil
end

#done? ⇒ Boolean

Checks if the job's state is DONE. When true, the job has stopped running. However, a DONE state does not mean that the job completed successfully. Use #failed? to detect if an error occurred or if the job was successful.

Returns:

  • (Boolean) —

    true when DONE, false otherwise.



143
144
145
146
 
# File 'lib/google/cloud/bigquery/job.rb', line 143

def done?
  return false if state.nil?
  "done".casecmp(state).zero?
end

#ended_at ⇒ Time?

The time when the job ended. This field is present when the job's state is DONE.

Returns:

  • (Time, nil) —

    The end time from the job statistics.



188
189
190
191
192
 
# File 'lib/google/cloud/bigquery/job.rb', line 188

def ended_at
  ::Time.at(Integer(@gapi.statistics.end_time) / 1000.0)
rescue
  nil
end

#error ⇒ Hash?

The last error for the job, if any errors have occurred. Returns a hash.

Returns:

  • (Hash, nil) —

    Returns a hash containing reason and message keys:

    { "reason"=>"notFound", "message"=>"Not found: Table publicdata:samples.BAD_ID" }

See Also:



242
243
244
 
# File 'lib/google/cloud/bigquery/job.rb', line 242

def error
  status["errorResult"]
end

#errors ⇒ Array<Hash>?

The errors for the job, if any errors have occurred. Returns an array of hash objects. See #error.

Returns:

  • (Array<Hash>, nil) —

    Returns an array of hashes containing reason and message keys:

    { "reason"=>"notFound", "message"=>"Not found: Table publicdata:samples.BAD_ID" }



258
259
260
 
# File 'lib/google/cloud/bigquery/job.rb', line 258

def errors
  Array status["errors"]
end

#failed? ⇒ Boolean

Checks if an error is present. Use #error to access the error object.

Returns:

  • (Boolean) —

    true when there is an error, false otherwise.



154
155
156
 
# File 'lib/google/cloud/bigquery/job.rb', line 154

def failed?
  !error.nil?
end

#job_id ⇒ String

The ID of the job.

Returns:

  • (String) —

    The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or dashes (-). The maximum length is 1,024 characters.



80
81
82
 
# File 'lib/google/cloud/bigquery/job.rb', line 80

def job_id
  @gapi.job_reference.job_id
end

#labels ⇒ Hash

A hash of user-provided labels associated with this job. Labels can be provided when the job is created, and used to organize and group jobs.

The returned hash is frozen and changes are not allowed. Use #labels= to replace the entire hash.

Returns:

  • (Hash) —

    The job labels.



273
274
275
276
277
 
# File 'lib/google/cloud/bigquery/job.rb', line 273

def labels
  m = @gapi.configuration.labels
  m = m.to_h if m.respond_to? :to_h
  m.dup.freeze
end

#pending? ⇒ Boolean

Checks if the job's state is PENDING.

Returns:

  • (Boolean) —

    true when PENDING, false otherwise.



130
131
132
133
 
# File 'lib/google/cloud/bigquery/job.rb', line 130

def pending?
  return false if state.nil?
  "pending".casecmp(state).zero?
end

#project_id ⇒ String

The ID of the project containing the job.

Returns:

  • (String) —

    The project ID.



89
90
91
 
# File 'lib/google/cloud/bigquery/job.rb', line 89

def project_id
  @gapi.job_reference.project_id
end

#reload! ⇒ Object Also known as: refresh!

Reloads the job with current data from the BigQuery service.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

job = bigquery.query_job "SELECT COUNT(word) as count FROM " \
                         "publicdata.samples.shakespeare"

job.done?
job.reload!
job.done? #=> true


334
335
336
337
338
 
# File 'lib/google/cloud/bigquery/job.rb', line 334

def reload!
  ensure_service!
  gapi = service.get_job job_id
  @gapi = gapi
end

#rerun! ⇒ Object

Created a new job with the current configuration.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

job = bigquery.query_job "SELECT COUNT(word) as count FROM " \
                         "publicdata.samples.shakespeare"

job.wait_until_done!
job.rerun!


313
314
315
316
317
 
# File 'lib/google/cloud/bigquery/job.rb', line 313

def rerun!
  ensure_service!
  gapi = service.insert_job @gapi.configuration
  Job.from_gapi gapi, service
end

#running? ⇒ Boolean

Checks if the job's state is RUNNING.

Returns:

  • (Boolean) —

    true when RUNNING, false otherwise.



120
121
122
123
 
# File 'lib/google/cloud/bigquery/job.rb', line 120

def running?
  return false if state.nil?
  "running".casecmp(state).zero?
end

#started_at ⇒ Time?

The time when the job was started. This field is present after the job's state changes from PENDING to either RUNNING or DONE.

Returns:

  • (Time, nil) —

    The start time from the job statistics.



176
177
178
179
180
 
# File 'lib/google/cloud/bigquery/job.rb', line 176

def started_at
  ::Time.at(Integer(@gapi.statistics.start_time) / 1000.0)
rescue
  nil
end

#state ⇒ String

The current state of the job. A DONE state does not mean that the job completed successfully. Use #failed? to discover if an error occurred or if the job was successful.

Returns:

  • (String) —

    The state code. The possible values are PENDING, RUNNING, and DONE.



110
111
112
113
 
# File 'lib/google/cloud/bigquery/job.rb', line 110

def state
  return nil if @gapi.status.nil?
  @gapi.status.state
end

#statistics ⇒ Hash Also known as: stats

The statistics for the job. Returns a hash.

Returns:

  • (Hash) —

    The job statistics.

See Also:



212
213
214
 
# File 'lib/google/cloud/bigquery/job.rb', line 212

def statistics
  JSON.parse @gapi.statistics.to_json
end

#status ⇒ Hash

The job's status. Returns a hash. The values contained in the hash are also exposed by #state, #error, and #errors.

Returns:

  • (Hash) —

    The job status.



223
224
225
 
# File 'lib/google/cloud/bigquery/job.rb', line 223

def status
  JSON.parse @gapi.status.to_json
end

#user_email ⇒ String

The email address of the user who ran the job.

Returns:

  • (String) —

    The email address.



98
99
100
 
# File 'lib/google/cloud/bigquery/job.rb', line 98

def user_email
  @gapi.user_email
end

#wait_until_done! ⇒ Object

Refreshes the job until the job is DONE. The delay between refreshes starts at 5 seconds and increases exponentially to a maximum of 60 seconds.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

extract_job = table.extract_job "gs://my-bucket/file-name.json",
                                format: "json"
extract_job.wait_until_done!
extract_job.done? #=> true


358
359
360
361
362
363
364
365
366
367
368
369
 
# File 'lib/google/cloud/bigquery/job.rb', line 358

def wait_until_done!
  backoff = lambda do |retries|
    delay = [retries ** 2 + 5, 60].min # Maximum delay is 60
    sleep delay
  end
  retries = 0
  until done?
    backoff.call retries
    retries += 1
    reload!
  end
end