Class: Google::Cloud::Bigquery::LoadJob

Inherits:
Job
  • Object
show all
Defined in:
lib/google/cloud/bigquery/load_job.rb

Overview

LoadJob

A Job subclass representing a load operation that may be performed on a Table. A LoadJob instance is created when you call Table#load.

See Also:

Instance Method Summary collapse

Methods inherited from Job

#configuration, #created_at, #done?, #ended_at, #error, #errors, #failed?, #job_id, #pending?, #project_id, #reload!, #rerun!, #running?, #started_at, #state, #statistics, #status, #wait_until_done!

Instance Method Details

#allow_jagged_rows? ⇒ Boolean

Checks if the load operation accepts rows that are missing trailing optional columns. The missing values are treated as nulls. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an error is returned. The default value is false. Only applicable to CSV, ignored for other formats.

Returns:

  • (Boolean) 


141
142
143
144
145
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 141

def allow_jagged_rows?
  val = @gapi.configuration.load.allow_jagged_rows
  val = false if val.nil?
  val
end

#backup? ⇒ Boolean

Checks if the source data is a Google Cloud Datastore backup.

Returns:

  • (Boolean) 


130
131
132
133
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 130

def backup?
  val = @gapi.configuration.load.source_format
  val == "DATASTORE_BACKUP"
end

#csv? ⇒ Boolean

Checks if the format of the source data is CSV. The default is true.

Returns:

  • (Boolean) 


122
123
124
125
126
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 122

def csv?
  val = @gapi.configuration.load.source_format
  return true if val.nil?
  val == "CSV"
end

#delimiter ⇒ Object

The delimiter used between fields in the source data. The default is a comma (,).



54
55
56
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 54

def delimiter
  @gapi.configuration.load.field_delimiter || ","
end

#destination ⇒ Object

The table into which the operation loads data. This is the table on which Table#load was invoked. Returns a Table instance.



43
44
45
46
47
48
49
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 43

def destination
  table = @gapi.configuration.load.destination_table
  return nil unless table
  retrieve_table table.project_id,
                 table.dataset_id,
                 table.table_id
end

#ignore_unknown_values? ⇒ Boolean

Checks if the load operation allows extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned. The default is false.

Returns:

  • (Boolean) 


153
154
155
156
157
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 153

def ignore_unknown_values?
  val = @gapi.configuration.load.ignore_unknown_values
  val = false if val.nil?
  val
end

#input_file_bytes ⇒ Object

The number of bytes of source data.



177
178
179
180
181
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 177

def input_file_bytes
  Integer @gapi.statistics.load.input_file_bytes
rescue
  nil
end

#input_files ⇒ Object

The number of source files.



169
170
171
172
173
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 169

def input_files
  Integer @gapi.statistics.load.input_files
rescue
  nil
end

#iso8859_1? ⇒ Boolean

Checks if the character encoding of the data is ISO-8859-1.

Returns:

  • (Boolean) 


76
77
78
79
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 76

def iso8859_1?
  val = @gapi.configuration.load.encoding
  val == "ISO-8859-1"
end

#json? ⇒ Boolean

Checks if the format of the source data is newline-delimited JSON. The default is false.

Returns:

  • (Boolean) 


115
116
117
118
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 115

def json?
  val = @gapi.configuration.load.source_format
  val == "NEWLINE_DELIMITED_JSON"
end

#max_bad_records ⇒ Object

The maximum number of bad records that the load operation can ignore. If the number of bad records exceeds this value, an error is returned. The default value is 0, which requires that all records be valid.



97
98
99
100
101
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 97

def max_bad_records
  val = @gapi.configuration.load.max_bad_records
  val = 0 if val.nil?
  val
end

#output_bytes ⇒ Object

The number of bytes that have been loaded into the table. While an import job is in the running state, this value may change.



195
196
197
198
199
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 195

def output_bytes
  Integer @gapi.statistics.load.output_bytes
rescue
  nil
end

#output_rows ⇒ Object

The number of rows that have been loaded into the table. While an import job is in the running state, this value may change.



186
187
188
189
190
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 186

def output_rows
  Integer @gapi.statistics.load.output_rows
rescue
  nil
end

#quote ⇒ Object

The value that is used to quote data sections in a CSV file. The default value is a double-quote ("). If your data does not contain quoted sections, the value should be an empty string. If your data contains quoted newline characters, #quoted_newlines? should return true.



87
88
89
90
91
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 87

def quote
  val = @gapi.configuration.load.quote
  val = "\"" if val.nil?
  val
end

#quoted_newlines? ⇒ Boolean

Checks if quoted data sections may contain newline characters in a CSV file. The default is false.

Returns:

  • (Boolean) 


106
107
108
109
110
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 106

def quoted_newlines?
  val = @gapi.configuration.load.allow_quoted_newlines
  val = true if val.nil?
  val
end

#schema ⇒ Object

The schema for the data. Returns a hash. Can be empty if the table has already has the correct schema (see Table#schema), or if the schema can be inferred from the loaded data.



163
164
165
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 163

def schema
  Schema.from_gapi(@gapi.configuration.load.schema).freeze
end

#skip_leading_rows ⇒ Object

The number of header rows at the top of a CSV file to skip. The default value is 0.



61
62
63
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 61

def skip_leading_rows
  @gapi.configuration.load.skip_leading_rows || 0
end

#sources ⇒ Object

The URI or URIs representing the Google Cloud Storage files from which the operation loads data.



36
37
38
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 36

def sources
  Array @gapi.configuration.load.source_uris
end

#utf8? ⇒ Boolean

Checks if the character encoding of the data is UTF-8. This is the default.

Returns:

  • (Boolean) 


68
69
70
71
72
 
# File 'lib/google/cloud/bigquery/load_job.rb', line 68

def utf8?
  val = @gapi.configuration.load.encoding
  return true if val.nil?
  val == "UTF-8"
end