Class: Google::Cloud::Bigquery::LoadJob
- Defined in:
- lib/google/cloud/bigquery/load_job.rb
Overview
LoadJob
A Job subclass representing a load operation that may be performed on a Table. A LoadJob instance is created when you call Table#load_job.
Instance Method Summary collapse
-
#allow_jagged_rows? ⇒ Boolean
Checks if the load operation accepts rows that are missing trailing optional columns.
-
#autodetect? ⇒ Boolean
Checks if BigQuery should automatically infer the options and schema for CSV and JSON sources.
-
#backup? ⇒ Boolean
Checks if the source data is a Google Cloud Datastore backup.
-
#csv? ⇒ Boolean
Checks if the format of the source data is CSV.
-
#delimiter ⇒ String
The delimiter used between fields in the source data.
-
#destination ⇒ Table
The table into which the operation loads data.
-
#ignore_unknown_values? ⇒ Boolean
Checks if the load operation allows extra values that are not represented in the table schema.
-
#input_file_bytes ⇒ Integer
The number of bytes of source data in the load job.
-
#input_files ⇒ Integer
The number of source data files in the load job.
-
#iso8859_1? ⇒ Boolean
Checks if the character encoding of the data is ISO-8859-1.
-
#json? ⇒ Boolean
Checks if the format of the source data is newline-delimited JSON.
-
#max_bad_records ⇒ Integer
The maximum number of bad records that the load operation can ignore.
-
#null_marker ⇒ String
Specifies a string that represents a null value in a CSV file.
-
#output_bytes ⇒ Integer
The number of bytes that have been loaded into the table.
-
#output_rows ⇒ Integer
The number of rows that have been loaded into the table.
-
#quote ⇒ String
The value that is used to quote data sections in a CSV file.
-
#quoted_newlines? ⇒ Boolean
Checks if quoted data sections may contain newline characters in a CSV file.
-
#schema ⇒ Schema?
The schema for the destination table.
-
#skip_leading_rows ⇒ Integer
The number of rows at the top of a CSV file that BigQuery will skip when loading the data.
-
#sources ⇒ Object
The URI or URIs representing the Google Cloud Storage files from which the operation loads data.
-
#utf8? ⇒ Boolean
Checks if the character encoding of the data is UTF-8.
Methods inherited from Job
#cancel, #configuration, #created_at, #done?, #ended_at, #error, #errors, #failed?, #job_id, #labels, #pending?, #project_id, #reload!, #rerun!, #running?, #started_at, #state, #statistics, #status, #user_email, #wait_until_done!
Instance Method Details
#allow_jagged_rows? ⇒ Boolean
Checks if the load operation accepts rows that are missing trailing
optional columns. The missing values are treated as nulls. If false
,
records with missing trailing columns are treated as bad records, and
if there are too many bad records, an error is returned. The default
value is false
. Only applicable to CSV, ignored for other formats.
236 237 238 239 240 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 236 def allow_jagged_rows? val = @gapi.configuration.load.allow_jagged_rows val = false if val.nil? val end |
#autodetect? ⇒ Boolean
Checks if BigQuery should automatically infer the options and schema
for CSV and JSON sources. The default is false
.
185 186 187 188 189 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 185 def autodetect? val = @gapi.configuration.load.autodetect val = false if val.nil? val end |
#backup? ⇒ Boolean
Checks if the source data is a Google Cloud Datastore backup.
221 222 223 224 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 221 def backup? val = @gapi.configuration.load.source_format val == "DATASTORE_BACKUP" end |
#csv? ⇒ Boolean
Checks if the format of the source data is CSV. The default is true
.
209 210 211 212 213 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 209 def csv? val = @gapi.configuration.load.source_format return true if val.nil? val == "CSV" end |
#delimiter ⇒ String
The delimiter used between fields in the source data. The default is a comma (,).
80 81 82 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 80 def delimiter @gapi.configuration.load.field_delimiter || "," end |
#destination ⇒ Table
The table into which the operation loads data. This is the table on which Table#load_job was invoked.
66 67 68 69 70 71 72 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 66 def destination table = @gapi.configuration.load.destination_table return nil unless table retrieve_table table.project_id, table.dataset_id, table.table_id end |
#ignore_unknown_values? ⇒ Boolean
Checks if the load operation allows extra values that are not
represented in the table schema. If true
, the extra values are
ignored. If false
, records with extra columns are treated as bad
records, and if there are too many bad records, an invalid error is
returned. The default is false
.
252 253 254 255 256 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 252 def ignore_unknown_values? val = @gapi.configuration.load.ignore_unknown_values val = false if val.nil? val end |
#input_file_bytes ⇒ Integer
The number of bytes of source data in the load job.
288 289 290 291 292 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 288 def input_file_bytes Integer @gapi.statistics.load.input_file_bytes rescue nil end |
#input_files ⇒ Integer
The number of source data files in the load job.
277 278 279 280 281 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 277 def input_files Integer @gapi.statistics.load.input_files rescue nil end |
#iso8859_1? ⇒ Boolean
Checks if the character encoding of the data is ISO-8859-1.
115 116 117 118 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 115 def iso8859_1? val = @gapi.configuration.load.encoding val == "ISO-8859-1" end |
#json? ⇒ Boolean
Checks if the format of the source data is newline-delimited
JSON. The default is false
.
198 199 200 201 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 198 def json? val = @gapi.configuration.load.source_format val == "NEWLINE_DELIMITED_JSON" end |
#max_bad_records ⇒ Integer
The maximum number of bad records that the load operation can ignore.
If the number of bad records exceeds this value, an error is returned.
The default value is 0
, which requires that all records be valid.
142 143 144 145 146 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 142 def max_bad_records val = @gapi.configuration.load.max_bad_records val = 0 if val.nil? val end |
#null_marker ⇒ String
Specifies a string that represents a null value in a CSV file. For
example, if you specify \N
, BigQuery interprets \N
as a null value
when loading a CSV file. The default value is the empty string. If you
set this property to a custom value, BigQuery throws an error if an
empty string is present for all data types except for STRING and BYTE.
For STRING and BYTE columns, BigQuery interprets the empty string as
an empty value.
159 160 161 162 163 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 159 def null_marker val = @gapi.configuration.load.null_marker val = "" if val.nil? val end |
#output_bytes ⇒ Integer
The number of bytes that have been loaded into the table. While an import job is in the running state, this value may change.
312 313 314 315 316 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 312 def output_bytes Integer @gapi.statistics.load.output_bytes rescue nil end |
#output_rows ⇒ Integer
The number of rows that have been loaded into the table. While an import job is in the running state, this value may change.
300 301 302 303 304 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 300 def output_rows Integer @gapi.statistics.load.output_rows rescue nil end |
#quote ⇒ String
The value that is used to quote data sections in a CSV file. The
default value is a double-quote ("
). If your data does not contain
quoted sections, the value should be an empty string. If your data
contains quoted newline characters, #quoted_newlines? should return
true
.
129 130 131 132 133 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 129 def quote val = @gapi.configuration.load.quote val = "\"" if val.nil? val end |
#quoted_newlines? ⇒ Boolean
Checks if quoted data sections may contain newline characters in a CSV
file. The default is false
.
172 173 174 175 176 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 172 def quoted_newlines? val = @gapi.configuration.load.allow_quoted_newlines val = false if val.nil? val end |
#schema ⇒ Schema?
The schema for the destination table. The schema can be omitted if the destination table already exists, or if you're loading data from Google Cloud Datastore.
The returned object is frozen and changes are not allowed. Use Table#schema to update the schema.
268 269 270 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 268 def schema Schema.from_gapi(@gapi.configuration.load.schema).freeze end |
#skip_leading_rows ⇒ Integer
The number of rows at the top of a CSV file that BigQuery will skip when loading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped.
92 93 94 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 92 def skip_leading_rows @gapi.configuration.load.skip_leading_rows || 0 end |
#sources ⇒ Object
The URI or URIs representing the Google Cloud Storage files from which the operation loads data.
56 57 58 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 56 def sources Array @gapi.configuration.load.source_uris end |
#utf8? ⇒ Boolean
Checks if the character encoding of the data is UTF-8. This is the default.
103 104 105 106 107 |
# File 'lib/google/cloud/bigquery/load_job.rb', line 103 def utf8? val = @gapi.configuration.load.encoding return true if val.nil? val == "UTF-8" end |