Class: Google::Cloud::Bigquery::External::DataSource
- Inherits:
-
Object
- Object
- Google::Cloud::Bigquery::External::DataSource
- Defined in:
- lib/google/cloud/bigquery/external.rb
Overview
DataSource
External::DataSource and its subclasses represents an external data source that can be queried from directly, even though the data is not stored in BigQuery. Instead of loading or streaming the data, this object references the external data source.
The AVRO and Datastore Backup formats use DataSource. See CsvSource, JsonSource, SheetsSource, BigtableSource for the other formats.
Direct Known Subclasses
Instance Method Summary collapse
-
#autodetect ⇒ Boolean
Indicates if the schema and format options are detected automatically.
-
#autodetect=(new_autodetect) ⇒ Object
Set whether to detect schema and format options automatically.
-
#avro? ⇒ Boolean
Whether the data format is "AVRO".
-
#backup? ⇒ Boolean
Whether the data format is "DATASTORE_BACKUP".
-
#bigtable? ⇒ Boolean
Whether the data format is "BIGTABLE".
-
#compression ⇒ String
The compression type of the data source.
-
#compression=(new_compression) ⇒ Object
Set the compression type of the data source.
-
#csv? ⇒ Boolean
Whether the data format is "CSV".
-
#format ⇒ String
The data format.
-
#ignore_unknown ⇒ Boolean
Indicates if BigQuery should allow extra values that are not represented in the table schema.
-
#ignore_unknown=(new_ignore_unknown) ⇒ Object
Set whether BigQuery should allow extra values that are not represented in the table schema.
-
#json? ⇒ Boolean
Whether the data format is "NEWLINE_DELIMITED_JSON".
-
#max_bad_records ⇒ Integer
The maximum number of bad records that BigQuery can ignore when reading data.
-
#max_bad_records=(new_max_bad_records) ⇒ Object
Set the maximum number of bad records that BigQuery can ignore when reading data.
-
#sheets? ⇒ Boolean
Whether the data format is "GOOGLE_SHEETS".
-
#urls ⇒ Array<String>
The fully-qualified URIs that point to your data in Google Cloud.
Instance Method Details
#autodetect ⇒ Boolean
Indicates if the schema and format options are detected automatically.
352 353 354 |
# File 'lib/google/cloud/bigquery/external.rb', line 352 def autodetect @gapi.autodetect end |
#autodetect=(new_autodetect) ⇒ Object
Set whether to detect schema and format options automatically. Any option specified explicitly will be honored.
374 375 376 377 |
# File 'lib/google/cloud/bigquery/external.rb', line 374 def autodetect= new_autodetect frozen_check! @gapi.autodetect = new_autodetect end |
#avro? ⇒ Boolean
Whether the data format is "AVRO".
263 264 265 |
# File 'lib/google/cloud/bigquery/external.rb', line 263 def avro? @gapi.source_format == "AVRO" end |
#backup? ⇒ Boolean
Whether the data format is "DATASTORE_BACKUP".
283 284 285 |
# File 'lib/google/cloud/bigquery/external.rb', line 283 def backup? @gapi.source_format == "DATASTORE_BACKUP" end |
#bigtable? ⇒ Boolean
Whether the data format is "BIGTABLE".
303 304 305 |
# File 'lib/google/cloud/bigquery/external.rb', line 303 def bigtable? @gapi.source_format == "BIGTABLE" end |
#compression ⇒ String
The compression type of the data source. Possible values include
"GZIP"
and nil
. The default value is nil
. This setting is
ignored for Google Cloud Bigtable, Google Cloud Datastore backups
and Avro formats. Optional.
398 399 400 |
# File 'lib/google/cloud/bigquery/external.rb', line 398 def compression @gapi.compression end |
#compression=(new_compression) ⇒ Object
Set the compression type of the data source. Possible values include
"GZIP"
and nil
. The default value is nil
. This setting is
ignored for Google Cloud Bigtable, Google Cloud Datastore backups
and Avro formats. Optional.
422 423 424 425 |
# File 'lib/google/cloud/bigquery/external.rb', line 422 def compression= new_compression frozen_check! @gapi.compression = new_compression end |
#csv? ⇒ Boolean
Whether the data format is "CSV".
203 204 205 |
# File 'lib/google/cloud/bigquery/external.rb', line 203 def csv? @gapi.source_format == "CSV" end |
#format ⇒ String
The data format. For CSV files, specify "CSV". For Google sheets, specify "GOOGLE_SHEETS". For newline-delimited JSON, specify "NEWLINE_DELIMITED_JSON". For Avro files, specify "AVRO". For Google Cloud Datastore backups, specify "DATASTORE_BACKUP". [Beta] For Google Cloud Bigtable, specify "BIGTABLE".
183 184 185 |
# File 'lib/google/cloud/bigquery/external.rb', line 183 def format @gapi.source_format end |
#ignore_unknown ⇒ Boolean
Indicates if BigQuery should allow extra values that are not
represented in the table schema. If true
, the extra values are
ignored. If false
, records with extra columns are treated as bad
records, and if there are too many bad records, an invalid error is
returned in the job result. The default value is false
.
BigQuery treats trailing columns as an extra in CSV
, named values
that don't match any column names in JSON
. This setting is ignored
for Google Cloud Bigtable, Google Cloud Datastore backups and Avro
formats. Optional.
453 454 455 |
# File 'lib/google/cloud/bigquery/external.rb', line 453 def ignore_unknown @gapi.ignore_unknown_values end |
#ignore_unknown=(new_ignore_unknown) ⇒ Object
Set whether BigQuery should allow extra values that are not
represented in the table schema. If true
, the extra values are
ignored. If false
, records with extra columns are treated as bad
records, and if there are too many bad records, an invalid error is
returned in the job result. The default value is false
.
BigQuery treats trailing columns as an extra in CSV
, named values
that don't match any column names in JSON
. This setting is ignored
for Google Cloud Bigtable, Google Cloud Datastore backups and Avro
formats. Optional.
483 484 485 486 |
# File 'lib/google/cloud/bigquery/external.rb', line 483 def ignore_unknown= new_ignore_unknown frozen_check! @gapi.ignore_unknown_values = new_ignore_unknown end |
#json? ⇒ Boolean
Whether the data format is "NEWLINE_DELIMITED_JSON".
223 224 225 |
# File 'lib/google/cloud/bigquery/external.rb', line 223 def json? @gapi.source_format == "NEWLINE_DELIMITED_JSON" end |
#max_bad_records ⇒ Integer
The maximum number of bad records that BigQuery can ignore when reading data. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups and Avro formats.
510 511 512 |
# File 'lib/google/cloud/bigquery/external.rb', line 510 def max_bad_records @gapi.max_bad_records end |
#max_bad_records=(new_max_bad_records) ⇒ Object
Set the maximum number of bad records that BigQuery can ignore when reading data. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups and Avro formats.
536 537 538 539 |
# File 'lib/google/cloud/bigquery/external.rb', line 536 def max_bad_records= new_max_bad_records frozen_check! @gapi.max_bad_records = new_max_bad_records end |
#sheets? ⇒ Boolean
Whether the data format is "GOOGLE_SHEETS".
243 244 245 |
# File 'lib/google/cloud/bigquery/external.rb', line 243 def sheets? @gapi.source_format == "GOOGLE_SHEETS" end |
#urls ⇒ Array<String>
The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one '' wildcard character and it must come after the 'bucket' name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified, and it must end with '.backup_info'. Also, the '' wildcard character is not allowed.
330 331 332 |
# File 'lib/google/cloud/bigquery/external.rb', line 330 def urls @gapi.source_uris end |