Class: Google::Cloud::Bigquery::Table
- Inherits:
-
Object
- Object
- Google::Cloud::Bigquery::Table
- Defined in:
- lib/google/cloud/bigquery/table.rb,
lib/google/cloud/bigquery/table/list.rb,
lib/google/cloud/bigquery/table/async_inserter.rb
Overview
Table
A named resource representing a BigQuery table that holds zero or more records. Every table is defined by a schema that may contain nested and repeated fields.
The Table class can also represent a view, which is a virtual table defined by a SQL query. BigQuery's views are logical views, not materialized views, which means that the query that defines the view is re-executed every time the view is queried. Queries are billed according to the total amount of data in all table fields referenced directly or indirectly by the top-level query. (See #view?, #query, #query=, and Dataset#create_view.)
Direct Known Subclasses
Defined Under Namespace
Classes: AsyncInserter, List, Updater
Attributes collapse
-
#api_url ⇒ String?
A URL that can be used to access the table using the REST API.
-
#buffer_bytes ⇒ Integer?
A lower-bound estimate of the number of bytes currently in this table's streaming buffer, if one is present.
-
#buffer_oldest_at ⇒ Time?
The time of the oldest entry currently in this table's streaming buffer, if one is present.
-
#buffer_rows ⇒ Integer?
A lower-bound estimate of the number of rows currently in this table's streaming buffer, if one is present.
-
#created_at ⇒ Time?
The time when this table was created.
-
#dataset_id ⇒ String
The ID of the
Dataset
containing this table. -
#description ⇒ String?
A user-friendly description of the table.
-
#description=(new_description) ⇒ Object
Updates the user-friendly description of the table.
-
#encryption ⇒ EncryptionConfiguration?
The EncryptionConfiguration object that represents the custom encryption method used to protect the table.
-
#encryption=(value) ⇒ Object
Set the EncryptionConfiguration object that represents the custom encryption method used to protect the table.
-
#etag ⇒ String?
The ETag hash of the table.
-
#expires_at ⇒ Time?
The time when this table expires.
-
#external ⇒ External::DataSource?
The External::DataSource (or subclass) object that represents the external data source that the table represents.
-
#external=(external) ⇒ Object
Set the External::DataSource (or subclass) object that represents the external data source that the table represents.
-
#external? ⇒ Boolean?
Checks if the table's type is "EXTERNAL", indicating that the table represents an External Data Source.
-
#fields ⇒ Array<Schema::Field>?
The fields of the table, obtained from its schema.
-
#headers ⇒ Array<Symbol>?
The names of the columns in the table, obtained from its schema.
-
#id ⇒ String?
The combined Project ID, Dataset ID, and Table ID for this table, in the format specified by the Legacy SQL Query Reference:
project_name:datasetId.tableId
. -
#labels ⇒ Hash<String, String>?
A hash of user-provided labels associated with this table.
-
#labels=(labels) ⇒ Object
Updates the hash of user-provided labels associated with this table.
-
#location ⇒ String?
The geographic location where the table should reside.
-
#modified_at ⇒ Time?
The date when this table was last modified.
-
#name ⇒ String?
The name of the table.
-
#name=(new_name) ⇒ Object
Updates the name of the table.
-
#project_id ⇒ String
The ID of the
Project
containing this table. -
#query ⇒ String
The query that executes each time the view is loaded.
-
#query_id(standard_sql: nil, legacy_sql: nil) ⇒ String
The value returned by #id, wrapped in square brackets if the Project ID contains dashes, as specified by the Query Reference.
-
#query_legacy_sql? ⇒ Boolean
Checks if the view's query is using legacy sql.
-
#query_standard_sql? ⇒ Boolean
Checks if the view's query is using standard sql.
-
#query_udfs ⇒ Array<String>
The user-defined function resources used in the view's query.
-
#schema(replace: false) {|schema| ... } ⇒ Google::Cloud::Bigquery::Schema?
Returns the table's schema.
-
#table? ⇒ Boolean?
Checks if the table's type is "TABLE".
-
#table_id ⇒ String
A unique ID for this table.
-
#time_partitioning? ⇒ Boolean?
Checks if the table is time-partitioned.
-
#time_partitioning_expiration ⇒ Integer?
The expiration for the table partitions, if any, in seconds.
-
#time_partitioning_expiration=(expiration) ⇒ Object
Sets the partition expiration for the table.
-
#time_partitioning_field ⇒ String?
The field on which the table is partitioned, if any.
-
#time_partitioning_field=(field) ⇒ Object
Sets the field on which to partition the table.
-
#time_partitioning_type ⇒ String?
The period for which the table is partitioned, if any.
-
#time_partitioning_type=(type) ⇒ Object
Sets the partitioning for the table.
-
#view? ⇒ Boolean?
Checks if the table's type is "VIEW", indicating that the table represents a BigQuery view.
Data collapse
-
#bytes_count ⇒ Integer?
The number of bytes in the table.
-
#copy(destination_table, create: nil, write: nil) {|job| ... } ⇒ Boolean
Copies the data from the table to another table using a synchronous method that blocks for a response.
-
#copy_job(destination_table, create: nil, write: nil, dryrun: nil, job_id: nil, prefix: nil, labels: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::CopyJob
Copies the data from the table to another table using an asynchronous method.
-
#data(token: nil, max: nil, start: nil) ⇒ Google::Cloud::Bigquery::Data
Retrieves data from the table.
-
#extract(extract_url, format: nil, compression: nil, delimiter: nil, header: nil) {|job| ... } ⇒ Boolean
Extracts the data from the table to a Google Cloud Storage file using a synchronous method that blocks for a response.
-
#extract_job(extract_url, format: nil, compression: nil, delimiter: nil, header: nil, dryrun: nil, job_id: nil, prefix: nil, labels: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::ExtractJob
Extracts the data from the table to a Google Cloud Storage file using an asynchronous method.
-
#insert(rows, insert_ids: nil, skip_invalid: nil, ignore_unknown: nil) ⇒ Google::Cloud::Bigquery::InsertResponse
Inserts data into the table for near-immediate querying, without the need to complete a load operation before the data can appear in query results.
-
#insert_async(skip_invalid: nil, ignore_unknown: nil, max_bytes: 10000000, max_rows: 500, interval: 10, threads: 4) {|response| ... } ⇒ Table::AsyncInserter
Create an asynchronous inserter object used to insert rows in batches.
-
#load(files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, autodetect: nil, null_marker: nil) {|updater| ... } ⇒ Boolean
Loads data into the table.
-
#load_job(files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, dryrun: nil, job_id: nil, prefix: nil, labels: nil, autodetect: nil, null_marker: nil) {|load_job| ... } ⇒ Google::Cloud::Bigquery::LoadJob
Loads data into the table.
-
#rows_count ⇒ Integer?
The number of rows in the table.
Lifecycle collapse
-
#delete ⇒ Boolean
Permanently deletes the table.
-
#exists? ⇒ Boolean
Determines whether the table exists in the BigQuery service.
-
#query=(new_query) ⇒ Object
Updates the query that executes each time the view is loaded.
-
#reference? ⇒ Boolean
Whether the table was created without retrieving the resource representation from the BigQuery service.
-
#reload! ⇒ Google::Cloud::Bigquery::Table
(also: #refresh!)
Reloads the table with current data from the BigQuery service.
-
#resource? ⇒ Boolean
Whether the table was created with a resource representation from the BigQuery service.
-
#resource_full? ⇒ Boolean
Whether the table was created with a full resource representation from the BigQuery service.
-
#resource_partial? ⇒ Boolean
Whether the table was created with a partial resource representation from the BigQuery service by retrieval through Dataset#tables.
-
#set_query(query, standard_sql: nil, legacy_sql: nil, udfs: nil) ⇒ Object
Updates the query that executes each time the view is loaded.
Instance Method Details
#api_url ⇒ String?
A URL that can be used to access the table using the REST API.
437 438 439 440 441 |
# File 'lib/google/cloud/bigquery/table.rb', line 437 def api_url return nil if reference? ensure_full_data! @gapi.self_link end |
#buffer_bytes ⇒ Integer?
A lower-bound estimate of the number of bytes currently in this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
911 912 913 914 915 |
# File 'lib/google/cloud/bigquery/table.rb', line 911 def buffer_bytes return nil if reference? ensure_full_data! @gapi.streaming_buffer.estimated_bytes if @gapi.streaming_buffer end |
#buffer_oldest_at ⇒ Time?
The time of the oldest entry currently in this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
945 946 947 948 949 950 951 952 953 954 955 |
# File 'lib/google/cloud/bigquery/table.rb', line 945 def buffer_oldest_at return nil if reference? ensure_full_data! return nil unless @gapi.streaming_buffer oldest_entry_time = @gapi.streaming_buffer.oldest_entry_time begin ::Time.at(Integer(oldest_entry_time) / 1000.0) rescue StandardError nil end end |
#buffer_rows ⇒ Integer?
A lower-bound estimate of the number of rows currently in this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
929 930 931 932 933 |
# File 'lib/google/cloud/bigquery/table.rb', line 929 def buffer_rows return nil if reference? ensure_full_data! @gapi.streaming_buffer.estimated_rows if @gapi.streaming_buffer end |
#bytes_count ⇒ Integer?
The number of bytes in the table.
482 483 484 485 486 487 488 489 490 |
# File 'lib/google/cloud/bigquery/table.rb', line 482 def bytes_count return nil if reference? ensure_full_data! begin Integer @gapi.num_bytes rescue StandardError nil end end |
#copy(destination_table, create: nil, write: nil) {|job| ... } ⇒ Boolean
Copies the data from the table to another table using a synchronous method that blocks for a response. Timeouts and transient errors are generally handled as needed to complete the job. See also #copy_job.
The geographic location for the job ("US", "EU", etc.) can be set via CopyJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1306 1307 1308 1309 1310 1311 |
# File 'lib/google/cloud/bigquery/table.rb', line 1306 def copy destination_table, create: nil, write: nil, &block job = copy_job destination_table, create: create, write: write, &block job.wait_until_done! ensure_job_succeeded! job true end |
#copy_job(destination_table, create: nil, write: nil, dryrun: nil, job_id: nil, prefix: nil, labels: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::CopyJob
Copies the data from the table to another table using an asynchronous method. In this method, a CopyJob is immediately returned. The caller may poll the service by repeatedly calling Job#reload! and Job#done? to detect when the job is done, or simply block until the job is done by calling #Job#wait_until_done!. See also #copy.
The geographic location for the job ("US", "EU", etc.) can be set via CopyJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 |
# File 'lib/google/cloud/bigquery/table.rb', line 1224 def copy_job destination_table, create: nil, write: nil, dryrun: nil, job_id: nil, prefix: nil, labels: nil ensure_service! = { create: create, write: write, dryrun: dryrun, labels: labels, job_id: job_id, prefix: prefix } updater = CopyJob::Updater.( service, table_ref, get_table_ref(destination_table), ) updater.location = location if location # may be table reference yield updater if block_given? job_gapi = updater.to_gapi gapi = service.copy_table job_gapi Job.from_gapi gapi, service end |
#created_at ⇒ Time?
The time when this table was created.
518 519 520 521 522 523 524 525 526 |
# File 'lib/google/cloud/bigquery/table.rb', line 518 def created_at return nil if reference? ensure_full_data! begin ::Time.at(Integer(@gapi.creation_time) / 1000.0) rescue StandardError nil end end |
#data(token: nil, max: nil, start: nil) ⇒ Google::Cloud::Bigquery::Data
Retrieves data from the table.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the data retrieval.
1130 1131 1132 1133 1134 1135 1136 1137 |
# File 'lib/google/cloud/bigquery/table.rb', line 1130 def data token: nil, max: nil, start: nil ensure_service! reload! unless resource_full? = { token: token, max: max, start: start } data_json = service.list_tabledata \ dataset_id, table_id, Data.from_gapi_json data_json, gapi, service end |
#dataset_id ⇒ String
The ID of the Dataset
containing this table.
129 130 131 132 |
# File 'lib/google/cloud/bigquery/table.rb', line 129 def dataset_id return reference.dataset_id if reference? @gapi.table_reference.dataset_id end |
#delete ⇒ Boolean
Permanently deletes the table.
2016 2017 2018 2019 2020 |
# File 'lib/google/cloud/bigquery/table.rb', line 2016 def delete ensure_service! service.delete_table dataset_id, table_id true end |
#description ⇒ String?
A user-friendly description of the table.
451 452 453 454 455 |
# File 'lib/google/cloud/bigquery/table.rb', line 451 def description return nil if reference? ensure_full_data! @gapi.description end |
#description=(new_description) ⇒ Object
Updates the user-friendly description of the table.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
468 469 470 471 472 |
# File 'lib/google/cloud/bigquery/table.rb', line 468 def description= new_description reload! unless resource_full? @gapi.update! description: new_description patch_gapi! :description end |
#encryption ⇒ EncryptionConfiguration?
The EncryptionConfiguration object that represents the custom encryption method used to protect the table. If not set, default encryption is used.
Present only if the table is using custom encryption.
815 816 817 818 819 820 821 |
# File 'lib/google/cloud/bigquery/table.rb', line 815 def encryption return nil if reference? ensure_full_data! return nil if @gapi.encryption_configuration.nil? EncryptionConfiguration.from_gapi(@gapi.encryption_configuration) .freeze end |
#encryption=(value) ⇒ Object
Set the EncryptionConfiguration object that represents the custom encryption method used to protect the table. If not set, default encryption is used.
Present only if the table is using custom encryption.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
842 843 844 845 846 |
# File 'lib/google/cloud/bigquery/table.rb', line 842 def encryption= value reload! unless resource_full? @gapi.encryption_configuration = value.to_gapi patch_gapi! :encryption_configuration end |
#etag ⇒ String?
The ETag hash of the table.
423 424 425 426 427 |
# File 'lib/google/cloud/bigquery/table.rb', line 423 def etag return nil if reference? ensure_full_data! @gapi.etag end |
#exists? ⇒ Boolean
Determines whether the table exists in the BigQuery service. The result is cached locally.
2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 |
# File 'lib/google/cloud/bigquery/table.rb', line 2063 def exists? # Always true if we have a gapi object return true unless reference? # If we have a value, return it return @exists unless @exists.nil? ensure_gapi! @exists = true rescue Google::Cloud::NotFoundError @exists = false end |
#expires_at ⇒ Time?
The time when this table expires. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.
538 539 540 541 542 543 544 545 546 |
# File 'lib/google/cloud/bigquery/table.rb', line 538 def expires_at return nil if reference? ensure_full_data! begin ::Time.at(Integer(@gapi.expiration_time) / 1000.0) rescue StandardError nil end end |
#external ⇒ External::DataSource?
The External::DataSource (or subclass) object that represents the external data source that the table represents. Data can be queried the table, even though the data is not stored in BigQuery. Instead of loading or streaming the data, this object references the external data source.
Present only if the table represents an External Data Source. See #external? and External::DataSource.
865 866 867 868 869 870 |
# File 'lib/google/cloud/bigquery/table.rb', line 865 def external return nil if reference? ensure_full_data! return nil if @gapi.external_data_configuration.nil? External.from_gapi(@gapi.external_data_configuration).freeze end |
#external=(external) ⇒ Object
Set the External::DataSource (or subclass) object that represents the external data source that the table represents. Data can be queried the table, even though the data is not stored in BigQuery. Instead of loading or streaming the data, this object references the external data source.
Use only if the table represents an External Data Source. See #external? and External::DataSource.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
893 894 895 896 897 |
# File 'lib/google/cloud/bigquery/table.rb', line 893 def external= external reload! unless resource_full? @gapi.external_data_configuration = external.to_gapi patch_gapi! :external_data_configuration end |
#external? ⇒ Boolean?
Checks if the table's type is "EXTERNAL", indicating that the table represents an External Data Source. See #external? and External::DataSource.
606 607 608 609 |
# File 'lib/google/cloud/bigquery/table.rb', line 606 def external? return nil if reference? @gapi.type == "EXTERNAL" end |
#extract(extract_url, format: nil, compression: nil, delimiter: nil, header: nil) {|job| ... } ⇒ Boolean
Extracts the data from the table to a Google Cloud Storage file using a synchronous method that blocks for a response. Timeouts and transient errors are generally handled as needed to complete the job. See also #extract_job.
The geographic location for the job ("US", "EU", etc.) can be set via ExtractJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 |
# File 'lib/google/cloud/bigquery/table.rb', line 1467 def extract extract_url, format: nil, compression: nil, delimiter: nil, header: nil, &block job = extract_job extract_url, format: format, compression: compression, delimiter: delimiter, header: header, &block job.wait_until_done! ensure_job_succeeded! job true end |
#extract_job(extract_url, format: nil, compression: nil, delimiter: nil, header: nil, dryrun: nil, job_id: nil, prefix: nil, labels: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::ExtractJob
Extracts the data from the table to a Google Cloud Storage file using an asynchronous method. In this method, an ExtractJob is immediately returned. The caller may poll the service by repeatedly calling Job#reload! and Job#done? to detect when the job is done, or simply block until the job is done by calling #Job#wait_until_done!. See also #extract.
The geographic location for the job ("US", "EU", etc.) can be set via ExtractJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 |
# File 'lib/google/cloud/bigquery/table.rb', line 1388 def extract_job extract_url, format: nil, compression: nil, delimiter: nil, header: nil, dryrun: nil, job_id: nil, prefix: nil, labels: nil ensure_service! = { format: format, compression: compression, delimiter: delimiter, header: header, dryrun: dryrun, job_id: job_id, prefix: prefix, labels: labels } updater = ExtractJob::Updater. service, table_ref, extract_url, updater.location = location if location # may be table reference yield updater if block_given? job_gapi = updater.to_gapi gapi = service.extract_table job_gapi Job.from_gapi gapi, service end |
#fields ⇒ Array<Schema::Field>?
The fields of the table, obtained from its schema.
773 774 775 776 |
# File 'lib/google/cloud/bigquery/table.rb', line 773 def fields return nil if reference? schema.fields end |
#headers ⇒ Array<Symbol>?
The names of the columns in the table, obtained from its schema.
796 797 798 799 |
# File 'lib/google/cloud/bigquery/table.rb', line 796 def headers return nil if reference? schema.headers end |
#id ⇒ String?
The combined Project ID, Dataset ID, and Table ID for this table, in
the format specified by the Legacy SQL Query
Reference:
project_name:datasetId.tableId
. To use this value in queries see
#query_id.
343 344 345 346 |
# File 'lib/google/cloud/bigquery/table.rb', line 343 def id return nil if reference? @gapi.id end |
#insert(rows, insert_ids: nil, skip_invalid: nil, ignore_unknown: nil) ⇒ Google::Cloud::Bigquery::InsertResponse
Inserts data into the table for near-immediate querying, without the need to complete a load operation before the data can appear in query results.
1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 |
# File 'lib/google/cloud/bigquery/table.rb', line 1926 def insert rows, insert_ids: nil, skip_invalid: nil, ignore_unknown: nil rows = [rows] if rows.is_a? Hash insert_ids = Array insert_ids if insert_ids.count > 0 && insert_ids.count != rows.count raise ArgumentError, "insert_ids must be the same size as rows" end rows = [rows] if rows.is_a? Hash raise ArgumentError, "No rows provided" if rows.empty? ensure_service! = { skip_invalid: skip_invalid, ignore_unknown: ignore_unknown, insert_ids: insert_ids } gapi = service.insert_tabledata dataset_id, table_id, rows, InsertResponse.from_gapi rows, gapi end |
#insert_async(skip_invalid: nil, ignore_unknown: nil, max_bytes: 10000000, max_rows: 500, interval: 10, threads: 4) {|response| ... } ⇒ Table::AsyncInserter
Create an asynchronous inserter object used to insert rows in batches.
1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 |
# File 'lib/google/cloud/bigquery/table.rb', line 1989 def insert_async skip_invalid: nil, ignore_unknown: nil, max_bytes: 10000000, max_rows: 500, interval: 10, threads: 4, &block ensure_service! AsyncInserter.new self, skip_invalid: skip_invalid, ignore_unknown: ignore_unknown, max_bytes: max_bytes, max_rows: max_rows, interval: interval, threads: threads, &block end |
#labels ⇒ Hash<String, String>?
A hash of user-provided labels associated with this table. Labels are used to organize and group tables. See Using Labels.
The returned hash is frozen and changes are not allowed. Use #labels= to replace the entire hash.
647 648 649 650 651 652 |
# File 'lib/google/cloud/bigquery/table.rb', line 647 def labels return nil if reference? m = @gapi.labels m = m.to_h if m.respond_to? :to_h m.dup.freeze end |
#labels=(labels) ⇒ Object
Updates the hash of user-provided labels associated with this table. Labels are used to organize and group tables. See Using Labels.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
684 685 686 687 688 |
# File 'lib/google/cloud/bigquery/table.rb', line 684 def labels= labels reload! unless resource_full? @gapi.labels = labels patch_gapi! :labels end |
#load(files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, autodetect: nil, null_marker: nil) {|updater| ... } ⇒ Boolean
Loads data into the table. You can pass a google-cloud storage file path or a google-cloud storage file instance. Or, you can upload a file directly. See Loading Data with a POST Request.
The geographic location for the job ("US", "EU", etc.) can be set via LoadJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 |
# File 'lib/google/cloud/bigquery/table.rb', line 1852 def load files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, autodetect: nil, null_marker: nil, &block job = load_job files, format: format, create: create, write: write, projection_fields: projection_fields, jagged_rows: jagged_rows, quoted_newlines: quoted_newlines, encoding: encoding, delimiter: delimiter, ignore_unknown: ignore_unknown, max_bad_records: max_bad_records, quote: quote, skip_leading: skip_leading, autodetect: autodetect, null_marker: null_marker, &block job.wait_until_done! ensure_job_succeeded! job true end |
#load_job(files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, dryrun: nil, job_id: nil, prefix: nil, labels: nil, autodetect: nil, null_marker: nil) {|load_job| ... } ⇒ Google::Cloud::Bigquery::LoadJob
Loads data into the table. You can pass a google-cloud storage file path or a google-cloud storage file instance. Or, you can upload a file directly. See Loading Data with a POST Request.
The geographic location for the job ("US", "EU", etc.) can be set via LoadJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 |
# File 'lib/google/cloud/bigquery/table.rb', line 1658 def load_job files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, dryrun: nil, job_id: nil, prefix: nil, labels: nil, autodetect: nil, null_marker: nil ensure_service! updater = load_job_updater format: format, create: create, write: write, projection_fields: projection_fields, jagged_rows: jagged_rows, quoted_newlines: quoted_newlines, encoding: encoding, delimiter: delimiter, ignore_unknown: ignore_unknown, max_bad_records: max_bad_records, quote: quote, skip_leading: skip_leading, dryrun: dryrun, job_id: job_id, prefix: prefix, schema: schema, labels: labels, autodetect: autodetect, null_marker: null_marker yield updater if block_given? job_gapi = updater.to_gapi return load_local(files, job_gapi) if local_file? files load_storage files, job_gapi end |
#location ⇒ String?
The geographic location where the table should reside. Possible
values include EU
and US
. The default value is US
.
619 620 621 622 623 |
# File 'lib/google/cloud/bigquery/table.rb', line 619 def location return nil if reference? ensure_full_data! @gapi.location end |
#modified_at ⇒ Time?
The date when this table was last modified.
556 557 558 559 560 561 562 563 564 |
# File 'lib/google/cloud/bigquery/table.rb', line 556 def modified_at return nil if reference? ensure_full_data! begin ::Time.at(Integer(@gapi.last_modified_time) / 1000.0) rescue StandardError nil end end |
#name ⇒ String?
The name of the table.
393 394 395 396 |
# File 'lib/google/cloud/bigquery/table.rb', line 393 def name return nil if reference? @gapi.friendly_name end |
#name=(new_name) ⇒ Object
Updates the name of the table.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
409 410 411 412 413 |
# File 'lib/google/cloud/bigquery/table.rb', line 409 def name= new_name reload! unless resource_full? @gapi.update! friendly_name: new_name patch_gapi! :friendly_name end |
#project_id ⇒ String
The ID of the Project
containing this table.
141 142 143 144 |
# File 'lib/google/cloud/bigquery/table.rb', line 141 def project_id return reference.project_id if reference? @gapi.table_reference.project_id end |
#query ⇒ String
The query that executes each time the view is loaded.
964 965 966 |
# File 'lib/google/cloud/bigquery/table.rb', line 964 def query @gapi.view.query if @gapi.view end |
#query=(new_query) ⇒ Object
Updates the query that executes each time the view is loaded.
This sets the query using standard SQL. To specify legacy SQL or to use user-defined function resources use (#set_query) instead.
991 992 993 |
# File 'lib/google/cloud/bigquery/table.rb', line 991 def query= new_query set_query new_query end |
#query_id(standard_sql: nil, legacy_sql: nil) ⇒ String
The value returned by #id, wrapped in square brackets if the Project ID contains dashes, as specified by the Query Reference. Useful in queries.
377 378 379 380 381 382 383 |
# File 'lib/google/cloud/bigquery/table.rb', line 377 def query_id standard_sql: nil, legacy_sql: nil if Convert.resolve_legacy_sql standard_sql, legacy_sql "[#{project_id}:#{dataset_id}.#{table_id}]" else "`#{project_id}.#{dataset_id}.#{table_id}`" end end |
#query_legacy_sql? ⇒ Boolean
Checks if the view's query is using legacy sql.
1049 1050 1051 1052 1053 |
# File 'lib/google/cloud/bigquery/table.rb', line 1049 def query_legacy_sql? val = @gapi.view.use_legacy_sql return true if val.nil? val end |
#query_standard_sql? ⇒ Boolean
Checks if the view's query is using standard sql.
1062 1063 1064 |
# File 'lib/google/cloud/bigquery/table.rb', line 1062 def query_standard_sql? !query_legacy_sql? end |
#query_udfs ⇒ Array<String>
The user-defined function resources used in the view's query. May be
either a code resource to load from a Google Cloud Storage URI
(gs://bucket/path
), or an inline resource that contains code for a
user-defined function (UDF). Providing an inline code resource is
equivalent to providing a URI for a file containing the same code. See
User-Defined
Functions.
1080 1081 1082 1083 1084 |
# File 'lib/google/cloud/bigquery/table.rb', line 1080 def query_udfs udfs_gapi = @gapi.view.user_defined_function_resources return [] if udfs_gapi.nil? Array(udfs_gapi).map { |udf| udf.inline_code || udf.resource_uri } end |
#reference? ⇒ Boolean
Whether the table was created without retrieving the resource representation from the BigQuery service.
2093 2094 2095 |
# File 'lib/google/cloud/bigquery/table.rb', line 2093 def reference? @gapi.nil? end |
#reload! ⇒ Google::Cloud::Bigquery::Table Also known as: refresh!
Reloads the table with current data from the BigQuery service.
2040 2041 2042 2043 2044 |
# File 'lib/google/cloud/bigquery/table.rb', line 2040 def reload! ensure_service! gapi = service.get_table dataset_id, table_id @gapi = gapi end |
#resource? ⇒ Boolean
Whether the table was created with a resource representation from the BigQuery service.
2116 2117 2118 |
# File 'lib/google/cloud/bigquery/table.rb', line 2116 def resource? !@gapi.nil? end |
#resource_full? ⇒ Boolean
Whether the table was created with a full resource representation from the BigQuery service.
2165 2166 2167 |
# File 'lib/google/cloud/bigquery/table.rb', line 2165 def resource_full? @gapi.is_a? Google::Apis::BigqueryV2::Table end |
#resource_partial? ⇒ Boolean
Whether the table was created with a partial resource representation from the BigQuery service by retrieval through Dataset#tables. See Tables: list response for the contents of the partial representation. Accessing any attribute outside of the partial representation will result in loading the full representation.
2144 2145 2146 |
# File 'lib/google/cloud/bigquery/table.rb', line 2144 def resource_partial? @gapi.is_a? Google::Apis::BigqueryV2::TableList::Table end |
#rows_count ⇒ Integer?
The number of rows in the table.
500 501 502 503 504 505 506 507 508 |
# File 'lib/google/cloud/bigquery/table.rb', line 500 def rows_count return nil if reference? ensure_full_data! begin Integer @gapi.num_rows rescue StandardError nil end end |
#schema(replace: false) {|schema| ... } ⇒ Google::Cloud::Bigquery::Schema?
Returns the table's schema. If the table is not a view (See #view?), this method can also be used to set, replace, or add to the schema by passing a block. See Schema for available methods.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved.
740 741 742 743 744 745 746 747 748 749 750 751 752 753 |
# File 'lib/google/cloud/bigquery/table.rb', line 740 def schema replace: false return nil if reference? && !block_given? reload! unless resource_full? schema_builder = Schema.from_gapi @gapi.schema if block_given? schema_builder = Schema.from_gapi if replace yield schema_builder if schema_builder.changed? @gapi.schema = schema_builder.to_gapi patch_gapi! :schema end end schema_builder.freeze end |
#set_query(query, standard_sql: nil, legacy_sql: nil, udfs: nil) ⇒ Object
Updates the query that executes each time the view is loaded. Allows setting of standard vs. legacy SQL and user-defined function resources.
1033 1034 1035 1036 1037 1038 1039 1040 |
# File 'lib/google/cloud/bigquery/table.rb', line 1033 def set_query query, standard_sql: nil, legacy_sql: nil, udfs: nil @gapi.view = Google::Apis::BigqueryV2::ViewDefinition.new \ query: query, use_legacy_sql: Convert.resolve_legacy_sql(standard_sql, legacy_sql), user_defined_function_resources: udfs_gapi(udfs) patch_gapi! :view end |
#table? ⇒ Boolean?
Checks if the table's type is "TABLE".
575 576 577 578 |
# File 'lib/google/cloud/bigquery/table.rb', line 575 def table? return nil if reference? @gapi.type == "TABLE" end |
#table_id ⇒ String
A unique ID for this table.
116 117 118 119 |
# File 'lib/google/cloud/bigquery/table.rb', line 116 def table_id return reference.table_id if reference? @gapi.table_reference.table_id end |
#time_partitioning? ⇒ Boolean?
Checks if the table is time-partitioned. See Partitioned Tables.
166 167 168 169 |
# File 'lib/google/cloud/bigquery/table.rb', line 166 def time_partitioning? return nil if reference? !@gapi.time_partitioning.nil? end |
#time_partitioning_expiration ⇒ Integer?
The expiration for the table partitions, if any, in seconds. See Partitioned Tables.
289 290 291 292 293 294 295 |
# File 'lib/google/cloud/bigquery/table.rb', line 289 def time_partitioning_expiration return nil if reference? ensure_full_data! @gapi.time_partitioning.expiration_ms / 1_000 if time_partitioning? && !@gapi.time_partitioning.expiration_ms.nil? end |
#time_partitioning_expiration=(expiration) ⇒ Object
Sets the partition expiration for the table. See Partitioned Tables. The table must also be partitioned.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
323 324 325 326 327 328 329 |
# File 'lib/google/cloud/bigquery/table.rb', line 323 def time_partitioning_expiration= expiration reload! unless resource_full? @gapi.time_partitioning ||= \ Google::Apis::BigqueryV2::TimePartitioning.new @gapi.time_partitioning.expiration_ms = expiration * 1000 patch_gapi! :time_partitioning end |
#time_partitioning_field ⇒ String?
The field on which the table is partitioned, if any. See Partitioned Tables.
231 232 233 234 235 |
# File 'lib/google/cloud/bigquery/table.rb', line 231 def time_partitioning_field return nil if reference? ensure_full_data! @gapi.time_partitioning.field if time_partitioning? end |
#time_partitioning_field=(field) ⇒ Object
Sets the field on which to partition the table. See Partitioned Tables. The table must also be partitioned.
You can only set the partitioning field while creating a table as in the example below. BigQuery does not allow you to change partitioning on an existing table.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
271 272 273 274 275 276 277 |
# File 'lib/google/cloud/bigquery/table.rb', line 271 def time_partitioning_field= field reload! unless resource_full? @gapi.time_partitioning ||= \ Google::Apis::BigqueryV2::TimePartitioning.new @gapi.time_partitioning.field = field patch_gapi! :time_partitioning end |
#time_partitioning_type ⇒ String?
The period for which the table is partitioned, if any. See Partitioned Tables.
181 182 183 184 185 |
# File 'lib/google/cloud/bigquery/table.rb', line 181 def time_partitioning_type return nil if reference? ensure_full_data! @gapi.time_partitioning.type if time_partitioning? end |
#time_partitioning_type=(type) ⇒ Object
Sets the partitioning for the table. See Partitioned Tables.
You can only set partitioning when creating a table as in the example below. BigQuery does not allow you to change partitioning on an existing table.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
213 214 215 216 217 218 219 |
# File 'lib/google/cloud/bigquery/table.rb', line 213 def time_partitioning_type= type reload! unless resource_full? @gapi.time_partitioning ||= \ Google::Apis::BigqueryV2::TimePartitioning.new @gapi.time_partitioning.type = type patch_gapi! :time_partitioning end |
#view? ⇒ Boolean?
Checks if the table's type is "VIEW", indicating that the table represents a BigQuery view. See Dataset#create_view.
590 591 592 593 |
# File 'lib/google/cloud/bigquery/table.rb', line 590 def view? return nil if reference? @gapi.type == "VIEW" end |