Sergei Lilichenko created BEAM-13612:
----------------------------------------
Summary: When BigqueryIO's STORAGE_API_AT_LEAST_ONCE method is
used and the table spec doesn't contain the project id the pipeline fails
Key: BEAM-13612
URL: https://issues.apache.org/jira/browse/BEAM-13612
Project: Beam
Issue Type: Bug
Components: io-java-gcp
Affects Versions: 2.35.0
Reporter: Sergei Lilichenko
The validation method correctly uses BigqueryIO's
getTableWithDefaultProject(BigQueryOptions bqOptions), but at the run time the
table spec is not checked for absence of the project id, resulting in
RuntimeException.
Stack trace:
Caused by: java.lang.NullPointerException: Required parameter projectId must be
specified.
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:897)
at
com.google.api.client.util.Preconditions.checkNotNull(Preconditions.java:138)
at
com.google.api.services.bigquery.Bigquery$Tables$Get.<init>(Bigquery.java:5325)
at com.google.api.services.bigquery.Bigquery$Tables.get(Bigquery.java:5298)
at
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.getTable(BigQueryServicesImpl.java:553)
at
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.getTable(BigQueryServicesImpl.java:542)
at
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.getTable(BigQueryServicesImpl.java:536)
at
org.apache.beam.sdk.io.gcp.bigquery.StorageApiDynamicDestinationsTableRow$1.lambda$$0(StorageApiDynamicDestinationsTableRow.java:66)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4876)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
... 35 more
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
