Jeff Klukas created BEAM-8023:
---------------------------------
Summary: Allow specifying BigQuery Storage API readOptions at
runtime
Key: BEAM-8023
URL: https://issues.apache.org/jira/browse/BEAM-8023
Project: Beam
Issue Type: Improvement
Components: io-java-gcp
Reporter: Jeff Klukas
We have support in the Java SDK for using the BigQuery Storage API for reads,
but only the target query or table is supported as a ValueProvider to be
specified at runtime. AFAICT, there is no reason we can't delay specifying
readOptions until runtime as well.
The readOptions are accessed by BigQueryStorageTableSource in getTargetTable; I
believe that's occurring at runtime, but I'd love for someone with deeper
BoundedSource knowledge to confirm that.
I'd advocate for adding new methods
`TypedRead.withSelectedFields(ValueProvider<List<String>> value)` and
`TypedRead.withRowRestriction(ValueProvider<String> value)`. The existing
`withReadOptions` method would then populate the other two as
StaticValueProviders. Perhaps we'd want to deprecate `withReadOptions` in favor
or specifying individual read options as separate parameters.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
