[
https://issues.apache.org/jira/browse/BEAM-9305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050415#comment-17050415
]
Pablo Estrada commented on BEAM-9305:
-------------------------------------
[~eliasdjur] that is the right approach, but you want to call `get` on the
valueprovider at pipeline runtime. The constructor of the source runs at
pipeline construction time, when we dont yet know the value from the value
provider. The query is actually executed in `split`, which runs at pipeline
runtime:
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/bigquery.py#L658-L680
So you'd have to make that change inside the split method.
I hope my explanation made sense. I can clarify if necessary : )
> Support ValueProvider for BigQuerySource query string
> -----------------------------------------------------
>
> Key: BEAM-9305
> URL: https://issues.apache.org/jira/browse/BEAM-9305
> Project: Beam
> Issue Type: New Feature
> Components: io-py-gcp
> Reporter: Elias Djurfeldt
> Assignee: Elias Djurfeldt
> Priority: Minor
>
> Users should be able to use ValueProviders for the query string in
> BigQuerySource.
> Ref:
> [https://stackoverflow.com/questions/60146887/expected-eta-to-avail-pipeline-i-o-and-runtime-parameters-in-apache-beam-gcp-dat/60170614?noredirect=1#comment106464448_60170614]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
