[
https://issues.apache.org/jira/browse/BEAM-9787?focusedWorklogId=425426&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-425426
]
ASF GitHub Bot logged work on BEAM-9787:
----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Apr/20 19:20
Start Date: 20/Apr/20 19:20
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#11466:
URL: https://github.com/apache/beam/pull/11466#discussion_r411627605
##########
File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner.py
##########
@@ -91,6 +91,12 @@
_LOGGER = logging.getLogger(__name__)
+BQ_SOURCE_UW_ERROR = (
+ 'The Read(BigQuerySource(...)) transform is not supported with newer stack
'
+ 'features (Fn API, Unified Worker, etc). Please use the transform '
Review comment:
s/Unified Worker/Dataflow runner V2 I believe.
cc: @robertwb @ananvay for the correct wording here.
##########
File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner.py
##########
@@ -1182,6 +1188,11 @@ def run_Read(self, transform_node, options):
raise ValueError(
'BigQuery source is not currently available for use '
'in streaming pipelines.')
+ debug_options = options.view_as(DebugOptions)
+ use_fn_api = (debug_options.experiments and
+ 'beam_fn_api' in debug_options.experiments)
+ if use_fn_api:
Review comment:
Should we also check for experiment use_runner_v2 or is this adequate ?
##########
File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py
##########
@@ -258,6 +259,18 @@ def test_biqquery_read_streaming_fail(self):
PipelineOptions(self.default_properties)) as p:
_ = p | beam.io.Read(beam.io.BigQuerySource('some.table'))
+ def test_biqquery_read_uw_fail(self):
Review comment:
fnapi_fail (to be consistent with the above check).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 425426)
Time Spent: 0.5h (was: 20m)
> Send clear error to users trying to use BigQuerySource on FnApi pipelines on
> Python SDK
> ---------------------------------------------------------------------------------------
>
> Key: BEAM-9787
> URL: https://issues.apache.org/jira/browse/BEAM-9787
> Project: Beam
> Issue Type: Bug
> Components: io-py-gcp
> Reporter: Pablo Estrada
> Assignee: Pablo Estrada
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
