TheNeuralBit commented on a change in pull request #12355:
URL: https://github.com/apache/beam/pull/12355#discussion_r459785878
##########
File path: sdks/python/apache_beam/examples/wordcount_xlang_sql.py
##########
@@ -101,12 +89,33 @@ def main():
# workflow rely on global context (e.g., a module imported at module level).
pipeline_options.view_as(SetupOptions).save_main_session = True
- p = beam.Pipeline(options=pipeline_options)
- # Preemptively start due to BEAM-6666.
- p.runner.create_job_service(pipeline_options)
+ with beam.Pipeline(options=pipeline_options) as p:
+ if isinstance(p.runner, portable_runner.PortableRunner):
+ # Preemptively start due to BEAM-6666.
+ p.runner.create_job_service(pipeline_options)
+
+ run(p, known_args.input, known_args.output)
- run(p, known_args.input, known_args.output)
+# Some more fun queries:
+# ------
+# SELECT
+# word as key,
+# COUNT(*) as `count`
+# FROM PCOLLECTION
+# GROUP BY word
Review comment:
Good idea. I ran all of them on FnApiRunner and Dataflow on 2.23.0-RC2
(with the job service line commented out) and they worked as intended :tada:
Job ids for the Dataflow runs:
- 2020-07-23_16_00_29-9161755024487428146
- 2020-07-23_16_11_11-9635287738762184253
- 2020-07-23_16_19_49-9899271126466068112
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
