[
https://issues.apache.org/jira/browse/BEAM-10559?focusedWorklogId=462746&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-462746
]
ASF GitHub Bot logged work on BEAM-10559:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 23/Jul/20 23:39
Start Date: 23/Jul/20 23:39
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on a change in pull request
#12355:
URL: https://github.com/apache/beam/pull/12355#discussion_r459785878
##########
File path: sdks/python/apache_beam/examples/wordcount_xlang_sql.py
##########
@@ -101,12 +89,33 @@ def main():
# workflow rely on global context (e.g., a module imported at module level).
pipeline_options.view_as(SetupOptions).save_main_session = True
- p = beam.Pipeline(options=pipeline_options)
- # Preemptively start due to BEAM-6666.
- p.runner.create_job_service(pipeline_options)
+ with beam.Pipeline(options=pipeline_options) as p:
+ if isinstance(p.runner, portable_runner.PortableRunner):
+ # Preemptively start due to BEAM-6666.
+ p.runner.create_job_service(pipeline_options)
+
+ run(p, known_args.input, known_args.output)
- run(p, known_args.input, known_args.output)
+# Some more fun queries:
+# ------
+# SELECT
+# word as key,
+# COUNT(*) as `count`
+# FROM PCOLLECTION
+# GROUP BY word
Review comment:
Good idea. I ran all of them on FnApiRunner and Dataflow on 2.23.0-RC2
(with the job service line commented out) and they worked as intended :tada:
Job ids for the Dataflow runs:
- 2020-07-23_16_00_29-9161755024487428146
- 2020-07-23_16_11_11-9635287738762184253
- 2020-07-23_16_19_49-9899271126466068112
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 462746)
Time Spent: 40m (was: 0.5h)
> Python SqlTransform examples
> ----------------------------
>
> Key: BEAM-10559
> URL: https://issues.apache.org/jira/browse/BEAM-10559
> Project: Beam
> Issue Type: Improvement
> Components: dsl-sql, sdk-py-core
> Reporter: Brian Hulette
> Assignee: Brian Hulette
> Priority: P2
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Add an example like
> https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/kafkataxi
> for SqlTransform use-cases. Perhaps one word-count, one streaming taxi data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
