[
https://issues.apache.org/jira/browse/BEAM-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090888#comment-17090888
]
Luke Cwik commented on BEAM-7199:
---------------------------------
The optimizations that Python does should really be moved out of the Python SDK
and implemented on the runner/job server side.
This was done for splittable DoFns but other optimizations such as combiner
lifting are still needed.
> Better optimize Portable pipelines
> ----------------------------------
>
> Key: BEAM-7199
> URL: https://issues.apache.org/jira/browse/BEAM-7199
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Reporter: Ankur Goenka
> Priority: Major
> Labels: portability
>
> Python has an experimental flag pre_optimize=all which does pre-optimization
> of python pipelines by fusing operators.
> Python optimization is expected to be better than the one in java because it
> has more information about the pipeline.
> Make java pipeline optimization at par with python so that the benefits can
> be shared by all languages.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
