[
https://issues.apache.org/jira/browse/BEAM-838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Weise updated BEAM-838:
------------------------------
Description: The Apex runner should leverage the partitioning features of
Apex for Beam pipeline translation (parallel partitions, dynamic partitioning
and cascading unifiers etc.). Specifically for ParDo and GroupByKey (GBK
partitioning will also need the appropriate stream codec that will partition
the stream based on the encoded key). Enabling the partitioning will also
require watermarks to be broadcasted on a separate Apex operator port. (was:
The Apex runner should leverage the partitioning features of Apex for Beam
pipeline translation (parallel partitions, dynamic partitioning and cascading
unifiers etc.). Specifically for ParDo and GroupByKey (GBK partitioning will
also need the appropriate stream codec that will partition the stream based on
the encoded key). )
> Support parallelism in Apex runner
> ----------------------------------
>
> Key: BEAM-838
> URL: https://issues.apache.org/jira/browse/BEAM-838
> Project: Beam
> Issue Type: Improvement
> Components: runner-apex
> Reporter: Thomas Weise
>
> The Apex runner should leverage the partitioning features of Apex for Beam
> pipeline translation (parallel partitions, dynamic partitioning and cascading
> unifiers etc.). Specifically for ParDo and GroupByKey (GBK partitioning will
> also need the appropriate stream codec that will partition the stream based
> on the encoded key). Enabling the partitioning will also require watermarks
> to be broadcasted on a separate Apex operator port.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
