[
https://issues.apache.org/jira/browse/BEAM-7049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16912980#comment-16912980
]
Rui Wang commented on BEAM-7049:
--------------------------------
Although usually most of the use case we might only need <5 UNIONs, it still
helpful to have a general n implementation.
Does this implementation help:
1. KeyedPCollectionTuple.of(), and then for (i -> 1, n)
{KeyedPCollectionTuple.and(current i);}
2. SetOperatorFilteringDoFn(List<Tag)).
3. With BeamSetOperatorsTransforms you will have a List<Tag> so you pretty much
will have a n loop to finish "union" operation.
> Merge multiple input to one BeamUnionRel
> ----------------------------------------
>
> Key: BEAM-7049
> URL: https://issues.apache.org/jira/browse/BEAM-7049
> Project: Beam
> Issue Type: Improvement
> Components: dsl-sql
> Reporter: Rui Wang
> Assignee: sridhar Reddy
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> BeamUnionRel assumes inputs are two and rejects more. So `a UNION b UNION c`
> will have to be created as UNION(a, UNION(b, c)) and have two shuffles. If
> BeamUnionRel can handle multiple shuffles, we will have only one shuffle
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
