[
https://issues.apache.org/jira/browse/BEAM-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897371#comment-16897371
]
Rahul Patwari edited comment on BEAM-6114 at 8/1/19 5:43 PM:
-------------------------------------------------------------
[~amaliujia]
I will add 3 [converter
rules|https://github.com/apache/calcite/blob/d98856bf1a5f5c151d004b769e14bdd368a67234/core/src/main/java/org/apache/calcite/rel/convert/ConverterRule.java#L75]
with predicates to identify the type of Join.
The predicate for SideInputJoin: One Bounded Input, One Unbounded Input
The predicate for SideInputLookupJoin: One Seekable Input, One Non-Seekable
Input
The predicate for StandardJoin: Both Bounded Inputs (or) Both Unbounded Inputs
These three Predicates will cover all the possible Joins, right?
We have to add cost for each Rel so that the least cost Rel is chosen when more
than one rules match. For Example [One Bounded Seekable Input, One Unbounded
Input] match both the Predicates for SideInputJoin and SideInputLookupJoin. So,
by providing costs in increasing order for SideInputLookupJoin, SideInputJoin,
StandardJoin SideInputLookupJoin will be used for the above case.
For the Unsupported Join operations such as FULL OUTER Join for SideInputJoin,
an UnsupportedOperationException will be thrown in BeamSideInputJoinRel.java
I will add _isView_ as part of BEAM-7835
was (Author: rahul8383):
[~amaliujia]
I will add 3 [converter
rules|https://github.com/apache/calcite/blob/d98856bf1a5f5c151d004b769e14bdd368a67234/core/src/main/java/org/apache/calcite/rel/convert/ConverterRule.java#L75]
with predicates to identify the type of Join.
The predicate for SideInputJoin: One Bounded Input, One Unbounded Input
The predicate for SideInputLookupJoin: One Seekable Input Index, One
Non-Seekable Input Index
The predicate for CoGBKJoin: Both Bounded Inputs (or) Both Unbounded Inputs
These three Predicates will cover all the possible Joins, right?
For the Unsupported Join operations such as FULL OUTER Join for SideInputJoin,
an UnsupportedOperationException will be thrown in BeamSideInputJoinRel.java
I will add _isView_ as part of BEAM-7835
> SQL join selection should be done in planner, not in expansion to PTransform
> ----------------------------------------------------------------------------
>
> Key: BEAM-6114
> URL: https://issues.apache.org/jira/browse/BEAM-6114
> Project: Beam
> Issue Type: Improvement
> Components: dsl-sql
> Reporter: Kenneth Knowles
> Assignee: Rahul Patwari
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently Beam SQL joins all go through a single physical operator which has
> a single PTransform that does all join algorithms based on properties of its
> input PCollections as well as the relational algebra.
> A first step is to make the needed information part of the relational
> algebra, so it can choose a PTransform based on that, and the PTransforms can
> be simpler.
> Second step is to have separate (physical) relational operators for different
> join algorithms.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
