[
https://issues.apache.org/jira/browse/BEAM-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16272540#comment-16272540
]
ASF GitHub Bot commented on BEAM-3171:
--------------------------------------
xumingming commented on a change in pull request #4196: [BEAM-3171] convert a
join into lookup
URL: https://github.com/apache/beam/pull/4196#discussion_r154046003
##########
File path:
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamJoinRel.java
##########
@@ -98,9 +101,14 @@ public BeamJoinRel(RelOptCluster cluster, RelTraitSet
traits, RelNode left, RelN
throws Exception {
BeamRelNode leftRelNode = BeamSqlRelUtils.getBeamRelInput(left);
BeamRecordSqlType leftRowType =
CalciteUtils.toBeamRowType(left.getRowType());
- PCollection<BeamRecord> leftRows =
leftRelNode.buildBeamPipeline(inputPCollections, sqlEnv);
-
final BeamRelNode rightRelNode = BeamSqlRelUtils.getBeamRelInput(right);
+
+ if (!seekable(leftRelNode, sqlEnv) && seekable(rightRelNode, sqlEnv)) {
Review comment:
if left side is seekable and right side is not seekable, can we also do a
lookup join?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> convert a join into lookup
> --------------------------
>
> Key: BEAM-3171
> URL: https://issues.apache.org/jira/browse/BEAM-3171
> Project: Beam
> Issue Type: New Feature
> Components: dsl-sql
> Reporter: Xu Mingmin
> Assignee: Xu Mingmin
> Labels: experimental
>
> We use BeamSQL to run streaming jobs mostly, and add a join_as_lookup
> improvement(internal branch) to cover the streaming-to-batch case(similar as
> [1]). I could submit a PR as experimental if people are interested.
> The rough solution is, if one source of join node implements
> {{BeamSeekableTable}} and the other is not, then the join node is converted
> to a fact-lookup operation.
> Ref:
> [1]
> https://docs.google.com/document/d/1B-XnUwXh64lbswRieckU0BxtygSV58hysqZbpZmk03A/edit?usp=sharing
>
> [~xumingming] [~takidau] for any comments
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
