[
https://issues.apache.org/jira/browse/BEAM-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376279#comment-16376279
]
Kenneth Knowles commented on BEAM-3634:
---------------------------------------
The basic refactor was OK, but there is an issue with it: When you have an
expression like JOIN(A, B) you might actually want the composite structure to
look like:
A -> JOIN
B -> JOIN
All inside whatever the parent context is.
The refactor will actually cause a JOIN(A, B) to look like:
JOIN/A -> JOIN/ActualJoin
JOIN/B -> JOIN/ActualJoin
(if that makes sense).
It is essentially the issue where nested expressions should look like
upstream/downstream transforms, not nested transforms.
> [SQL] Refactor BeamRelNodes into PTransforms
> --------------------------------------------
>
> Key: BEAM-3634
> URL: https://issues.apache.org/jira/browse/BEAM-3634
> Project: Beam
> Issue Type: Improvement
> Components: dsl-sql
> Reporter: Anton Kedin
> Assignee: Kenneth Knowles
> Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> BeamRelNode exposes PCollection<BeamRecord> buildBeamPipeline() which builds
> a pipeline when parsing.
> It feels like it should instead implement a
> PTransform<PCollection<BeamRecord>, PCollection<BeamRecord>> which would
> receive a prepared PCollection, and apply sub-expressions instead of manually
> invoking expression evaluation to get the input.
> And maybe consider building it lazily.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
