[
https://issues.apache.org/jira/browse/BEAM-12853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17411454#comment-17411454
]
Andrew Pilloud commented on BEAM-12853:
---------------------------------------
Before the calcite upgrade this plan was produced:
```
BeamIOSinkRel(table=[[beam, message]], operation=[INSERT], flattened=[true])
LogicalValues(tuples=[[{ 1970-01-01 00:00:00.001, 'person1', 80, true },
{ 1970-01-01 00:00:00.002, 'person2', 70, false }]])```
After the calcite upgrade this plan was produced:
```
BeamIOSinkRel(table=[[beam, message]], operation=[INSERT], flattened=[true])
LogicalUnion(all=[true])
LogicalProject(EXPR$0=[1970-01-01 00:00:00.001:TIMESTAMP(6)],
EXPR$1=['person1'], EXPR$2=[80], EXPR$3=[true])
LogicalValues(tuples=[[{ 0 }]])
LogicalProject(EXPR$0=[1970-01-01 00:00:00.002:TIMESTAMP(6)],
EXPR$1=['person2'], EXPR$2=[70], EXPR$3=[false])
LogicalValues(tuples=[[{ 0 }]])
```
The LogicalUnion gets turned into BeamUnionRel, which eventually becomes a
CoGroup. Calcite could probably produce a simpler plan (we'd need to write or
enable some more optimizer rules) but PubSubTableProvider is also broken here
(it shouldn't depend on the pipeline not containing a CoGroup). I would expect
#15469 to be reverted when the underlying (preexisting) issue with
PubSubTableProvider is fixed.
> PubSubTableProvider tries to set windows after GroupByKey
> ---------------------------------------------------------
>
> Key: BEAM-12853
> URL: https://issues.apache.org/jira/browse/BEAM-12853
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql
> Reporter: Andrew Pilloud
> Assignee: Andrew Pilloud
> Priority: P2
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> A SqlTransform may contain a GroupByKey in the global window, this prevents
> setting windows afterwards. The PubSubTableProvider tries anyway.
> This was exposed by a INSERT INTO pubsub VALUES (1), (2), which implicitly
> becomes INSERT INTO pubsub SELECT 1 UNION SELECT 2. This then becomes a
> GroupByKey union.
> For now I'm going to simplify the test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
