[
https://issues.apache.org/jira/browse/BEAM-12853?focusedWorklogId=647550&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647550
]
ASF GitHub Bot logged work on BEAM-12853:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Sep/21 20:00
Start Date: 07/Sep/21 20:00
Worklog Time Spent: 10m
Work Description: apilloud edited a comment on pull request #15469:
URL: https://github.com/apache/beam/pull/15469#issuecomment-914584242
Before the calcite upgrade this plan was produced:
```
BeamIOSinkRel(table=[[beam, message]], operation=[INSERT],
flattened=[true])
LogicalValues(tuples=[[{ 1970-01-01 00:00:00.001, 'person1', 80, true
}, { 1970-01-01 00:00:00.002, 'person2', 70, false }]])
```
After the calcite upgrade this plan was produced:
```
BeamIOSinkRel(table=[[beam, message]], operation=[INSERT],
flattened=[true])
LogicalUnion(all=[true])
LogicalProject(EXPR$0=[1970-01-01 00:00:00.001:TIMESTAMP(6)],
EXPR$1=['person1'], EXPR$2=[80], EXPR$3=[true])
LogicalValues(tuples=[[{ 0 }]])
LogicalProject(EXPR$0=[1970-01-01 00:00:00.002:TIMESTAMP(6)],
EXPR$1=['person2'], EXPR$2=[70], EXPR$3=[false])
LogicalValues(tuples=[[{ 0 }]])
```
The LogicalUnion gets turned into BeamUnionRel, which eventually becomes a
CoGroup. Calcite could probably produce a simpler plan (we'd need to write or
enable some more optimizer rules) but PubSubTableProvider is also broken here
(it shouldn't depend on the pipeline not containing a CoGroup).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 647550)
Time Spent: 40m (was: 0.5h)
> PubSubTableProvider tries to set windows after GroupByKey
> ---------------------------------------------------------
>
> Key: BEAM-12853
> URL: https://issues.apache.org/jira/browse/BEAM-12853
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql
> Reporter: Andrew Pilloud
> Assignee: Andrew Pilloud
> Priority: P2
> Time Spent: 40m
> Remaining Estimate: 0h
>
> A SqlTransform may contain a GroupByKey in the global window, this prevents
> setting windows afterwards. The PubSubTableProvider tries anyway.
> This was exposed by a INSERT INTO pubsub VALUES (1), (2), which implicitly
> becomes INSERT INTO pubsub SELECT 1 UNION SELECT 2. This then becomes a
> GroupByKey union.
> For now I'm going to simplify the test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
