[
https://issues.apache.org/jira/browse/CALCITE-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18095453#comment-18095453
]
Mihai Budiu commented on CALCITE-7608:
--------------------------------------
Any chance we can resume the conversation about this issue?
In the implementation I have adopted the approach suggested by [~julianhyde] :
add new capabilities to the Uncollect operator, and do not extend the set of
relational operators. There are 3 new capabilities:
* support for LEFT JOIN
* support for Trino's semantics when expanding collections of structs
* ability to carry over other input fields unchanged to the output
The first two capabilities were missing from the old Uncollect implementation,
and hampered its ability to properly support some legitimate queries.
The last capability strictly makes Uncollect more powerful, and is a
backwards-compatible change: if the set of input fields carried over is empty,
this operator behaves like classic Uncollect. SqlToRelConverter only
synthesizes such restricted forms of Uncollect.
However, when the set of carried over fields is non-empty, this enables
Uncollect to express by itself sub-plans which previously required a Correlate
node, which could not be decorrelated by either of the two decorrelators that
are in Calcite. There are new optimization rules which convert Correlate +
Uncollect to this new form of Uncollect.
> Enhance Uncollect
> ------------------
>
> Key: CALCITE-7608
> URL: https://issues.apache.org/jira/browse/CALCITE-7608
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.42.0
> Reporter: Mihai Budiu
> Assignee: Mihai Budiu
> Priority: Minor
> Labels: pull-request-available
>
> Today SQL UNNEST is implemented using the Uncollect operator. We propose
> adding two additional capabilities to Uncollect, which generalize its current
> behavior:
> * support for LEFT JOIN UNNEST
> * support for carrying over non-collection input fields to the output
> unchanged
> The first capability is needed because the semantics of UNCOLLECT on the
> right-hand-side of a LEFT JOIN cannot be implemented using the existing
> semantics: UNCOLLECT will need to produce a NULL for empty or NULL
> collections in such cases.
> The second capability is independent on the first one: it would strictly
> increase the expressive power of Uncollect, allowing it to not only Unnest
> collections, but also copy some input fields unchanged for every element of
> an unnested collection.
> The main reason for the second capability is that it would enable us to
> represent plans using SQL UNNEST without using Correlate nodes. A sub-plan
> containing Project + Correlate + Uncollect + Values would become a single
> Uncollect node. Neither the old nor the new decorrelator can actually
> eliminate Correlate + Uncollect (there is no other representation to express
> the same behavior). Using the new representation we can decorrelate many more
> plans (perhaps all plans).
> Although these two changes are logically independent, I propose to make them
> in a single combined PR, because they would both change the type checking and
> runtime implementation of the operator, and it is desirable to minimize the
> churn on consumers of the new operator.
> A temporary configuration flag would be introduced to control whether
> SqlToRelConverter uses the new form of the operator, or just the old form.
> Hopefully programs that only use the old form would remain unchanged. This
> will give Calcite users time to upgrade to the new representation at their
> own pace.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
