[
https://issues.apache.org/jira/browse/FLINK-36715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17898515#comment-17898515
]
Sergey Nuyanzin commented on FLINK-36715:
-----------------------------------------
The main issue is that after such optimization plan for some queries changed
e.g. for
{code:sql}
SELECT * FROM
(SELECT * FROM src WHERE k = 0) src1
LEFT OUTER JOIN
(SELECT * from src WHERE k = 0) src2
ON (src1.k = src2.k AND src2.k > 10)
{code}
before the change
{code}
LogicalProject(k=[$0], v=[$1], k0=[$2], v0=[$3])
+- LogicalJoin(condition=[AND(=($0, $2), >($2, 10))], joinType=[left])
:- LogicalProject(k=[$0], v=[$1])
: +- LogicalFilter(condition=[=($0, 0)])
: +- LogicalTableScan(table=[[default_catalog, default_database, src,
source: [TestTableSource(k, v)]]])
+- LogicalProject(k=[$0], v=[$1])
+- LogicalFilter(condition=[=($0, 0)])
+- LogicalTableScan(table=[[default_catalog, default_database, src,
source: [TestTableSource(k, v)]]])
{code}
after the change
{code}
LogicalProject(exprs=[[CAST(0:BIGINT):BIGINT, $0, $1, $2]])
+- LogicalJoin(condition=[true], joinType=[left])
:- LogicalProject(exprs=[[$1]])
: +- LogicalFilter(condition=[=($0, 0)])
: +- LogicalTableScan(table=[[default_catalog, default_database, src,
source: [TestTableSource(k, v)]]])
+- LogicalValues(type=[RecordType(BIGINT k, VARCHAR(2147483647) v)],
tuples=[[]])
{code}
So from one side it was able to detect that from one side {{k}} should be 0 and
could be moved to projection and {{src2.k > 10}} could be pushed down.
>From the other side join condition became {{true}} and some joins stopped
>working e.g. SortMergeJoin since they expect {{equi}} join condition which is
>now not present here...
> Changed behavior for filter push down for IS NOT DISTINCT FROM after Calcite
> upgrade to 1.33
> ---------------------------------------------------------------------------------------------
>
> Key: FLINK-36715
> URL: https://issues.apache.org/jira/browse/FLINK-36715
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API, Table SQL / Planner
> Reporter: Sergey Nuyanzin
> Priority: Major
>
> After moving to 1.33 need to put RexUtil into Filnk repo to prevent a number
> of failures (more than 30)
> like
> {noformat}
> FlinkLogicalCalc(select=[CAST(0:BIGINT AS BIGINT) AS k, v, k AS k0, v0])
> +- FlinkLogicalJoin(condition=[true], joinType=[left])
> :- FlinkLogicalCalc(select=[v], where=[=(k, 0:BIGINT)])
> : +- FlinkLogicalLegacyTableSourceScan(table=[[default_catalog,
> default_database, src, source: [TestTableSource(k, v)]]], fields=[k, v])
> +- FlinkLogicalValues(type=[RecordType(BIGINT k, VARCHAR(2147483647) v)],
> tuples=[[]])
> This exception indicates that the query uses an unsupported SQL feature.
> Please check the documentation for the set of currently supported SQL
> features.
> {noformat}
> This is a result of https://issues.apache.org/jira/browse/CALCITE-5336
> the jira issue is to adapt Flink behavior and to remove RexUtil after that
> from Flink repo
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
