[jira] [Commented] (FLINK-29873) CannotPlanException raised after "CROSS JOIN UNNEST"
[
https://issues.apache.org/jira/browse/FLINK-29873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630173#comment-17630173
]
Daeyeon Bae commented on FLINK-29873:
-
If it works well on the master branch, does it mean that the 1.15.2 version
doesn't also throw an error?
> CannotPlanException raised after "CROSS JOIN UNNEST"
>
>
> Key: FLINK-29873
> URL: https://issues.apache.org/jira/browse/FLINK-29873
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.15.2
>Reporter: Daeyeon Bae
>Priority: Major
> Attachments: error-log.txt
>
>
> When I create a VIEW as a result of "CROSS JOIN UNNEST" and then use the
> condition in the WHERE clause of the VIEW, it throws an exception
> "org.apache.calcite.plan.RelOptPlanner$CannotPlanException".
> Why am I getting this exception and how should I handle it the right way?
> The following is the test code in which an error occurs.
> {code:java}
> it should "filter with object_key" in {
> tEnv.executeSql(
> s"""CREATE TABLE s3_put_event (
> | Records ARRAY<
> |ROW<
> | s3 ROW<
> |bucket ROW,
> |object ROW
> | >
> |>
> | >
> |) WITH (
> | 'connector' = 'datagen',
> | 'number-of-rows' = '3',
> | 'rows-per-second' = '1',
> | 'fields.Records.element.s3.bucket.name.length' = '8',
> | 'fields.Records.element.s3.object.key.length' = '15',
> | 'fields.Records.element.s3.object.size.min' = '1',
> | 'fields.Records.element.s3.object.size.max' = '1000'
> |)
> |""".stripMargin
> )
> tEnv.executeSql(
> s"""CREATE TEMPORARY VIEW s3_objects AS
> |SELECT object_key, bucket_name
> |FROM (
> | SELECT
> |r.s3.bucket.name AS bucket_name,
> |r.s3.object.key AS object_key,
> |r.s3.object.size AS object_size
> | FROM s3_put_event
> | CROSS JOIN UNNEST(s3_put_event.Records) AS r(s3)
> |) rs
> |WHERE object_size > 0
> |""".stripMargin
> )
> tEnv.executeSql(
> s"""CREATE TEMPORARY VIEW filtered_s3_objects AS
> |SELECT bucket_name, object_key
> |FROM s3_objects
> |WHERE object_key > ''
> |""".stripMargin)
> val result = tEnv.sqlQuery("SELECT * FROM filtered_s3_objects")
> tEnv.toChangelogStream(result).print()
> env.execute()
> } {code}
> If I remove the condition object_key > '' in the "filtered_s3_objects" VIEW,
> and do it in the "s3_objects" VIEW, no exception is thrown.However, my actual
> query is complicated, so it is not easy to move the condition of the WHERE
> clause like this. It's hard to use especially if I need to separate the
> output stream.
> I have attached the error log.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-29873) CannotPlanException raised after "CROSS JOIN UNNEST"
[
https://issues.apache.org/jira/browse/FLINK-29873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17628630#comment-17628630
]
luoyuxia commented on FLINK-29873:
--
I run the sql statements you post in the master branch, it won't throw any
exception. The plan can be generated normally:
{code:java}
LogicalSink(table=[*anonymous_datastream_sink$1*], fields=[bucket_name,
object_key])
LogicalProject(bucket_name=[$1], object_key=[$0])
LogicalFilter(condition=[>($0, _UTF-16LE'')])
LogicalProject(object_key=[$1], bucket_name=[$0])
LogicalFilter(condition=[>($2, 0)])
LogicalProject(bucket_name=[$1.bucket.name],
object_key=[$1.object.key], object_size=[$1.object.size])
LogicalCorrelate(correlation=[$cor2], joinType=[inner],
requiredColumns=[{0}])
LogicalTableScan(table=[[default_catalog, default_database,
s3_put_event]])
Uncollect
LogicalProject(Records=[$cor2.Records])
LogicalValues(tuples=[[{ 0
}]])StreamPhysicalSink(table=[*anonymous_datastream_sink$1*],
fields=[bucket_name, object_key])
StreamPhysicalCalc(select=[s3.bucket.name AS bucket_name, s3.object.key AS
object_key])
StreamPhysicalCorrelate(invocation=[$UNNEST_ROWS$1($cor2.Records)],
correlate=[table($UNNEST_ROWS$1($cor2.Records))], select=[Records,s3],
rowType=[RecordType(RecordType:peek_no_expand(RecordType:peek_no_expand(RecordType:peek_no_expand(VARCHAR(2147483647)
name) bucket, RecordType:peek_no_expand(VARCHAR(2147483647) key, BIGINT size)
object) s3) ARRAY Records,
RecordType:peek_no_expand(RecordType:peek_no_expand(VARCHAR(2147483647) name)
bucket, RecordType:peek_no_expand(VARCHAR(2147483647) key, BIGINT size) object)
s3)], joinType=[INNER], condition=[AND(>($0.object.size, 0), >($0.object.key,
_UTF-16LE''))])
StreamPhysicalTableSourceScan(table=[[default_catalog, default_database,
s3_put_event]], fields=[Records]) {code}
> CannotPlanException raised after "CROSS JOIN UNNEST"
>
>
> Key: FLINK-29873
> URL: https://issues.apache.org/jira/browse/FLINK-29873
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.15.2
>Reporter: Daeyeon Bae
>Priority: Major
> Attachments: error-log.txt
>
>
> When I create a VIEW as a result of "CROSS JOIN UNNEST" and then use the
> condition in the WHERE clause of the VIEW, it throws an exception
> "org.apache.calcite.plan.RelOptPlanner$CannotPlanException".
> Why am I getting this exception and how should I handle it the right way?
> The following is the test code in which an error occurs.
> {code:java}
> it should "filter with object_key" in {
> tEnv.executeSql(
> s"""CREATE TABLE s3_put_event (
> | Records ARRAY<
> |ROW<
> | s3 ROW<
> |bucket ROW,
> |object ROW
> | >
> |>
> | >
> |) WITH (
> | 'connector' = 'datagen',
> | 'number-of-rows' = '3',
> | 'rows-per-second' = '1',
> | 'fields.Records.element.s3.bucket.name.length' = '8',
> | 'fields.Records.element.s3.object.key.length' = '15',
> | 'fields.Records.element.s3.object.size.min' = '1',
> | 'fields.Records.element.s3.object.size.max' = '1000'
> |)
> |""".stripMargin
> )
> tEnv.executeSql(
> s"""CREATE TEMPORARY VIEW s3_objects AS
> |SELECT object_key, bucket_name
> |FROM (
> | SELECT
> |r.s3.bucket.name AS bucket_name,
> |r.s3.object.key AS object_key,
> |r.s3.object.size AS object_size
> | FROM s3_put_event
> | CROSS JOIN UNNEST(s3_put_event.Records) AS r(s3)
> |) rs
> |WHERE object_size > 0
> |""".stripMargin
> )
> tEnv.executeSql(
> s"""CREATE TEMPORARY VIEW filtered_s3_objects AS
> |SELECT bucket_name, object_key
> |FROM s3_objects
> |WHERE object_key > ''
> |""".stripMargin)
> val result = tEnv.sqlQuery("SELECT * FROM filtered_s3_objects")
> tEnv.toChangelogStream(result).print()
> env.execute()
> } {code}
> If I remove the condition object_key > '' in the "filtered_s3_objects" VIEW,
> and do it in the "s3_objects" VIEW, no exception is thrown.However, my actual
> query is complicated, so it is not easy to move the condition of the WHERE
> clause like this. It's hard to use especially if I need to separate the
> output stream.
> I have attached the error log.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-29873) CannotPlanException raised after "CROSS JOIN UNNEST"
[
https://issues.apache.org/jira/browse/FLINK-29873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17628625#comment-17628625
]
dalongliu commented on FLINK-29873:
---
CC [~godfreyhe]
> CannotPlanException raised after "CROSS JOIN UNNEST"
>
>
> Key: FLINK-29873
> URL: https://issues.apache.org/jira/browse/FLINK-29873
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.15.2
>Reporter: Daeyeon Bae
>Priority: Major
> Attachments: error-log.txt
>
>
> When I create a VIEW as a result of "CROSS JOIN UNNEST" and then use the
> condition in the WHERE clause of the VIEW, it throws an exception
> "org.apache.calcite.plan.RelOptPlanner$CannotPlanException".
> Why am I getting this exception and how should I handle it the right way?
> The following is the test code in which an error occurs.
> {code:java}
> it should "filter with object_key" in {
> tEnv.executeSql(
> s"""CREATE TABLE s3_put_event (
> | Records ARRAY<
> |ROW<
> | s3 ROW<
> |bucket ROW,
> |object ROW
> | >
> |>
> | >
> |) WITH (
> | 'connector' = 'datagen',
> | 'number-of-rows' = '3',
> | 'rows-per-second' = '1',
> | 'fields.Records.element.s3.bucket.name.length' = '8',
> | 'fields.Records.element.s3.object.key.length' = '15',
> | 'fields.Records.element.s3.object.size.min' = '1',
> | 'fields.Records.element.s3.object.size.max' = '1000'
> |)
> |""".stripMargin
> )
> tEnv.executeSql(
> s"""CREATE TEMPORARY VIEW s3_objects AS
> |SELECT object_key, bucket_name
> |FROM (
> | SELECT
> |r.s3.bucket.name AS bucket_name,
> |r.s3.object.key AS object_key,
> |r.s3.object.size AS object_size
> | FROM s3_put_event
> | CROSS JOIN UNNEST(s3_put_event.Records) AS r(s3)
> |) rs
> |WHERE object_size > 0
> |""".stripMargin
> )
> tEnv.executeSql(
> s"""CREATE TEMPORARY VIEW filtered_s3_objects AS
> |SELECT bucket_name, object_key
> |FROM s3_objects
> |WHERE object_key > ''
> |""".stripMargin)
> val result = tEnv.sqlQuery("SELECT * FROM filtered_s3_objects")
> tEnv.toChangelogStream(result).print()
> env.execute()
> } {code}
> If I remove the condition object_key > '' in the "filtered_s3_objects" VIEW,
> and do it in the "s3_objects" VIEW, no exception is thrown.However, my actual
> query is complicated, so it is not easy to move the condition of the WHERE
> clause like this. It's hard to use especially if I need to separate the
> output stream.
> I have attached the error log.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
