[jira] [Updated] (FLINK-30282) Logical type ROW lost inner field's nullability after convert to RelDataType
[
https://issues.apache.org/jira/browse/FLINK-30282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-30282:
---
Labels: pull-request-available (was: )
> Logical type ROW lost inner field's nullability after convert to RelDataType
>
>
> Key: FLINK-30282
> URL: https://issues.apache.org/jira/browse/FLINK-30282
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Affects Versions: 1.16.0, 1.16.1
>Reporter: Jane Chan
>Priority: Major
> Labels: pull-request-available
>
> h3. Issue History
> This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344
> spared efforts to fix it (but did not tweak the ut case mentioned in
> FLINK-13604, i.e.
> SqlToOperationConverterTest#testCreateTableWithFullDataTypes). Nevertheless,
> the FunctionITCase added by FLINK-16344, which validates the fix, has been
> removed in FLINK-16377.
> h3. How to Reproduce
> c.c2 lost nullability
> {code:java}
> Flink SQL> create table dummy (a array not null, b array not null>, c row) with ('connector' = 'datagen');
> [INFO] Execute statement succeed.
> Flink SQL> desc dummy;
> +--++---+-++---+
> | name | type | null | key | extras | watermark |
> +--++---+-++---+
> | a | ARRAY | FALSE | | | |
> | b | ARRAY | TRUE | | | |
> | c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
> +--++---+-++---+
> 3 rows in set
> {code}
> h3. Root Cause
> Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
> 1. dt.deriveType should also pass dt's nullability as well. See
> [https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
>
> 2. StructKind should be PEEK_FIELDS_NO_EXPAND instead of FULLY_QUALIFIED(see
> [https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/type/StructKind.java]),
> so that FlinkTypeFactory#createTypeWithNullability will not fall back to
> super implement. See
> [https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-30282) Logical type ROW lost inner field's nullability after convert to RelDataType
[
https://issues.apache.org/jira/browse/FLINK-30282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jane Chan updated FLINK-30282:
--
Description:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344
spared efforts to fix it (but did not tweak the ut case mentioned in
FLINK-13604, i.e.
SqlToOperationConverterTest#testCreateTableWithFullDataTypes). Nevertheless,
the FunctionITCase added by FLINK-16344, which validates the fix, has been
removed in FLINK-16377.
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Root Cause
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND instead of FULLY_QUALIFIED(see
[https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/type/StructKind.java]),
so that FlinkTypeFactory#createTypeWithNullability will not fall back to super
implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
was:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344 has
also fixed it. However, the FunctionITCase added by FLINK-16344, which
validates the fix, has been removed in FLINK-16377. From that moment, there
might be some other changes happening and breaking the fix(sorry, I haven't
spent too much time digging into these changes)
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Root Cause
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND instead of FULLY_QUALIFIED(see
[https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/type/StructKind.java]),
so that FlinkTypeFactory#createTypeWithNullability will not fall back to super
implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
> Logical type ROW lost inner field's nullability after convert to RelDataType
>
>
> Key: FLINK-30282
> URL: https://issues.apache.org/jira/browse/FLINK-30282
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Affects Versions: 1.16.0, 1.16.1
>Reporter: Jane Chan
>Priority: Major
>
> h3. Issue History
> This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344
> spared efforts to fix it (but did not tweak the ut case mentioned in
> FLINK-13604, i.e.
> SqlToOperationConverterTest#testCreateTableWithFullDataTypes). Nevertheless,
> the FunctionITCase added by FLINK-16344, which validates the fix, has been
> removed in FLINK-16377.
> h3. How to Reproduce
> c.c2 lost nullability
> {code:java}
> Flink SQL> create table dummy (a array not null, b
[jira] [Updated] (FLINK-30282) Logical type ROW lost inner field's nullability after convert to RelDataType
[
https://issues.apache.org/jira/browse/FLINK-30282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jane Chan updated FLINK-30282:
--
Description:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344 has
also fixed it. However, the FunctionITCase added by FLINK-16344, which
validates the fix, has been removed in FLINK-16377. From that moment, there
might be some other changes happening and breaking the fix(sorry, I haven't
spent too much time digging into these changes)
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Root Cause
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND instead of FULLY_QUALIFIED(see
[https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/type/StructKind.java]),
so that FlinkTypeFactory#createTypeWithNullability will not fall back to super
implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
was:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344 has
also fixed it. However, the FunctionITCase added by FLINK-16344, which
validates the fix, has been removed in FLINK-16377. From that moment, there
might be some other changes happening and breaking the fix(sorry, I haven't
spent too much time digging into these changes)
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Root Cause
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND, so that
FlinkTypeFactory#createTypeWithNullability will not fall back to super
implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
> Logical type ROW lost inner field's nullability after convert to RelDataType
>
>
> Key: FLINK-30282
> URL: https://issues.apache.org/jira/browse/FLINK-30282
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Affects Versions: 1.16.0, 1.16.1
>Reporter: Jane Chan
>Priority: Major
>
> h3. Issue History
> This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344
> has also fixed it. However, the FunctionITCase added by FLINK-16344, which
> validates the fix, has been removed in FLINK-16377. From that moment, there
> might be some other changes happening and breaking the fix(sorry, I haven't
> spent too much time digging into these changes)
> h3. How to Reproduce
> c.c2 lost nullability
> {code:java}
> Flink SQL> create table dummy (a array not null, b array not null>, c row) with ('connector' = 'datagen');
> [INFO] Execute statement succeed.
> Flink SQL> desc dummy;
>
[jira] [Updated] (FLINK-30282) Logical type ROW lost inner field's nullability after convert to RelDataType
[
https://issues.apache.org/jira/browse/FLINK-30282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jane Chan updated FLINK-30282:
--
Description:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344 has
also fixed it. However, the FunctionITCase added by FLINK-16344, which
validates the fix, has been removed in FLINK-16377. From that moment, there
might be some other changes happening and breaking the fix(sorry, I haven't
spent too much time digging into these changes)
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Root Cause
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND, so that
FlinkTypeFactory#createTypeWithNullability will not fall back to super
implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
was:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344 has
also fixed it. However, the FunctionITCase added by FLINK-16344, which
validates the fix, has been removed in FLINK-16377. From that moment, there
might be some other changes happening and breaking the fix(sorry, I haven't
spent too much time digging into these changes)
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Reason
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND, so that
FlinkTypeFactory#createTypeWithNullability will not fall back to super
implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
> Logical type ROW lost inner field's nullability after convert to RelDataType
>
>
> Key: FLINK-30282
> URL: https://issues.apache.org/jira/browse/FLINK-30282
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Affects Versions: 1.16.0, 1.16.1
>Reporter: Jane Chan
>Priority: Major
>
> h3. Issue History
> This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344
> has also fixed it. However, the FunctionITCase added by FLINK-16344, which
> validates the fix, has been removed in FLINK-16377. From that moment, there
> might be some other changes happening and breaking the fix(sorry, I haven't
> spent too much time digging into these changes)
> h3. How to Reproduce
> c.c2 lost nullability
> {code:java}
> Flink SQL> create table dummy (a array not null, b array not null>, c row) with ('connector' = 'datagen');
> [INFO] Execute statement succeed.
> Flink SQL> desc dummy;
> +--++---+-++---+
> | name | type | null | key | extras | watermark |
>
[jira] [Updated] (FLINK-30282) Logical type ROW lost inner field's nullability after convert to RelDataType
[
https://issues.apache.org/jira/browse/FLINK-30282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jane Chan updated FLINK-30282:
--
Parent: FLINK-12251
Issue Type: Sub-task (was: Bug)
> Logical type ROW lost inner field's nullability after convert to RelDataType
>
>
> Key: FLINK-30282
> URL: https://issues.apache.org/jira/browse/FLINK-30282
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Affects Versions: 1.16.0, 1.16.1
>Reporter: Jane Chan
>Priority: Major
>
> h3. Issue History
> This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344
> has also fixed it. However, the FunctionITCase added by FLINK-16344, which
> validates the fix, has been removed in FLINK-16377. From that moment, there
> might be some other changes happening and breaking the fix(sorry, I haven't
> spent too much time digging into these changes)
>
> h3. How to Reproduce
> c.c2 lost nullability
> {code:java}
> Flink SQL> create table dummy (a array not null, b array not null>, c row) with ('connector' = 'datagen');
> [INFO] Execute statement succeed.
> Flink SQL> desc dummy;
> +--++---+-++---+
> | name | type | null | key | extras | watermark |
> +--++---+-++---+
> | a | ARRAY | FALSE | | | |
> | b | ARRAY | TRUE | | | |
> | c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
> +--++---+-++---+
> 3 rows in set
> {code}
> h3. Reason
> Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
> 1. dt.deriveType should also pass dt's nullability as well. See
> [https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
>
> 2. StructKind should be PEEK_FIELDS_NO_EXPAND, so that
> FlinkTypeFactory#createTypeWithNullability will not fall back to super
> implement. See
> [https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-30282) Logical type ROW lost inner field's nullability after convert to RelDataType
[
https://issues.apache.org/jira/browse/FLINK-30282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jane Chan updated FLINK-30282:
--
Description:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344 has
also fixed it. However, the FunctionITCase added by FLINK-16344, which
validates the fix, has been removed in FLINK-16377. From that moment, there
might be some other changes happening and breaking the fix(sorry, I haven't
spent too much time digging into these changes)
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Reason
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND, so that
FlinkTypeFactory#createTypeWithNullability will not fall back to super
implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
was:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344 has
also fixed it. However, the FunctionITCase added by FLINK-16344, which
validates the fix, has been removed in FLINK-16377. From that moment, there
might be some other changes happening and breaking the fix(sorry, I haven't
spent too much time digging into these changes)
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Reason
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND, so that FlinkTypeFactory#
createTypeWithNullability will not fall back to super implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
> Logical type ROW lost inner field's nullability after convert to RelDataType
>
>
> Key: FLINK-30282
> URL: https://issues.apache.org/jira/browse/FLINK-30282
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Affects Versions: 1.16.0, 1.16.1
>Reporter: Jane Chan
>Priority: Major
>
> h3. Issue History
> This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344
> has also fixed it. However, the FunctionITCase added by FLINK-16344, which
> validates the fix, has been removed in FLINK-16377. From that moment, there
> might be some other changes happening and breaking the fix(sorry, I haven't
> spent too much time digging into these changes)
>
> h3. How to Reproduce
> c.c2 lost nullability
> {code:java}
> Flink SQL> create table dummy (a array not null, b array not null>, c row) with ('connector' = 'datagen');
> [INFO] Execute statement succeed.
> Flink SQL> desc dummy;
> +--++---+-++---+
> | name | type | null | key | extras | watermark |
>
[jira] [Updated] (FLINK-30282) Logical type ROW lost inner field's nullability after convert to RelDataType
[
https://issues.apache.org/jira/browse/FLINK-30282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jane Chan updated FLINK-30282:
--
Description:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344 has
also fixed it. However, the FunctionITCase added by FLINK-16344, which
validates the fix, has been removed in FLINK-16377. From that moment, there
might be some other changes happening and breaking the fix(sorry, I haven't
spent too much time digging into these changes)
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Reason
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND, so that FlinkTypeFactory#
createTypeWithNullability will not fall back to super implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
was:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344 has
also fixed it. However, the FunctionITCase added by FLINK-16344, which
validates the fix, has been removed in FLINK-16377. From that moment, there
might be some other changes happening and breaking the fix(sorry, I haven't
spent too much time digging into these changes)
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Reason
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND, so that FlinkTypeFactory#
createTypeWithNullability will not fall back to super implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
> Logical type ROW lost inner field's nullability after convert to RelDataType
>
>
> Key: FLINK-30282
> URL: https://issues.apache.org/jira/browse/FLINK-30282
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Affects Versions: 1.16.0, 1.16.1
>Reporter: Jane Chan
>Priority: Major
>
> h3. Issue History
> This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344
> has also fixed it. However, the FunctionITCase added by FLINK-16344, which
> validates the fix, has been removed in FLINK-16377. From that moment, there
> might be some other changes happening and breaking the fix(sorry, I haven't
> spent too much time digging into these changes)
>
> h3. How to Reproduce
> c.c2 lost nullability
> {code:java}
> Flink SQL> create table dummy (a array not null, b array not null>, c row) with ('connector' = 'datagen');
> [INFO] Execute statement succeed.
> Flink SQL> desc dummy;
> +--++---+-++---+
> | name | type | null | key | extras | watermark |
>
[jira] [Updated] (FLINK-30282) Logical type ROW lost inner field's nullability after convert to RelDataType
[
https://issues.apache.org/jira/browse/FLINK-30282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jane Chan updated FLINK-30282:
--
Description:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344 has
also fixed it. However, the FunctionITCase added by FLINK-16344, which
validates the fix, has been removed in FLINK-16377. From that moment, there
might be some other changes happening and breaking the fix(sorry, I haven't
spent too much time digging into these changes)
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Reason
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND, so that FlinkTypeFactory#
createTypeWithNullability will not fall back to super implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
was:
h3. Issue History
This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344 has
also fixed it. However, the FunctionITCase added by FLINK-16344, which
validates the fix, has been removed in FLINK-16377. From that moment, there
might be some other changes happening and breaking the fix(sorry, I haven't
spent too much time digging into these changes)
h3. How to Reproduce
c.c2 lost nullability
{code:java}
Flink SQL> create table dummy (a array not null, b array, c row) with ('connector' = 'datagen');
[INFO] Execute statement succeed.
Flink SQL> desc dummy;
+--++---+-++---+
| name | type | null | key | extras | watermark |
+--++---+-++---+
| a | ARRAY | FALSE | | | |
| b | ARRAY | TRUE | | | |
| c | ROW<`c1` INT, `c2` DOUBLE> | TRUE | | | |
+--++---+-++---+
3 rows in set
{code}
h3. Reason
Two places are causing this problem in ExtendedSqlRowTypeNameSpec.
1. dt.deriveType should also pass dt's nullability as well. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/type/ExtendedSqlRowTypeNameSpec.java#L159]
2. StructKind should be PEEK_FIELDS_NO_EXPAND, so that FlinkTypeFactory#
createTypeWithNullability will not fall back to super implement. See
[https://github.com/apache/flink/blob/fb27e6893506006b9a3b1ac3e9b878fb6cad061a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/calcite/FlinkTypeFactory.scala#L417]
> Logical type ROW lost inner field's nullability after convert to RelDataType
>
>
> Key: FLINK-30282
> URL: https://issues.apache.org/jira/browse/FLINK-30282
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Affects Versions: 1.16.0, 1.16.1
>Reporter: Jane Chan
>Priority: Major
>
> h3. Issue History
> This is not a new issue, FLINK-13604 has tracked it before, and FLINK-16344
> has also fixed it. However, the FunctionITCase added by FLINK-16344, which
> validates the fix, has been removed in FLINK-16377. From that moment, there
> might be some other changes happening and breaking the fix(sorry, I haven't
> spent too much time digging into these changes)
>
> h3. How to Reproduce
> c.c2 lost nullability
> {code:java}
> Flink SQL> create table dummy (a array not null, b array not null>, c row) with ('connector' = 'datagen');
> [INFO] Execute statement succeed.
> Flink SQL> desc dummy;
> +--++---+-++---+
> | name | type | null | key | extras | watermark |
>
