[
https://issues.apache.org/jira/browse/CALCITE-6579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881382#comment-17881382
]
Mihai Budiu commented on CALCITE-6579:
--------------------------------------
My PR had two components: changing the parser, and changing the way these
nested types are constructed. We could only take the second part of the
solution and reuse it to solve your problem.
For our compiler we changed to spec to say that in arrays or user-defined types
the fields are always nullable and there is no way around it.
But since ROW allows specifying the nullability field by field, clearly this is
a bug which needs to be fixed.
> Unexpected nullable checks for nullable types with non-nullable children
> -------------------------------------------------------------------------
>
> Key: CALCITE-6579
> URL: https://issues.apache.org/jira/browse/CALCITE-6579
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.37.0
> Reporter: david radley
> Priority: Major
>
> This Jira is raised for the issue discussed in
> [https://lists.apache.org/thread/x857njnd42gcvrldbcd7y85to6fb58dn]
>
> I am using flink and have a table definition like this.
> _CREATE TABLE source_1 ( `order_id` STRING NOT NULL, `order_time` STRING NOT
> NULL, `buyer` ROW< `first_name` STRING, `last_name` STRING NOT NULL, `title`
> STRING NOT NULL_
> _) WITH ( 'connector' = 'kafka', 'topic' = 'mytopic',
> 'properties.bootstrap.servers' = 'localhost:9092', 'value.format' = 'avro',
> 'value.fields-include' = 'ALL', 'scan.startup.mode' = 'earliest-offset' );_
>
> This errors as code in Calcite does not create the correct schema.
> It sets`last_name` STRING NOT NULL, `title` STRING NOT NULL to both be
> nullable.
> The cause of this is in the table planner when we convert SqlDataTypeSpec to
> the RelDataType. We push the nullable of buyer onto all of its fields, losing
> the children’s nullable status . In the debugger I see that it is taking the
> nullable true (from buyer) and putting this on all the children in
> [fixupNullability|http://example.com]https://github.com/davidradl/calcite/blob/ad2e843c5d9b3bec001d22e680ebe6b5de4e2078/core/src/main/java/org/apache/calcite/sql/SqlDataTypeSpec.java#L238.
> I can see that the SqlDataTypeSpec has this information but it is not used.
>
> Flink maps this directly to an Avro schema that incorreclty polices the Avro
> payload.
>
> In terms of fixing this, I wonder if the type could be created with the
> correct nullable so it doesn’t need to be fixed up or whether we should pass
> down the SqlDataTypeSpec so it has the right information to set the nullable
> status.
>
> This subject has previously been raised on the dev list
> [https://lists.apache.org/thread/s4nd9rk0fzckoctokl7kjdbtfbvxncy7] by
> [~dwysakowicz]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
