Hi Peter,
thanks for reaching out to the Flink community. This is indeed a serious
issue. As the author of the Flink type system, DataType and many related
utilities I strongly vote for reverting FLINK-33523:
- It changes the Flink type system without a FLIP.
- It breaks backwards compatibility with UDFs and connectors.
Regards,
Timo
On 07.12.23 07:38, Péter Váry wrote:
Hi Team,
We are working on upgrading the Iceberg-Flink connector from 1.17 to 1.18,
and found that some of our tests are failing. Prabhu Joseph created a jira
[1] to discuss this issue, along with short example code.
In a nutshell:
- Create a table with an 'ARRAY<INT NOT NULL>' column
- Run a select which returns this column
- The return type changes:
- From 'Object[]' - in 1.17
- To 'int[]' - in 1.18
The change is introduced by this jira [2].
While I understand the reasoning behind this change, this will break some
users existing workflow as evidenced by Xingcan Cui finding this
independently [3].
What is the opinion of the community about this change?
- Do we want to revert the change?
- Do we ask the owners of the change to make this behavior configurable?
- Do we accept this behavior change in a minor release?
Thanks,
Peter
[1] - https://issues.apache.org/jira/browse/FLINK-33523 - DataType
ARRAY<INT NOT NULL> fails to cast into Object[]
[2] - https://issues.apache.org/jira/browse/FLINK-31835 - DataTypeHint
don't support Row<i Array<int>>
[3] - https://issues.apache.org/jira/browse/FLINK-33547 - SQL primitive
array type after upgrading to Flink 1.18.0
