[
https://issues.apache.org/jira/browse/FLINK-40658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu updated FLINK-40658:
----------------------------
Parent: FLINK-40185
Issue Type: Sub-task (was: Bug)
> UNNEST silently drops null ROW elements from ARRAY
> --------------------------------------------------
>
> Key: FLINK-40658
> URL: https://issues.apache.org/jira/browse/FLINK-40658
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Runtime
> Affects Versions: 2.4.0
> Reporter: Bettle
> Assignee: Bettle
> Priority: Major
>
> While implementing DataFrame.explode() for FLINK-40435, I noticed that UNNEST
> silently drops null ROW elements from an ARRAY.
> This reproduces through direct Table API/SQL calls on an unchanged baseline,
> without importing or using the new DataFrame implementation.
> h3. Reproduction
> Run the following code in a configured PyFlink environment:
> {code:python}
> from pyflink.table import EnvironmentSettings, TableEnvironment
> t_env = TableEnvironment.create(EnvironmentSettings.in_batch_mode())
> source = t_env.sql_query(
> "SELECT ARRAY[CAST(NULL AS ROW<n INT, s STRING>), "
> "ROW(1, 'a')] AS items"
> )
> for join, condition in [
> ("CROSS JOIN", ""),
> ("LEFT JOIN", " ON TRUE"),
> ]:
> query = (
> f"SELECT expanded.* FROM `{str(source)}` AS src "
> f"{join} UNNEST(src.items) AS expanded(n, s){condition}"
> )
> with t_env.sql_query(query).execute().collect() as rows:
> actual = list(rows)
> print(join, actual)
> {code}
> h3. Expected result
> Both join variants should return two rows, regardless of order:
> {code}
> (NULL, NULL)
> (1, 'a')
> {code}
> h3. Actual result
> Both join variants return only:
> {code}
> (1, 'a')
> {code}
> Execution completes without an exception, but the null ROW element is missing.
> The input is a nonempty array containing a null ROW element. This is
> different from an empty array, a null array, or a non-null ROW whose fields
> are all null.
> h3. Environment
> * Flink 2.4-SNAPSHOT, baseline e01bbcacd96
> * Batch mode
> * Java 17.0.14
> * Python 3.12.11
> The equivalent direct Table API/SQL tests reproduced this behavior for both
> CROSS JOIN and LEFT JOIN. Released versions and streaming mode have not been
> verified for this issue.
> h3. Related work
> This was discovered during FLINK-40435, but reproduces without the new
> DataFrame wrappers. I am reporting it separately to keep that work focused on
> the DataFrame API. No underlying runtime fix is included in that work.
> If this is confirmed to be a bug that needs fixing, I would be happy to
> investigate further and try to contribute a fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
