beetle0915 opened a new pull request, #29196: URL: https://github.com/apache/flink/pull/29196
## What is the purpose of the change Add `DataFrame.explode()` as specified in FLIP-591 and [FLINK-40435](https://issues.apache.org/jira/browse/FLINK-40435), exposing collection expansion through the DataFrame API. ## Brief change log - Expand ARRAY, MAP and MULTISET columns using existing SQL UNNEST, including ROW-valued elements and duplicate occurrences. - Preserve empty/null collections by default with LEFT JOIN; use CROSS JOIN when `ignore_empty_and_null=True`. - Validate expressions and output names, preserve retained columns, and return a lazy DataFrame without modifying the input. - Add API documentation and 20 tests covering argument validation, schemas, aliases, computed expressions, identifier quoting, batch execution and streaming changelog multiplicities. ## Verifying this change Local validation uses Python 3.12.11 and Java 17.0.14. The Python source is rebased onto master at `bd0a65afb94`; the local Java distribution is the existing 2.4-SNAPSHOT build from `e01bbcacd96`, not a fresh Java build of this master revision. - `python -m pytest -q --tb=short pyflink/dataframe/tests`: 371 passed, 8 existing deprecation warnings (93.26s). - `python -m flake8 --config=tox.ini pyflink/dataframe`: passed. - `python -m mypy --config-file tox.ini`: passed for 83 source files. - Sphinx HTML build with `-W --keep-going`: passed. - `git diff origin/master --check`: passed. Full-repository Maven verification, a fresh Java rebuild and community CI validation remain outstanding; this PR is initially a draft. The existing UNNEST behavior that drops null ROW elements is tracked separately in [FLINK-40658](https://issues.apache.org/jira/browse/FLINK-40658). It reproduces through direct Table API/SQL without this change. This PR does not fix that behavior; the separate reproduction is in that ticket. ## Does this pull request potentially affect one of the following parts: - Dependencies: no. - Public API: yes, adds an `@PublicEvolving` DataFrame method specified by FLIP-591. - Serializers: no. - Runtime per-record code paths: no changes; delegates to existing UNNEST execution. - Deployment or recovery: no. - S3 file system connector: no. ## Documentation - New feature: yes. - Documented in the method docstring and DataFrame API reference autosummary. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: OpenAI Codex CLI 0.154.0-alpha.6.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
