Visorgood opened a new pull request, #7627: URL: https://github.com/apache/datafusion-comet/pull/7627
## Which issue does this PR close? Closes #7622. ## Rationale for this change `IcebergWriteExec` and `CometIcebergWriteExec` create the `iceberg_commit_message` attribute they output, but they didn't declare it in `producedAttributes`. Spark therefore counted it as missing input, and EXPLAIN printed the nodes as `!IcebergWrite` and `!CometIcebergWrite`, which reads like a broken plan. The writes themselves were correct. ## What changes are included in this PR? - `override def producedAttributes: AttributeSet = outputSet` in `IcebergWriteExec` and `CometIcebergWriteExec`, as `CometProjectExec` already does. - `CometIcebergWriteActionSuite`: `captureWrite` now also asserts that these write nodes have no missing input, next to the existing columnar-contract check, so every write in the suite is checked. - A new test, "split-plan write nodes do not report their own output as missing input", that writes once through iceberg-java and once natively and asserts that both nodes are present and have empty `missingInput`. ## How are these changes tested? - The new test passes, and fails without the main-code change (`write nodes with missing input: IcebergWrite ...`). - The full `CometIcebergWriteActionSuite` passes on the default profile (Spark 4.1). - Checked manually in `spark-shell` (Spark 4.1.3, Iceberg 1.11.0): EXPLAIN of a native INSERT, of an ORC INSERT that stays on iceberg-java, and of an insert-only MERGE under AQE no longer shows `!`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
