singhpk234 opened a new pull request, #5056: URL: https://github.com/apache/iceberg/pull/5056
Closes https://github.com/apache/iceberg/issues/4713 - based on https://github.com/singhpk234/iceberg/pull/1 ---- ### About Change - Introduced a new Node `ReplaceIcebergData` to be used in place of `ReplaceData`, as it extends command and commands are eagerly executed, which was causing problems with `Update` (in case we allow allowScanDuplication) & `RowLevelCommandDynamicPruning` (filters not being injected) - Changed the Rule name `RewriteDeleteFromTable` -> `RewriteDeleteFromIcebergTable` to avoid conflicting rule in upstream which doesn't support DeltaWrites and uses ReplaceData ----- ### Other Notes : Have marked some UT's ignored for complete suite to run: [1] `TestAddFilesProcedure` failed due to bug in spark upstream, I have fixed it in upstream but since the snapshot is last based of 7th May it still failing hence ignored it. JIRA : https://issues.apache.org/jira/browse/SPARK-39417 [2] Investing the RC for the below failures : TestMerge#testMergeWithInvalidAssignments TestDeleteFrom#testDeleteFromUnpartitionedTable, TestDeleteFrom#testDeleteFromPartitionedTable ---- cc @rdblue @aokolnychyi @jackye1995 @RussellSpitzer @kbendick -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
