wypoon commented on pull request #3335: URL: https://github.com/apache/iceberg/pull/3335#issuecomment-950008345
@aokolnychyi I saw that in v3.2, quite a number of rules have been removed from `IcebergSparkSessionExtensions` (and the classes removed): `AlignRowLevelOperations`, `RowLevelOperationsPredicateCheck`, `OptimizeConditionsInRowLevelOperations`, `PullupCorrelatedPredicatesInRowLevelOperations`, `RewriteDelete`, `RewriteUpdate`, and `RewriteMergeInto`. I also saw that `TestCopyOnWriteDelete`, `TestCopyOnWriteMerge` and `TestCopyOnWriteUpdate` are all being skipped entirely. Do you plan follow up work rewriting support for delete/update/merge using Spark 3.2 features? As of this change, what functionality works in 3.0 but not in 3.2? I understand that some of the removed classes are things that are no longer needed because new API is now available in Spark 3.2 (some already in 3.1), e.g., `ExtendedBatchScanExec` and the dynamic file filter stuff. But I'd like to understand the delete/update/merge support situation (please pardon ignorance, I'm not familiar with this area yet). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
