dwsmith1983 opened a new issue, #5946: URL: https://github.com/apache/datafusion-comet/issues/5946
### What is the problem the feature request solves? `planningHelper` in `CometDeltaNativeScanExec.scala` reuses core's listing, splitting and dynamic partition pruning, but its contract is implicit: partition filters are supplied separately while data filters come from `originalPlan`, and predicates used for file selection are not distinguished from predicates the reader evaluates. ### Describe the potential solution A contract that takes the original scan plus the current partition and data filters and returns the final `FilePartition` objects with their metadata, with the existing reader-free helper as the implementation and the serializer consuming only its result. Plan copies and filter rewrites keep the current protection against stale cached selection. Regression tests compare serialized paths, order, split starts and lengths, partition values and metadata against Spark's final partitions, including a non-DV file with a nonzero split start and dynamic partition pruning with AQE on and off. ### Additional context Raised in the #5365 review (item 4); filed as a follow-up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
