rdblue commented on pull request #35395: URL: https://github.com/apache/spark/pull/35395#issuecomment-1075271152
> I assumed the delete condition (not negated) would be explicitly passed to both scan builders by Spark Good, I would prefer not to assume that both scans use information passed to one or the other. > pushing down the static predicate earlier to the main Scan doesn't hurt. I agree. In many cases this is actually required. For example, in a Hive table you wouldn't want to try to reconstruct a partition filter out of paths from the `IN` predicate. So you'd would also want to do partition pruning before file pruning. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
