peter-toth commented on issue #23802: [SPARK-26893][SQL] Allow pushdown of partition pruning subquery filters to file source URL: https://github.com/apache/spark/pull/23802#issuecomment-464118835 > @peter-toth I expressed my major concern on the JIRA actually. I don't see them addressed here and actually IIRC before that change subqueries were pushed down but not used for actually filtering out partitions (I don't see this change making them of use). I have not yet carefully reviewed this PR (will do as soon as I have time), so I may be missing something, I apologize in advance if I do. I think before your PR they were pushed down as `dataFilters` and that was the reason for not using them but still have them 2 times in the plan.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
