Github user davies commented on the pull request:
https://github.com/apache/spark/pull/11682#issuecomment-196433895
After some offline discussions with @marmbrus , we realized that these two
rules may still conflict with each other if the Filter can'be pushed throughout
the child (for example, outer join). A better solution could be split the
ColumnPruning as two rules: a) the first one add new Project 2) the second one
remove unnecessary projects, include the Project under Filter (the Project
should only prune some columns). The second rule ran just after the first role.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
