Github user gatorsmile commented on the pull request:
https://github.com/apache/spark/pull/10362#issuecomment-165804543
Thank you for the detailed explanation! @liancheng
Actually, in the long term, I like the fix by @yhuai . : ) I do not like
the tricky way. It took me a few hours to fully understand the current
implementation. I will try to work on the remaining follow-up issues.
BTW, the push-down design is very critical. It largely affects the overall
costs and performance. The current design of Push-down is based on the
implementation of data source APIs. As long as the data sources can do it, we
will push it down. That might not be always optimal. End users might have
different strategies due to various reasons. Anyway, let me think about it.
Thank you!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
