Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5044#issuecomment-82685347
While doing a more complete query subsumption calculation would be great,
in general this is a very hard problem that is very easy to mess up. You'll
need to clearly define what you mean by `part` and there will need to be a lot
more test cases showing that all of the edge cases are correct. The function
itself will also need more comments. In particular, its not clear to me how
you can implement such a function generally without understanding the semantics
of the arguments to the operator themselves. Finally such a significant change
to such a central system would need to be hidden behind a feature flag.
Ideally a full design would be proposed on the JIRA before more coding is
done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
