Github user andreweduffy commented on the issue:
https://github.com/apache/spark/pull/14649
Hey @paulata, sounds like you want to make the pruner pluggable at runtime.
That sounds like it would be good, though IMO is a bit orthogonal to this
change. This is more of an optimization than an FR, i.e. you're "no worse off"
with this change performance-wise and all pre-existing behavior is preserved,
whereas yours is a new feature.
You're welcome to reuse the code in this PR for your own needs, or to base
your changes on top of this. I have my doubts that the PR in its current form
will be merged into Spark, but it's already been merged into Palantir's fork
(https://github.com/palantir/spark/pull/40) so hopefully the folks there will
try and upstream it at some point in the future.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
