Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/3658#issuecomment-71550179
Yeah, so I think it would be better to just shade guava in YARN as well.
The main dependency-related constraint was that we didn't want Spark's yarn
shuffle service to have any external dependencies (i.e. it should work well out
of the box with YARN). But having an inlined shaded dependency doesn't matter
in this regard. So I don't see any other compelling reason to use YARN's guava.
Overall is easier to understand what is going on if we don't have a special
case there. And inlining things that we expose as semi-public classes (there
are some developer API's in those modules, IIRC), it's a bit off from the
normal maven model. So for all these reasons I think it would be good to just
shade it in that jar as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
