Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1813#issuecomment-51602338
@pwendell This is complicated, so let me summarize what I think I
understand and then give an opinion.
The plan at hand concerned changes to the assembly JARs, pretty much
exclusively. Spark continues to depend on Guava in order to compile, and so
that consumers of Spark also see this dependency in their builds.
(You can see the discussion with @vanzin about whether to alter how
downstream consumers of the Maven build see the Guava dependency -- `provided`
vs `compile` vs `optional` -- but this doesn't affect the assembly, and only
affects whether Guava automatically becomes a part of the downstream consumer's
classpath. I still have a minor issue with `provided`, but unless @pwendell has
any opinion on this, consider me outvoted.)
So I think the proposal was already to continue showing a dependency on
Guava, but, to alter the assembly to include only `Optional`, and shaded copies
of all else. And maybe to tweak the dependency to hide it by default for
downstream consumers.
Inlining the shaded JARs is all but like removing use of Guava, as far as
the world is concerned. That certainly takes it out of the picture. The little
snag is Optional. It will now inevitably collide with any other app that builds
Guava in too. And downstream consumers can't manage away the conflict normally.
It may either be a harmless warning or cause a real problem if Optional changes
later. In general I don't like pushing the shading into the compile-time world.
Remember the `hive-exec` awfulness? This would not be my first choice, versus:
I think your second paragraph is basically the same thing proposed already,
with one difference. Yes, the build still continues to say it depends on Guava.
The concrete assembly is put together differently. I think you *can* use
Guava's Optional rather than embedding a source copy of it and compiling that.
Right @vanzin? with some shading cleverness?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
