GitHub user fangshil opened a pull request:
https://github.com/apache/spark/pull/21339
[SPARK-24287][Core] Spark -packages option should support classifier,
no-transitive, and custom conf
## What changes were proposed in this pull request?
We should extend Spark's -package option to support:
1. Turn-off transitive dependency on a given artifact(like spark-avro)
2. Resolving a given artifact with classifier (like avro-mapred-1.7.4-h2.jar
3. Resolving a given artifact with custom ivy conf
4. Excluding particular transitive dependencies from a given artifact. We
currently only have top-level exclusion rule applies for all artifacts.
We have tested this patch internally and it greatly increases the
flexibility when user uses -packages option
## How was this patch tested?
added unit test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fangshil/spark SPARK-24287
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21339.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21339
----
commit 30f378f1ddba7a23b7a72ee96bb1648fd80dbbd3
Author: Fangshi Li <fli@...>
Date: 2017-04-11T08:27:05Z
[SPARK-24287][Core] Spark -packages option should support classifier,
no-transitive, and custom conf
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
