GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/21633
[SPARK-21917][CORE][FOLLOWUP] Minor change to
spark.yarn.dist.forceDownloadSchemes to support wildcard '*'
## What changes were proposed in this pull request?
In the case of getting tokens via customized `ServiceCredentialProvider`,
it is required that `ServiceCredentialProvider` be available in local
spark-submit process classpath. In this case, all the configured remote sources
should be forced to download to local.
For the ease of using this configuration, here propose to add wildcard '*'
support to `spark.yarn.dist.forceDownloadSchemes`, also clarify the usage of
this configuration.
## How was this patch tested?
New UT added.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark SPARK-21917-followup
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21633.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21633
----
commit 6d0a2b774d4f9c4206225b601e1374818e676b9c
Author: jerryshao <sshao@...>
Date: 2018-06-25T06:06:10Z
Minor change to spark.yarn.dist.forceDownloadSchemes to support wildcard '*'
commit f3c99f885da4245b6096975f9e2213a411262654
Author: jerryshao <sshao@...>
Date: 2018-06-25T06:38:32Z
Add unit test
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
