GitHub user xuanyuanking opened a pull request:
https://github.com/apache/spark/pull/21533
[SPARK-24195][Core] Bug fix for local:/ path in SparkContext.addFile
## What changes were proposed in this pull request?
In the chagnes in
[SPARK-6300](https://issues.apache.org/jira/browse/SPARK-6300), essentially it
change schemePath to
```
new File(path).getCanonicalFile.toURI.toString
```
. This has problem when path is local:, as `java.io.File` doesn't handle it.
eg.
new
File("local:///home/user/demo/logger.config").getCanonicalFile.toURI.toString
res1: String = file:/user/anotheruser/local:/home/user/demo/logger.config
## How was this patch tested?
Add test in `SparkContextSuite`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xuanyuanking/spark SPARK-24195
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21533.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21533
----
commit f922fd8c995164cada4a8b72e92c369a827def16
Author: Yuanjian Li <xyliyuanjian@...>
Date: 2018-06-12T01:51:44Z
bug fix for local:/ path in sc.addFile
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
