[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15131 @HyukjinKwon @shivaram Thanks for your comments. I will fix the URI issue in this PR as you suggested. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/15131
I just took a look. The problematic code is here,
[SparkContext.scala#L1429](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L1429).
We should not directly use `new URI` with Windows path because the paths
such as `C:\a\b\c` is not the valid URI as below:
```scala
scala> new
java.net.URI("C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\RtmpegI4mr\\hello92023051e13.txt")
java.net.URISyntaxException: Illegal character in opaque part at index 2:
C:\Users\appveyor\AppData\Local\Temp\1\RtmpegI4mr\hello92023051e13.txt
at java.net.URI$Parser.fail(URI.java:2848)
at java.net.URI$Parser.checkChars(URI.java:3021)
at java.net.URI$Parser.parse(URI.java:3058)
at java.net.URI.(URI.java:588)
... 33 elided
```
I took a look the APIs taking `path` as an argument in `SparkContext` and
it seems this one is only the case. It seems the similar case is being handled
in
[SparkContext.scala#L1702-L1707](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L1702-L1707).
I can open a separate PR, post a PR to @yanboliang's forked repo or just
let you fix this in here. Please let me know.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15131 @shivaram Thanks for cc'ing me. I will try to look closely within today. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15131 It looks like `addFile` isn't working on Windows because we try to convert the windows file path into a URI and that fails. Not sure what the fix is in this case. cc @HyukjinKwon who worked on this for `hadoopFile` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15131 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15131 **[Test build #65539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65539/consoleFull)** for PR 15131 at commit [`d3dd380`](https://github.com/apache/spark/commit/d3dd3808e88b3f4ba5af683eb7d7709fcc2710f7). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15131 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65539/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15131 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15131 **[Test build #65540 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65540/consoleFull)** for PR 15131 at commit [`5c49428`](https://github.com/apache/spark/commit/5c49428738d8817f43f23c60f85850864845e7b9). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15131 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65540/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15131 **[Test build #65540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65540/consoleFull)** for PR 15131 at commit [`5c49428`](https://github.com/apache/spark/commit/5c49428738d8817f43f23c60f85850864845e7b9). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15131 **[Test build #65539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65539/consoleFull)** for PR 15131 at commit [`d3dd380`](https://github.com/apache/spark/commit/d3dd3808e88b3f4ba5af683eb7d7709fcc2710f7). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
