GitHub user cbvoxel opened a pull request:
https://github.com/apache/spark/pull/7872
[SPARK-5754] [yarn] Spark AM not launching on Windows
This is my suggestion from the discussion in the ticket [SPARK-5745], the
PR #7829 and my ideas for fixes for the situation.
I added tests, which __fail__!! I need your support to define the correct
behavior.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cbvoxel/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7872.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7872
----
commit 65a972d428ae466c6822479d3050ff2fe44f70fd
Author: Carsten Blank <[email protected]>
Date: 2015-08-01T16:24:02Z
Use platform independent toURI Format instead platform dependent
toAbsolutePath.
commit 133fde2a1a1e9c9a7921b82704849829ed2e47c7
Author: Carsten Blank <[email protected]>
Date: 2015-08-01T16:24:46Z
Use platform independent toURI Format instead platform dependent
toAbsolutePath.
commit 6fef1755bdee444676879c9ac138343937406388
Author: Carsten Blank <[email protected]>
Date: 2015-08-01T16:25:12Z
Escaping for Windows.
commit 39107409a5534c3c8983a0a8b2d208d6ec18ce39
Author: Carsten Blank <[email protected]>
Date: 2015-08-02T16:19:41Z
Escaping for Windows:
Client.scala: possible whitespace must be escaped to an URI format.
ExecutorRunnable.scala: I was too eager to change things here.
escapeForShell must only be used once - at the end.
YarnSparkHadoopUtil.scala: As communicated in [SPARK-5754] Spark AM not
launching on Windows #7829 by vanzin, use something like CommandBuilderUtils.
commit 6b353e16982dd2693bfcdfee670f909b783bffa6
Author: Carsten Blank <[email protected]>
Date: 2015-08-02T18:46:44Z
Escaping for Windows: Test added - need help
commit 3f82e4df55f9c1d9813f65fb40925340ab61f58a
Author: Carsten Blank <[email protected]>
Date: 2015-08-02T18:53:28Z
Escaping for Windows: Ups, pushed this change, was not supposed to happen!
commit c545735136b3adbf42921895ebe9b42372adc88d
Author: Carsten Blank <[email protected]>
Date: 2015-08-02T18:59:23Z
Undoing my changes. They probably were problems related to the escaping.
commit f9704df298e5b6787bbc446430b95fd1d25860b5
Author: Carsten Blank <[email protected]>
Date: 2015-08-02T19:01:55Z
Undoing this.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
