GitHub user taoli91 opened a pull request:
https://github.com/apache/spark/pull/12694
[SPARK-14914] Fix Command too long for windows. Especially for test cases.
## What changes were proposed in this pull request?
Windows has a limitation of 8192 characters on command line. This
limitation will fail the test cases related to command builder in some test
cases. This pull request adopt the method in Hadoop, which create a Jar that
ref all the dependencies in the manifest to shorten the class path passing to
the command line.
## How was this patch tested?
Unit tests on windows and Linux. Note that this commit can't fix all the
unit test errors on windows.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/taoli91/spark fix_cmd_too_long
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12694.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12694
----
commit 2c1b9e28986c32d68e17d99928ace4a620c59f60
Author: U-FAREAST\tl <[email protected]>
Date: 2016-04-25T07:23:12Z
Fix Executor's Command too long problem
commit 79560bc2ccc4ce920982a72c7d1ec7568d0409f6
Author: U-FAREAST\tl <[email protected]>
Date: 2016-04-25T07:33:43Z
Fix Spark Launcher Suite
commit d092ef58bc889ec19bb5e4064cb751da9e58812c
Author: U-FAREAST\tl <[email protected]>
Date: 2016-04-25T07:48:39Z
Fix another too long
commit 6f8671ee6264ddcb0b45d395f4a5f911ec1e52e2
Author: Tao LI <[email protected]>
Date: 2016-04-25T08:18:11Z
Fix Abstract cmd builder
commit efc0df0a93b668829a67afcbf0f2bd2ed3e60450
Author: U-FAREAST\tl <[email protected]>
Date: 2016-04-25T08:47:00Z
Remove normalize path
commit df0f5dd67196cb8aa4deba49eca254a3dafe8e1f
Author: U-FAREAST\tl <[email protected]>
Date: 2016-04-25T09:11:07Z
Minor format fix
commit e26d9495a9d8f8f237c2328f3028185446b5370f
Author: U-FAREAST\tl <[email protected]>
Date: 2016-04-25T10:17:00Z
fix spark-submit.cmd
commit 4a7fb864e337db5b395d092a94085bbf92853c8d
Author: U-FAREAST\tl <[email protected]>
Date: 2016-04-25T10:47:12Z
Fix shorten class path
commit d40d61a04628c5729c3ebc10c46ec525bd77a0eb
Author: U-FAREAST\tl <[email protected]>
Date: 2016-04-26T03:38:26Z
Handle null files on create jar
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
