GitHub user vanzin opened a pull request:
https://github.com/apache/spark/pull/1724
[SPARK-2718] [yarn] Handle quotes and other characters in user args.
Due to the way Yarn runs things through bash, normal quoting doesn't
work as expected. This change applies the necessary voodoo to the user
args to avoid issues with bash and special characters.
The change also uncovered an issue with the event logger app name
sanitizing code; it wasn't cleaning up all "bad" characters, so
sometimes it would fail to create the log dirs. I just added some
more bad character replacements.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vanzin/spark SPARK-2718
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1724.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1724
----
commit 515613d759f01a81bc39a1bf230094ae6f012d6b
Author: Marcelo Vanzin <[email protected]>
Date: 2014-08-01T22:12:17Z
[SPARK-2718] [yarn] Handle quotes and other characters in user args.
Due to the way Yarn runs things through bash, normal quoting doesn't
work as expected. This change applies the necessary voodoo to the user
args to avoid issues with bash and special characters.
The change also uncovered an issue with the event logger app name
sanitizing code; it wasn't cleaning up all "bad" characters, so
sometimes it would fail to create the log dirs. I just added some
more bad character replacements.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
