Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/7872#discussion_r36244931
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -244,6 +253,47 @@ object YarnSparkHadoopUtil {
}
}
+ /**
+ * Escapes a string for inclusion in a command line executed by Yarn.
Yarn executes commands
+ * using a batch file (.cmd) Windows escaping is neccessary. The
+ * argument is enclosed in single quotes only if there are spaces inside
the string.
+ * Also some key characters are escaped that have a meaning in Windows.
+ *
+ * @param arg A single argument.
+ * @return Argument quoted for execution via Yarn's generated shell
script.
+ */
+ def escapeForShellWindows(arg: String): String = {
+ var needsQuotes = false
+ var i: Int = 0
+ while (i < arg.length && !needsQuotes) {
+ val c = arg.charAt(i)
+ if (Character.isWhitespace(c) || c == '"' || c == '=' || c == ',' ||
c == ';' || c == '\'') {
+ needsQuotes = true
+ }
+ i += 1
+ }
+ if (!needsQuotes) {
+ return arg
+ }
+
+ val quoted: StringBuilder = new StringBuilder("\"")
+ for (i <- 0 to arg.length() - 1) {
+ arg.charAt(i) match {
+ case '"' =>
+ quoted.append("\"\"")
+ case '%' =>
--- End diff --
Is there a reason why this change shouldn't be done in
`CommandBuilderUtils`?
What happens if you try to run something like `bin\spark-shell.cmd --master
local --name "%abcde"`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
