Wangda Tan created YARN-8257:
--------------------------------
Summary: Native service should automatically adding escapes for
environment/launch cmd before sending to YARN
Key: YARN-8257
URL: https://issues.apache.org/jira/browse/YARN-8257
Project: Hadoop YARN
Issue Type: Bug
Components: yarn-native-services
Reporter: Wangda Tan
Assignee: Gour Saha
Noticed this issue while using native service:
Basically, when a string for environment / launch command contains chars like
", /, `: it needs to be escaped twice.
The first time is from json spec, because of json accept double quote only, it
needs an escape.
The second time is from launch container, what we did for command line is:
(ContainerLaunch.java)
{code:java}
line("exec /bin/bash -c \"", StringUtils.join(" ", command), "\"");{code}
And for environment:
{code:java}
line("export ", key, "=\"", value, "\"");{code}
An example of launch_command:
{code:java}
"launch_command": "export CLASSPATH=\\`\\$HADOOP_HDFS_HOME/bin/hadoop classpath
--glob\\`"{code}
And example of environment:
{code:java}
"TF_CONFIG" : "{\\\"cluster\\\": {\\\"master\\\":
[\\\"master-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"], \\\"ps\\\":
[\\\"ps-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"], \\\"worker\\\":
[\\\"worker-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"]},
\\\"task\\\": {\\\"type\\\":\\\"${COMPONENT_NAME}\\\",
\\\"index\\\":${COMPONENT_ID}}, \\\"environment\\\":\\\"cloud\\\"}",{code}
To improve usability, I think we should auto escape the input string once. (For
example, if user specified
{code}
"TF_CONFIG": "\"key\""
{code}
We will automatically escape it to:
{code}
"TF_CONFIG": \\\"key\\\"
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
