[ https://issues.apache.org/jira/browse/YARN-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089423#comment-17089423 ]
Gabor Bota commented on YARN-8257: ---------------------------------- The hadoop release 3.1.4 code freeze is today (https://cwiki.apache.org/confluence/display/HADOOP/Roadmap). Please close this issue today or move it to a different target branch. Thank you! > Native service should automatically adding escapes for environment/launch cmd > before sending to YARN > ---------------------------------------------------------------------------------------------------- > > Key: YARN-8257 > URL: https://issues.apache.org/jira/browse/YARN-8257 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-native-services > Reporter: Wangda Tan > Assignee: Gour Saha > Priority: Critical > > Noticed this issue while using native service: > Basically, when a string for environment / launch command contains chars like > ", /, `: it needs to be escaped twice. > The first time is from json spec, because of json accept double quote only, > it needs an escape. > The second time is from launch container, what we did for command line is: > (ContainerLaunch.java) > {code:java} > line("exec /bin/bash -c \"", StringUtils.join(" ", command), "\"");{code} > And for environment: > {code:java} > line("export ", key, "=\"", value, "\"");{code} > An example of launch_command: > {code:java} > "launch_command": "export CLASSPATH=\\`\\$HADOOP_HDFS_HOME/bin/hadoop > classpath --glob\\`"{code} > And example of environment: > {code:java} > "TF_CONFIG" : "{\\\"cluster\\\": {\\\"master\\\": > [\\\"master-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"], \\\"ps\\\": > [\\\"ps-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"], \\\"worker\\\": > [\\\"worker-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"]}, > \\\"task\\\": {\\\"type\\\":\\\"${COMPONENT_NAME}\\\", > \\\"index\\\":${COMPONENT_ID}}, \\\"environment\\\":\\\"cloud\\\"}",{code} > To improve usability, I think we should auto escape the input string once. > (For example, if user specified > {code} > "TF_CONFIG": "\"key\"" > {code} > We will automatically escape it to: > {code} > "TF_CONFIG": \\\"key\\\" > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org