[ https://issues.apache.org/jira/browse/YARN-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16660947#comment-16660947 ]
Eric Yang commented on YARN-8913: --------------------------------- Launch command looks like this in patch 002: {code} cd /tmp/mawo/hadoop-applications-mawo-0.0.1-SNAPSHOT/bin;export JAVA_HOME=/usr/lib/jvm/java-1.8.0;/tmp/mawo/hadoop-applications-mawo-0.0.1-SNAPSHOT/bin/mawo --config /tmp/mawo-config master -job_descripiton_file_path /tmp/mawo/mawo-payload.json -mawo_app_name mawo-hadoop-ut {code} This is trying to do 3 things at the same time. # Change working directory to /tmp/mawo/hadoop-applications-mawo-0.0.1-SNAPSHOT/bin # Set JAVA_HOME environment variable # Run Mawo script with configuration parameters It will greatly simplify the complexity by using an entry-point.sh to bootstrap. Mawo script can source mawo-env.sh to locate JAVA_HOME location like hadoop's libexec/hadoop-config.sh. The working directory can be computed from basedir $0/.. in the entry-point.sh. This will change the launch command to call a fixed entry-point.sh to improve readability and prevent path traversal security exploits in the container. > Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop > Cluster > -------------------------------------------------------------------------------- > > Key: YARN-8913 > URL: https://issues.apache.org/jira/browse/YARN-8913 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Yesha Vora > Assignee: Yesha Vora > Priority: Blocker > Attachments: YARN-8913.001.patch, YARN-8913.002.patch > > > MaWo application can be used to run Hadoop UT faster in a Hadoop cluster. > Develop helper scripts to orchestrate end-to-end workflow for running Hadoop > UT using MaWo app. > Pre-requisite: > * A Hadoop Cluster with HDFS and YARN installed > * Enable Docker on YARN feature > > Helper-scripts > * MaWo_Driver > ** create a docker image with latest hadoop source code > ** create payload to MaWo app (This is input to mawo app where Each MaWo > Task = UT execution of each Hadoop Module) > ** Upload payload file to HDFS > ** Update MaWo-Launch.json to resolve RM_HOST / Docker Image etc dynamically > ** Launch MaWo app in Hadoop cluster -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org