[
https://issues.apache.org/jira/browse/PIG-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15213914#comment-15213914
]
liyunzhang_intel commented on PIG-4837:
---------------------------------------
When i check the code, there is no change about TestNativeMapReduce except we
update the hadoop version from 2.5 to 2.6 in ivy/libraries.properties
{code}
hadoop-common.version=2.6.0
hadoop-hdfs.version=2.6.0
hadoop-mapreduce.version=2.6.0
{code}
The error is thrown when yarn executes container-launch.sh and seems that it
exceed the bash max limitation.
{code}
Exception from container-launch.
Container id: container_1458599533105_0001_01_000004
Exit code: 0
Exception message: Cannot run program "bash" (in directory
"/x1/jenkins/jenkins-slave/workspace/Pig-spark/target/PigMiniCluster/PigMiniCluster-localDir-nm-0_0/usercache/jenkins/appcache/application_1458599533105_0001/container_1458599533105_0001_01_000004"):
error=7, Argument list too long
Stack trace: java.io.IOException: Cannot run program "bash" (in directory
"/x1/jenkins/jenkins-slave/workspace/Pig-spark/target/PigMiniCluster/PigMiniCluster-localDir-nm-0_0/usercache/jenkins/appcache/application_1458599533105_0001/container_1458599533105_0001_01_000004"):
error=7, Argument list too long
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:485)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.IOException: error=7, Argument list too long
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:135)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
... 10 more
{code}
The container-launcher.sh is like:
{code}
#!/bin/bash
export HADOOP_CONF_DIR="/home/zly/prj/oss/hadoop-2.6.0/etc/hadoop"
export MAX_APP_ATTEMPTS="2"
export JAVA_HOME="/usr/java/jdk1.8.0_20"
export APP_SUBMIT_TIME_ENV="1458808716291"
export NM_HOST="zly1.sh.intel.com"
export LD_LIBRARY_PATH="$PWD"
export HADOOP_HDFS_HOME="/home/zly/prj/oss/hadoop-2.6.0"
export LOGNAME="root"
export JVM_PID="$$"
export
PWD="/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1458806299263_0002/container_1458806299263_0002_01_000001"
export HADOOP_COMMON_HOME="/home/zly/prj/oss/hadoop-2.6.0"
export
LOCAL_DIRS="/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1458806299263_0002"
export APPLICATION_WEB_PROXY_BASE="/proxy/application_1458806299263_0002"
export SHELL="/bin/bash"
export NM_HTTP_PORT="8042"
export
LOG_DIRS="/home/zly/prj/oss/hadoop-2.6.0/logs/userlogs/application_1458806299263_0002/container_1458806299263_0002_01_000001"
export
NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
"
export NM_PORT="34884"
export USER="root"
export HADOOP_YARN_HOME="/home/zly/prj/oss/hadoop-2.6.0"
export
CLASSPATH="$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:$PWD/*"
export
HADOOP_TOKEN_FILE_LOCATION="/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1458806299263_0002/container_1458806299263_0002_01_000001/container_tokens"
export HOME="/home/"
export CONTAINER_ID="container_1458806299263_0002_01_000001"
export MALLOC_ARENA_MAX="4"
ln -sf
"/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1458806299263_0002/filecache/13/job.xml"
"job.xml"
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
mkdir -p jobSubmitDir
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
ln -sf
"/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1458806299263_0002/filecache/12/job.split"
"jobSubmitDir/job.split"
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
mkdir -p jobSubmitDir
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
ln -sf
"/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1458806299263_0002/filecache/10/job.splitmetainfo"
"jobSubmitDir/job.splitmetainfo"
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
ln -sf
"/tmp/hadoop-root/nm-local-dir/usercache/root/filecache/11/TestNMapReduceStopwFile"
"TestNMapReduceStopwFile"
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
ln -sf
"/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1458806299263_0002/filecache/11/job.jar"
"job.jar"
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
exec /bin/bash -c "$JAVA_HOME/bin/java
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.container.log.dir=/home/zly/prj/oss/hadoop-2.6.0/logs/userlogs/application_1458806299263_0002/container_1458806299263_0002_01_000001
-Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m
org.apache.hadoop.mapreduce.v2.app.MRAppMaster
1>/home/zly/prj/oss/hadoop-2.6.0/logs/userlogs/application_1458806299263_0002/container_1458806299263_0002_01_000001/stdout
2>/home/zly/prj/oss/hadoop-2.6.0/logs/userlogs/application_1458806299263_0002/container_1458806299263_0002_01_000001/stderr
"
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
{code}
The limit length of bash is: $(getconf ARG_MAX) - $(env | wc -c). The ARG_MAX
of the jenkins build server is 2097152. It seem that it is not very small to
throw the exception.
> TestNativeMapReduce test fix
> ----------------------------
>
> Key: PIG-4837
> URL: https://issues.apache.org/jira/browse/PIG-4837
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: liyunzhang_intel
> Assignee: liyunzhang_intel
> Fix For: spark-branch
>
> Attachments: PIG-4837.patch, build23.PNG
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
