[
https://issues.apache.org/jira/browse/PIG-5105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822327#comment-15822327
]
Rohini Palaniswamy commented on PIG-5105:
-----------------------------------------
This issue is same as PIG-4837. I also ran into this when running the unit
tests in parallel with a Jenkins Matrix multi-configuration job
(https://wiki.jenkins-ci.org/display/JENKINS/Building+a+matrix+project). When I
just had a slave axis with preselected nodes, it used to be fine. When I
transitioned the slave axis to use a node label and added a User configuration
axis SLAVE_ID to control the parallelism, ran into this issue. This was due to
two issues
- Each dependency jar was repeated 9 times in launcher_container.sh.
- In addition when the basedir path also became longer (due to jenkins
multi-configuration job) the CLASSPATH exceeded the bash argument limit for
TestNativeMapreduce and TestPredeployedJar. For eg:
/home/jenkins/workspace/Rohini-PigCI-v0.14-Parallel-UnitTests/SLAVE_ID/2/SLAVE_NAME/rhel6/ivy2/cache/asm/asm/jars/asm-3.3.1.jar
/home/jenkins/workspace/Rohini-PigCI-v0.14-Parallel-UnitTests/SLAVE_ID/2/SLAVE_NAME/rhel6/ivy2/cache/asm/asm/jars/asm-3.3.1.jar
/home/jenkins/workspace/Rohini-PigCI-v0.14-Parallel-UnitTests/SLAVE_ID/2/SLAVE_NAME/rhel6/ivy2/cache/asm/asm/jars/asm-3.3.1.jar
/home/jenkins/workspace/Rohini-PigCI-v0.14-Parallel-UnitTests/SLAVE_ID/2/SLAVE_NAME/rhel6/pig/build/ivy/lib/Pig/asm-3.3.1.jar
/home/jenkins/workspace/Rohini-PigCI-v0.14-Parallel-UnitTests/SLAVE_ID/2/SLAVE_NAME/rhel6/pig/build/ivy/lib/Pig/asm-3.3.1.jar
/home/jenkins/workspace/Rohini-PigCI-v0.14-Parallel-UnitTests/SLAVE_ID/2/SLAVE_NAME/rhel6/pig/build/ivy/lib/Pig/asm-3.3.1.jar
/home/jenkins/workspace/Rohini-PigCI-v0.14-Parallel-UnitTests/SLAVE_ID/2/SLAVE_NAME/rhel6/pig/lib/asm-3.3.1.jar
/home/jenkins/workspace/Rohini-PigCI-v0.14-Parallel-UnitTests/SLAVE_ID/2/SLAVE_NAME/rhel6/pig/lib/asm-3.3.1.jar
/home/jenkins/workspace/Rohini-PigCI-v0.14-Parallel-UnitTests/SLAVE_ID/2/SLAVE_NAME/rhel6/pig/lib/asm-3.3.1.jar
I am guessing most likely you are running into this now either because you have
a very long Jenkins job name or PIG-4728 might have added more jars from hbase
to go over the limit.
> Tez unit tests failing with "Argument list too long"
> ----------------------------------------------------
>
> Key: PIG-5105
> URL: https://issues.apache.org/jira/browse/PIG-5105
> Project: Pig
> Issue Type: Test
> Reporter: Koji Noguchi
> Assignee: Rohini Palaniswamy
>
> 5 Tests failed with the same error.
> * org.apache.pig.test.TestNativeMapReduce.testNativeMRJobTypeCastInserter
> * org.apache.pig.test.TestNativeMapReduce.testNativeMRJobSimple
> * org.apache.pig.test.TestNativeMapReduce.testNativeMRJobMultiStoreOnPred
> * org.apache.pig.test.TestNativeMapReduce.testNativeMRJobMultiQueryOpt
> * org.apache.pig.test.TestPredeployedJar.testPredeployedJar
> {noformat}
> 2017-01-12 01:06:12,996 [ContainersLauncher #0] WARN
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor -
> Exception from container-launch with container ID:
> container_1484183112208_0002_01_000002 and exit code: 126
> ExitCodeException exitCode=126: ./launch_container.sh: line 30: /bin/ln:
> Argument list too long
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)
> at org.apache.hadoop.util.Shell.run(Shell.java:479)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:773)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
