[jira] [Commented] (YARN-2857) ConcurrentModificationException in ContainerLogAppender
[
https://issues.apache.org/jira/browse/YARN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213547#comment-14213547
]
Hudson commented on YARN-2857:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #6 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/6/])
YARN-2857. ConcurrentModificationException in ContainerLogAppender. Contributed
by Mohammad Kamrul Islam (jlowe: rev f2fe8a800e5b0f3875931adba9ae20c6a95aa4ff)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLogAppender.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java
* hadoop-yarn-project/CHANGES.txt
ConcurrentModificationException in ContainerLogAppender
---
Key: YARN-2857
URL: https://issues.apache.org/jira/browse/YARN-2857
Project: Hadoop YARN
Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Priority: Critical
Fix For: 2.7.0
Attachments: ContainerLogAppender.java, MAPREDUCE-6139-test.01.patch,
MAPREDUCE-6139.1.patch, MAPREDUCE-6139.2.patch, MAPREDUCE-6139.3.patch,
YARN-2857.3.patch
Context:
* Hadoop-2.3.0
* Using Oozie 4.0.1
* Pig version 0.11.x
The job is submitted by Oozie to launch Pig script.
The following exception traces were found on MR task log:
In syslog:
{noformat}
2014-10-24 20:37:29,317 WARN [Thread-5]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook '' failed,
java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at org.apache.log4j.Hierarchy.shutdown(Hierarchy.java:471)
at org.apache.log4j.LogManager.shutdown(LogManager.java:267)
at org.apache.hadoop.mapred.TaskLog.syncLogsShutdown(TaskLog.java:286)
at org.apache.hadoop.mapred.TaskLog$2.run(TaskLog.java:339)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
2014-10-24 20:37:29,395 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
{noformat}
in stderr:
{noformat}
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:759)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
at org.apache.pig.Main.configureLog4J(Main.java:740)
at org.apache.pig.Main.run(Main.java:384)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
[jira] [Commented] (YARN-2857) ConcurrentModificationException in ContainerLogAppender
[
https://issues.apache.org/jira/browse/YARN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213538#comment-14213538
]
Hudson commented on YARN-2857:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk #744 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/744/])
YARN-2857. ConcurrentModificationException in ContainerLogAppender. Contributed
by Mohammad Kamrul Islam (jlowe: rev f2fe8a800e5b0f3875931adba9ae20c6a95aa4ff)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLogAppender.java
ConcurrentModificationException in ContainerLogAppender
---
Key: YARN-2857
URL: https://issues.apache.org/jira/browse/YARN-2857
Project: Hadoop YARN
Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Priority: Critical
Fix For: 2.7.0
Attachments: ContainerLogAppender.java, MAPREDUCE-6139-test.01.patch,
MAPREDUCE-6139.1.patch, MAPREDUCE-6139.2.patch, MAPREDUCE-6139.3.patch,
YARN-2857.3.patch
Context:
* Hadoop-2.3.0
* Using Oozie 4.0.1
* Pig version 0.11.x
The job is submitted by Oozie to launch Pig script.
The following exception traces were found on MR task log:
In syslog:
{noformat}
2014-10-24 20:37:29,317 WARN [Thread-5]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook '' failed,
java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at org.apache.log4j.Hierarchy.shutdown(Hierarchy.java:471)
at org.apache.log4j.LogManager.shutdown(LogManager.java:267)
at org.apache.hadoop.mapred.TaskLog.syncLogsShutdown(TaskLog.java:286)
at org.apache.hadoop.mapred.TaskLog$2.run(TaskLog.java:339)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
2014-10-24 20:37:29,395 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
{noformat}
in stderr:
{noformat}
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:759)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
at org.apache.pig.Main.configureLog4J(Main.java:740)
at org.apache.pig.Main.run(Main.java:384)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
[jira] [Commented] (YARN-2857) ConcurrentModificationException in ContainerLogAppender
[
https://issues.apache.org/jira/browse/YARN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213620#comment-14213620
]
Hudson commented on YARN-2857:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1934 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1934/])
YARN-2857. ConcurrentModificationException in ContainerLogAppender. Contributed
by Mohammad Kamrul Islam (jlowe: rev f2fe8a800e5b0f3875931adba9ae20c6a95aa4ff)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLogAppender.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java
ConcurrentModificationException in ContainerLogAppender
---
Key: YARN-2857
URL: https://issues.apache.org/jira/browse/YARN-2857
Project: Hadoop YARN
Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Priority: Critical
Fix For: 2.7.0
Attachments: ContainerLogAppender.java, MAPREDUCE-6139-test.01.patch,
MAPREDUCE-6139.1.patch, MAPREDUCE-6139.2.patch, MAPREDUCE-6139.3.patch,
YARN-2857.3.patch
Context:
* Hadoop-2.3.0
* Using Oozie 4.0.1
* Pig version 0.11.x
The job is submitted by Oozie to launch Pig script.
The following exception traces were found on MR task log:
In syslog:
{noformat}
2014-10-24 20:37:29,317 WARN [Thread-5]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook '' failed,
java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at org.apache.log4j.Hierarchy.shutdown(Hierarchy.java:471)
at org.apache.log4j.LogManager.shutdown(LogManager.java:267)
at org.apache.hadoop.mapred.TaskLog.syncLogsShutdown(TaskLog.java:286)
at org.apache.hadoop.mapred.TaskLog$2.run(TaskLog.java:339)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
2014-10-24 20:37:29,395 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
{noformat}
in stderr:
{noformat}
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:759)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
at org.apache.pig.Main.configureLog4J(Main.java:740)
at org.apache.pig.Main.run(Main.java:384)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
[jira] [Commented] (YARN-2857) ConcurrentModificationException in ContainerLogAppender
[
https://issues.apache.org/jira/browse/YARN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213629#comment-14213629
]
Hudson commented on YARN-2857:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #6 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/6/])
YARN-2857. ConcurrentModificationException in ContainerLogAppender. Contributed
by Mohammad Kamrul Islam (jlowe: rev f2fe8a800e5b0f3875931adba9ae20c6a95aa4ff)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLogAppender.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java
ConcurrentModificationException in ContainerLogAppender
---
Key: YARN-2857
URL: https://issues.apache.org/jira/browse/YARN-2857
Project: Hadoop YARN
Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Priority: Critical
Fix For: 2.7.0
Attachments: ContainerLogAppender.java, MAPREDUCE-6139-test.01.patch,
MAPREDUCE-6139.1.patch, MAPREDUCE-6139.2.patch, MAPREDUCE-6139.3.patch,
YARN-2857.3.patch
Context:
* Hadoop-2.3.0
* Using Oozie 4.0.1
* Pig version 0.11.x
The job is submitted by Oozie to launch Pig script.
The following exception traces were found on MR task log:
In syslog:
{noformat}
2014-10-24 20:37:29,317 WARN [Thread-5]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook '' failed,
java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at org.apache.log4j.Hierarchy.shutdown(Hierarchy.java:471)
at org.apache.log4j.LogManager.shutdown(LogManager.java:267)
at org.apache.hadoop.mapred.TaskLog.syncLogsShutdown(TaskLog.java:286)
at org.apache.hadoop.mapred.TaskLog$2.run(TaskLog.java:339)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
2014-10-24 20:37:29,395 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
{noformat}
in stderr:
{noformat}
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:759)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
at org.apache.pig.Main.configureLog4J(Main.java:740)
at org.apache.pig.Main.run(Main.java:384)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
[jira] [Commented] (YARN-2857) ConcurrentModificationException in ContainerLogAppender
[
https://issues.apache.org/jira/browse/YARN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213648#comment-14213648
]
Hudson commented on YARN-2857:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1958 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1958/])
YARN-2857. ConcurrentModificationException in ContainerLogAppender. Contributed
by Mohammad Kamrul Islam (jlowe: rev f2fe8a800e5b0f3875931adba9ae20c6a95aa4ff)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLogAppender.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java
* hadoop-yarn-project/CHANGES.txt
ConcurrentModificationException in ContainerLogAppender
---
Key: YARN-2857
URL: https://issues.apache.org/jira/browse/YARN-2857
Project: Hadoop YARN
Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Priority: Critical
Fix For: 2.7.0
Attachments: ContainerLogAppender.java, MAPREDUCE-6139-test.01.patch,
MAPREDUCE-6139.1.patch, MAPREDUCE-6139.2.patch, MAPREDUCE-6139.3.patch,
YARN-2857.3.patch
Context:
* Hadoop-2.3.0
* Using Oozie 4.0.1
* Pig version 0.11.x
The job is submitted by Oozie to launch Pig script.
The following exception traces were found on MR task log:
In syslog:
{noformat}
2014-10-24 20:37:29,317 WARN [Thread-5]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook '' failed,
java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at org.apache.log4j.Hierarchy.shutdown(Hierarchy.java:471)
at org.apache.log4j.LogManager.shutdown(LogManager.java:267)
at org.apache.hadoop.mapred.TaskLog.syncLogsShutdown(TaskLog.java:286)
at org.apache.hadoop.mapred.TaskLog$2.run(TaskLog.java:339)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
2014-10-24 20:37:29,395 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
{noformat}
in stderr:
{noformat}
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:759)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
at org.apache.pig.Main.configureLog4J(Main.java:740)
at org.apache.pig.Main.run(Main.java:384)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
[jira] [Commented] (YARN-2857) ConcurrentModificationException in ContainerLogAppender
[
https://issues.apache.org/jira/browse/YARN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213657#comment-14213657
]
Hudson commented on YARN-2857:
--
SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #6 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/6/])
YARN-2857. ConcurrentModificationException in ContainerLogAppender. Contributed
by Mohammad Kamrul Islam (jlowe: rev f2fe8a800e5b0f3875931adba9ae20c6a95aa4ff)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLogAppender.java
* hadoop-yarn-project/CHANGES.txt
ConcurrentModificationException in ContainerLogAppender
---
Key: YARN-2857
URL: https://issues.apache.org/jira/browse/YARN-2857
Project: Hadoop YARN
Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Priority: Critical
Fix For: 2.7.0
Attachments: ContainerLogAppender.java, MAPREDUCE-6139-test.01.patch,
MAPREDUCE-6139.1.patch, MAPREDUCE-6139.2.patch, MAPREDUCE-6139.3.patch,
YARN-2857.3.patch
Context:
* Hadoop-2.3.0
* Using Oozie 4.0.1
* Pig version 0.11.x
The job is submitted by Oozie to launch Pig script.
The following exception traces were found on MR task log:
In syslog:
{noformat}
2014-10-24 20:37:29,317 WARN [Thread-5]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook '' failed,
java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at org.apache.log4j.Hierarchy.shutdown(Hierarchy.java:471)
at org.apache.log4j.LogManager.shutdown(LogManager.java:267)
at org.apache.hadoop.mapred.TaskLog.syncLogsShutdown(TaskLog.java:286)
at org.apache.hadoop.mapred.TaskLog$2.run(TaskLog.java:339)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
2014-10-24 20:37:29,395 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
{noformat}
in stderr:
{noformat}
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:759)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
at org.apache.pig.Main.configureLog4J(Main.java:740)
at org.apache.pig.Main.run(Main.java:384)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
[jira] [Commented] (YARN-2857) ConcurrentModificationException in ContainerLogAppender
[
https://issues.apache.org/jira/browse/YARN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212792#comment-14212792
]
Jason Lowe commented on YARN-2857:
--
+1 lgtm. Committing this.
ConcurrentModificationException in ContainerLogAppender
---
Key: YARN-2857
URL: https://issues.apache.org/jira/browse/YARN-2857
Project: Hadoop YARN
Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Priority: Critical
Attachments: ContainerLogAppender.java, MAPREDUCE-6139-test.01.patch,
MAPREDUCE-6139.1.patch, MAPREDUCE-6139.2.patch, MAPREDUCE-6139.3.patch,
YARN-2857.3.patch
Context:
* Hadoop-2.3.0
* Using Oozie 4.0.1
* Pig version 0.11.x
The job is submitted by Oozie to launch Pig script.
The following exception traces were found on MR task log:
In syslog:
{noformat}
2014-10-24 20:37:29,317 WARN [Thread-5]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook '' failed,
java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at org.apache.log4j.Hierarchy.shutdown(Hierarchy.java:471)
at org.apache.log4j.LogManager.shutdown(LogManager.java:267)
at org.apache.hadoop.mapred.TaskLog.syncLogsShutdown(TaskLog.java:286)
at org.apache.hadoop.mapred.TaskLog$2.run(TaskLog.java:339)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
2014-10-24 20:37:29,395 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
{noformat}
in stderr:
{noformat}
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:759)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
at org.apache.pig.Main.configureLog4J(Main.java:740)
at org.apache.pig.Main.run(Main.java:384)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2857) ConcurrentModificationException in ContainerLogAppender
[
https://issues.apache.org/jira/browse/YARN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212810#comment-14212810
]
Hudson commented on YARN-2857:
--
FAILURE: Integrated in Hadoop-trunk-Commit #6545 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/6545/])
YARN-2857. ConcurrentModificationException in ContainerLogAppender. Contributed
by Mohammad Kamrul Islam (jlowe: rev f2fe8a800e5b0f3875931adba9ae20c6a95aa4ff)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLogAppender.java
ConcurrentModificationException in ContainerLogAppender
---
Key: YARN-2857
URL: https://issues.apache.org/jira/browse/YARN-2857
Project: Hadoop YARN
Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Priority: Critical
Fix For: 2.7.0
Attachments: ContainerLogAppender.java, MAPREDUCE-6139-test.01.patch,
MAPREDUCE-6139.1.patch, MAPREDUCE-6139.2.patch, MAPREDUCE-6139.3.patch,
YARN-2857.3.patch
Context:
* Hadoop-2.3.0
* Using Oozie 4.0.1
* Pig version 0.11.x
The job is submitted by Oozie to launch Pig script.
The following exception traces were found on MR task log:
In syslog:
{noformat}
2014-10-24 20:37:29,317 WARN [Thread-5]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook '' failed,
java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at org.apache.log4j.Hierarchy.shutdown(Hierarchy.java:471)
at org.apache.log4j.LogManager.shutdown(LogManager.java:267)
at org.apache.hadoop.mapred.TaskLog.syncLogsShutdown(TaskLog.java:286)
at org.apache.hadoop.mapred.TaskLog$2.run(TaskLog.java:339)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
2014-10-24 20:37:29,395 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
{noformat}
in stderr:
{noformat}
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:759)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
at org.apache.pig.Main.configureLog4J(Main.java:740)
at org.apache.pig.Main.run(Main.java:384)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
[jira] [Commented] (YARN-2857) ConcurrentModificationException in ContainerLogAppender
[
https://issues.apache.org/jira/browse/YARN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14211457#comment-14211457
]
Hadoop QA commented on YARN-2857:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12681381/YARN-2857.3.patch
against trunk revision 1a2e5cb.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5839//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5839//console
This message is automatically generated.
ConcurrentModificationException in ContainerLogAppender
---
Key: YARN-2857
URL: https://issues.apache.org/jira/browse/YARN-2857
Project: Hadoop YARN
Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Priority: Critical
Attachments: ContainerLogAppender.java, MAPREDUCE-6139-test.01.patch,
MAPREDUCE-6139.1.patch, MAPREDUCE-6139.2.patch, MAPREDUCE-6139.3.patch,
YARN-2857.3.patch
Context:
* Hadoop-2.3.0
* Using Oozie 4.0.1
* Pig version 0.11.x
The job is submitted by Oozie to launch Pig script.
The following exception traces were found on MR task log:
In syslog:
{noformat}
2014-10-24 20:37:29,317 WARN [Thread-5]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook '' failed,
java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at org.apache.log4j.Hierarchy.shutdown(Hierarchy.java:471)
at org.apache.log4j.LogManager.shutdown(LogManager.java:267)
at org.apache.hadoop.mapred.TaskLog.syncLogsShutdown(TaskLog.java:286)
at org.apache.hadoop.mapred.TaskLog$2.run(TaskLog.java:339)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
2014-10-24 20:37:29,395 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
{noformat}
in stderr:
{noformat}
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:759)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
at org.apache.pig.Main.configureLog4J(Main.java:740)
at org.apache.pig.Main.run(Main.java:384)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
[jira] [Commented] (YARN-2857) ConcurrentModificationException in ContainerLogAppender
[
https://issues.apache.org/jira/browse/YARN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14211519#comment-14211519
]
Gera Shegalov commented on YARN-2857:
-
Clean Jenkins in the last build demonstrates that the patch fixes the
reproducer in the previous build:
{code}
testAppendInClose(org.apache.hadoop.yarn.TestContainerLogAppender) Time
elapsed: 0.066 sec ERROR!
java.util.ConcurrentModificationException: null
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:761)
at java.util.LinkedList$ListItr.next(LinkedList.java:696)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:81)
at
org.apache.hadoop.yarn.TestContainerLogAppender.testAppendInClose(TestContainerLogAppender.java:44)
{code}
. +1(non-binding)
ConcurrentModificationException in ContainerLogAppender
---
Key: YARN-2857
URL: https://issues.apache.org/jira/browse/YARN-2857
Project: Hadoop YARN
Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Priority: Critical
Attachments: ContainerLogAppender.java, MAPREDUCE-6139-test.01.patch,
MAPREDUCE-6139.1.patch, MAPREDUCE-6139.2.patch, MAPREDUCE-6139.3.patch,
YARN-2857.3.patch
Context:
* Hadoop-2.3.0
* Using Oozie 4.0.1
* Pig version 0.11.x
The job is submitted by Oozie to launch Pig script.
The following exception traces were found on MR task log:
In syslog:
{noformat}
2014-10-24 20:37:29,317 WARN [Thread-5]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook '' failed,
java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at org.apache.log4j.Hierarchy.shutdown(Hierarchy.java:471)
at org.apache.log4j.LogManager.shutdown(LogManager.java:267)
at org.apache.hadoop.mapred.TaskLog.syncLogsShutdown(TaskLog.java:286)
at org.apache.hadoop.mapred.TaskLog$2.run(TaskLog.java:339)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
2014-10-24 20:37:29,395 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
{noformat}
in stderr:
{noformat}
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:759)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
at org.apache.pig.Main.configureLog4J(Main.java:740)
at org.apache.pig.Main.run(Main.java:384)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
[jira] [Commented] (YARN-2857) ConcurrentModificationException in ContainerLogAppender
[
https://issues.apache.org/jira/browse/YARN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14211574#comment-14211574
]
Mohammad Kamrul Islam commented on YARN-2857:
-
Can some (binding) people please review it?
ConcurrentModificationException in ContainerLogAppender
---
Key: YARN-2857
URL: https://issues.apache.org/jira/browse/YARN-2857
Project: Hadoop YARN
Issue Type: Bug
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Priority: Critical
Attachments: ContainerLogAppender.java, MAPREDUCE-6139-test.01.patch,
MAPREDUCE-6139.1.patch, MAPREDUCE-6139.2.patch, MAPREDUCE-6139.3.patch,
YARN-2857.3.patch
Context:
* Hadoop-2.3.0
* Using Oozie 4.0.1
* Pig version 0.11.x
The job is submitted by Oozie to launch Pig script.
The following exception traces were found on MR task log:
In syslog:
{noformat}
2014-10-24 20:37:29,317 WARN [Thread-5]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook '' failed,
java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at org.apache.log4j.Hierarchy.shutdown(Hierarchy.java:471)
at org.apache.log4j.LogManager.shutdown(LogManager.java:267)
at org.apache.hadoop.mapred.TaskLog.syncLogsShutdown(TaskLog.java:286)
at org.apache.hadoop.mapred.TaskLog$2.run(TaskLog.java:339)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
2014-10-24 20:37:29,395 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
{noformat}
in stderr:
{noformat}
java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.yarn.ContainerLogAppender.close(ContainerLogAppender.java:94)
at
org.apache.log4j.helpers.AppenderAttachableImpl.removeAllAppenders(AppenderAttachableImpl.java:141)
at org.apache.log4j.Category.removeAllAppenders(Category.java:891)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:759)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
at org.apache.pig.Main.configureLog4J(Main.java:740)
at org.apache.pig.Main.run(Main.java:384)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
