[
https://issues.apache.org/jira/browse/YARN-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14218160#comment-14218160
]
Tim Robertson commented on YARN-2875:
-------------------------------------
Sadly no.
It is used in the
[ContainerLogAppender|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java#L37]
and
[ContainerRollingLogAppender|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerRollingLogAppender.java#L34].
I tried to remove it and compile using the [log4j-over-slf4j v1.7.7
bridge|http://search.maven.org/#artifactdetails%7Corg.slf4j%7Clog4j-over-slf4j%7C1.7.7%7Cjar]
but that fails because the SLF4J classes are not the same API. For example
[the SLF4J RollingFileAppender|
https://github.com/qos-ch/slf4j/blob/master/log4j-over-slf4j/src/main/java/org/apache/log4j/RollingFileAppender.java]
does not implement methods like setFile(), setAppend() etc. The build will
fail with the following:
{code}
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR]
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerRollingLogAppender.java:[41,6]
error: cannot find symbol
[ERROR] symbol: method setFile(String)
location: class ContainerRollingLogAppender
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerRollingLogAppender.java:[42,6]
error: cannot find symbol
[ERROR] symbol: method setAppend(boolean)
location: class ContainerRollingLogAppender
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerRollingLogAppender.java:[43,11]
error: cannot find symbol
[ERROR] symbol: method activateOptions()
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerRollingLogAppender.java:[38,2]
error: method does not override or implement a method from a supertype
[ERROR]
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerRollingLogAppender.java:[49,8]
error: cannot find symbol
[ERROR] symbol: variable qw
location: class ContainerRollingLogAppender
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerRollingLogAppender.java:[50,6]
error: cannot find symbol
[ERROR] symbol: variable qw
location: class ContainerRollingLogAppender
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java:[37,7]
error: no suitable constructor found for FileAppender()
[ERROR] constructor
FileAppender.FileAppender(Layout,String,boolean,boolean,int) is not applicable
(actual and formal argument lists differ in length)
constructor FileAppender.FileAppender(Layout,String,boolean) is not
applicable
(actual and formal argument lists differ in length)
constructor FileAppender.FileAppender(Layout,String) is not applicable
(actual and formal argument lists differ in length)
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java:[52,6]
error: cannot find symbol
[ERROR] symbol: method setFile(String)
location: class ContainerLogAppender
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java:[53,6]
error: cannot find symbol
[ERROR] symbol: method setAppend(boolean)
location: class ContainerLogAppender
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java:[65,13]
error: cannot find symbol
[ERROR] symbol: method append(LoggingEvent)
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java:[58,2]
error: method does not override or implement a method from a supertype
[ERROR]
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java:[77,8]
error: cannot find symbol
[ERROR] symbol: variable qw
location: class ContainerLogAppender
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java:[78,6]
error: cannot find symbol
[ERROR] symbol: variable qw
location: class ContainerLogAppender
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java:[87,13]
error: cannot find symbol
[ERROR] symbol: method append(LoggingEvent)
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java:[90,9]
error: cannot find symbol
[ERROR] symbol: method close()
/Users/tim/dev/git/hadoop/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ContainerLogAppender.java:[82,2]
error: method does not override or implement a method from a supertype
[INFO] 16 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
{code}
> Bump SLF4J to 1.7.7 from 1.7.5
> -------------------------------
>
> Key: YARN-2875
> URL: https://issues.apache.org/jira/browse/YARN-2875
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Tim Robertson
> Priority: Minor
>
> hadoop-yarn-common [uses log4j
> directly|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/pom.xml#L167]
> and when trying to redirect that through an SLF4J bridge version 1.7.5 has
> issues, due to use of AppenderSkeleton which is missing in log4j-over-slf4j
> version 1.7.5.
> This is documented on the [1.7.6 release
> notes|http://www.slf4j.org/news.html] but 1.7.7 should be suitable.
> This is applicable to all the projects using Hadoop motherpom, but Yarn
> appears to be bringing Log4J in, rather than coding to the SLF4J API.
> The issue shows in the logs as follows in Yarn MR apps, which is painful to
> diagnose.
> {code}
> WARN [2014-11-18 09:58:06,390+0100] [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Caught exception in
> callback postStart
> java.lang.reflect.InvocationTargetException: null
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.7.0_71]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ~[na:1.7.0_71]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.7.0_71]
> at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_71]
> at
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:290)
> ~[job.jar:0.22-SNAPSHOT]
> at com.sun.proxy.$Proxy2.postStart(Unknown Source) [na:na]
> at
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:185)
> [job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:157)
> [job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:54)
> [job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)
> [job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1036)
> [job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> [job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1478)
> [job.jar:0.22-SNAPSHOT]
> at java.security.AccessController.doPrivileged(Native Method)
> [na:1.7.0_71]
> at javax.security.auth.Subject.doAs(Subject.java:415) [na:1.7.0_71]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
> [job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1474)
> [job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1407)
> [job.jar:0.22-SNAPSHOT]
> Caused by: java.lang.IncompatibleClassChangeError: Implementing class
> at java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.7.0_71]
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> ~[na:1.7.0_71]
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> ~[na:1.7.0_71]
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> ~[na:1.7.0_71]
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> ~[na:1.7.0_71]
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) ~[na:1.7.0_71]
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_71]
> at java.security.AccessController.doPrivileged(Native Method)
> [na:1.7.0_71]
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> ~[na:1.7.0_71]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_71]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> ~[na:1.7.0_71]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_71]
> at
> org.apache.hadoop.metrics2.source.JvmMetrics.getEventCounters(JvmMetrics.java:183)
> ~[job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.metrics2.source.JvmMetrics.getMetrics(JvmMetrics.java:100)
> ~[job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
> ~[job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
> ~[job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
> ~[job.jar:0.22-SNAPSHOT]
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassName(DefaultMBeanServerInterceptor.java:333)
> ~[na:1.7.0_71]
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:319)
> ~[na:1.7.0_71]
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
> ~[na:1.7.0_71]
> at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:57)
> ~[job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:221)
> ~[job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:96)
> ~[job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:245)
> [job.jar:0.22-SNAPSHOT]
> at
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:229)
> ~[job.jar:0.22-SNAPSHOT]
> ... 18 common frames omitted
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
