[ 
https://issues.apache.org/jira/browse/HDFS-15124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17016261#comment-17016261
 ] 

Wei-Chiu Chuang commented on HDFS-15124:
----------------------------------------

Please make sure you don't add TopAuditLogger twice by setting 
dfs.namenode.top.enabled to TopAuditLogger.

> Crashing bugs in NameNode when using a valid configuration for 
> `dfs.namenode.audit.loggers`
> -------------------------------------------------------------------------------------------
>
>                 Key: HDFS-15124
>                 URL: https://issues.apache.org/jira/browse/HDFS-15124
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.10.0
>            Reporter: Ctest
>            Priority: Critical
>         Attachments: HDFS-15124.000.patch
>
>
> I am using Hadoop-2.10.0.
> The configuration parameter `dfs.namenode.audit.loggers` allows `default` 
> (which is the default value) and 
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`.
> When I use `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, 
> namenode will not be started successfully because of an 
> `InstantiationException` thrown from 
> `org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers`. 
> The root cause is that while initializing namenode, `initAuditLoggers` will 
> be called and it will try to call the default constructor of 
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` which doesn't 
> have a default constructor. Thus the `InstantiationException` exception is 
> thrown.
>  
> *Symptom*
> *$ ./start-dfs.sh*
> {code:java}
> 2019-12-18 14:05:20,670 ERROR 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem 
> initialization failed.java.lang.RuntimeException: 
> java.lang.InstantiationException: 
> org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1024)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:858)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:677)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:674)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:736)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:961)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:940)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1714)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1782)
> Caused by: java.lang.InstantiationException: 
> org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger
> at java.lang.Class.newInstance(Class.java:427)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1017)...
> 8 more
> Caused by: java.lang.NoSuchMethodException: 
> org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger.<init>()
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.newInstance(Class.java:412)
> ... 9 more{code}
>  
>  
> *Detailed Root Cause*
> There is no default constructor in 
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`: 
> {code:java}
> /** 
>  * An {@link AuditLogger} that sends logged data directly to the metrics 
>  * systems. It is used when the top service is used directly by the name node 
>  */ 
> @InterfaceAudience.Private 
> public class TopAuditLogger implements AuditLogger {     
>   public static finalLogger LOG = 
> LoggerFactory.getLogger(TopAuditLogger.class); 
>   private final TopMetrics topMetrics; 
>   public TopAuditLogger(TopMetrics topMetrics) {
>     Preconditions.checkNotNull(topMetrics, "Cannot init with a null " + 
>         "TopMetrics");
>     this.topMetrics = topMetrics; 
>   }
>   @Override
>   public void initialize(Configuration conf) { 
>   }
> {code}
> As long as the configuration parameter `dfs.namenode.audit.loggers` is set to 
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, 
> `initAuditLoggers` will try to call its default constructor to make a new 
> instance: 
> {code:java}
> private List<AuditLogger> initAuditLoggers(Configuration conf) {
>   // Initialize the custom access loggers if configured.
>   Collection<String> alClasses =
>       conf.getTrimmedStringCollection(DFS_NAMENODE_AUDIT_LOGGERS_KEY);
>   List<AuditLogger> auditLoggers = Lists.newArrayList();
>   if (alClasses != null && !alClasses.isEmpty()) {
>     for (String className : alClasses) {
>       try {
>         AuditLogger logger;
>         if (DFS_NAMENODE_DEFAULT_AUDIT_LOGGER_NAME.equals(className)) {
>           logger = new DefaultAuditLogger();
>         } else {
>           logger = (AuditLogger) Class.forName(className).newInstance();
>         }
>         logger.initialize(conf);
>         auditLoggers.add(logger);
>       } catch (RuntimeException re) {
>         throw re;
>       } catch (Exception e) {
>         throw new RuntimeException(e);
>       }
>     }
>   }
> {code}
> `initAuditLoggers` tries to call the default constructor to make a new 
> instance in:
> {code:java}
> logger = (AuditLogger) Class.forName(className).newInstance();
> {code}
> This is very different from the default configuration, `default`, which 
> implements a default constructor so the default is fine.
>  
> *How To Reproduce* 
> The version of Hadoop: 2.10.0
>  # Set the value of configuration parameter `dfs.namenode.audit.loggers` to 
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` in 
> "hdfs-site.xml"(the default value is `default`)
>  # Start the namenode by running "start-dfs.sh"
>  # The namenode will not be started successfully.
> {code:java}
> <property>
>   <name>dfs.namenode.audit.loggers</name>
>   <value>org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger</value>
>   <description>
>     List of classes implementing audit loggers that will receive audit events.
>     These should be implementations of 
> org.apache.hadoop.hdfs.server.namenode.AuditLogger.
>     The special value "default" can be used to reference the default audit
>     logger, which uses the configured log system. Installing custom audit 
> loggers
>     may affect the performance and stability of the NameNode. Refer to the 
> custom
>     logger's documentation for more details.
>   </description>
> </property>
> {code}
>  
> *How To Fix*
> Add a default constructor for 
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`.
> I wrote a patch to add a default constructor for TopAuditLogger.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to