[
https://issues.apache.org/jira/browse/HDFS-14695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lujie updated HDFS-14695:
-------------------------
Attachment: hadoop-hires-namenode-hadoop11.log
> Reboot NN fails while NN is starting and creating image file
> ------------------------------------------------------------
>
> Key: HDFS-14695
> URL: https://issues.apache.org/jira/browse/HDFS-14695
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: lujie
> Priority: Critical
> Attachments: hadoop-hires-namenode-hadoop11.log
>
>
> We are doing test in our cluster, we find that NN can reboot fail due to "No
> valid image files found".
> {code:java}
> 2019-08-02 17:07:02,625 WARN
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception
> loading fsimage
> java.io.FileNotFoundException: No valid image files found
> at
> org.apache.hadoop.hdfs.server.namenode.FSImageTransactionalStorageInspector.getLatestImages(FSImageTransactionalStorageInspector.java:158)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:674)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:325)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1099)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:716)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:635)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:697)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:940)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:913)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1646)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1713)
> 2019-08-02 17:07:02,633 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Stopped o.e.j.w.WebAppContext@2c532cd8{/,null,UNAVAILABLE}{/hdfs}
> 2019-08-02 17:07:02,648 INFO org.eclipse.jetty.server.AbstractConnector:
> Stopped ServerConnector@2ceb80a1{HTTP/1.1,[http/1.1]}{0.0.0.0:9870}
> 2019-08-02 17:07:02,649 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Stopped
> o.e.j.s.ServletContextHandler@38aa816f{/static,file:///home/hires/cloudraid/hadoop/hadoop-3.2.0/share/hadoop/hdfs/webapps/static/,UNAVAILABLE}
> 2019-08-02 17:07:02,649 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Stopped
> o.e.j.s.ServletContextHandler@2f62ea70{/logs,file:///home/hires/cloudraid/hadoop/hadoop-3.2.0/logs/,UNAVAILABLE}
> 2019-08-02 17:07:02,652 INFO
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode metrics
> system...
> 2019-08-02 17:07:02,653 INFO
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system
> stopped.
> 2019-08-02 17:07:02,653 INFO
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system
> shutdown complete.
> 2019-08-02 17:07:02,653 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
> java.io.FileNotFoundException: No valid image files found
> at
> org.apache.hadoop.hdfs.server.namenode.FSImageTransactionalStorageInspector.getLatestImages(FSImageTransactionalStorageInspector.java:158)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:674)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:325)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1099)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:716)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:635)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:697)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:940)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:913)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1646)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1713)
> 2019-08-02 17:07:02,662 INFO org.apache.hadoop.util.ExitUtil: Exiting with
> status 1: java.io.FileNotFoundException: No valid image files found
> 2019-08-02 17:07:02,667 INFO org.apache.hadoop.hdfs.server.namenode.NameNode:
> SHUTDOWN_MSG:
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
