lujie created HDFS-14695:
----------------------------
Summary: Reboot NN fails while NN is starting and creating image
file
Key: HDFS-14695
URL: https://issues.apache.org/jira/browse/HDFS-14695
Project: Hadoop HDFS
Issue Type: Bug
Reporter: lujie
Attachments: hadoop-hires-namenode-hadoop11.log
We are doing test in our cluster, we find that NN can reboot fail due to "No
valid image files found".
{code:java}
2019-08-02 17:07:02,625 WARN
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception
loading fsimage
java.io.FileNotFoundException: No valid image files found
at
org.apache.hadoop.hdfs.server.namenode.FSImageTransactionalStorageInspector.getLatestImages(FSImageTransactionalStorageInspector.java:158)
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:674)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:325)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1099)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:716)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:635)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:697)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:940)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:913)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1646)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1713)
2019-08-02 17:07:02,633 INFO org.eclipse.jetty.server.handler.ContextHandler:
Stopped o.e.j.w.WebAppContext@2c532cd8{/,null,UNAVAILABLE}{/hdfs}
2019-08-02 17:07:02,648 INFO org.eclipse.jetty.server.AbstractConnector:
Stopped ServerConnector@2ceb80a1{HTTP/1.1,[http/1.1]}{0.0.0.0:9870}
2019-08-02 17:07:02,649 INFO org.eclipse.jetty.server.handler.ContextHandler:
Stopped
o.e.j.s.ServletContextHandler@38aa816f{/static,file:///home/hires/cloudraid/hadoop/hadoop-3.2.0/share/hadoop/hdfs/webapps/static/,UNAVAILABLE}
2019-08-02 17:07:02,649 INFO org.eclipse.jetty.server.handler.ContextHandler:
Stopped
o.e.j.s.ServletContextHandler@2f62ea70{/logs,file:///home/hires/cloudraid/hadoop/hadoop-3.2.0/logs/,UNAVAILABLE}
2019-08-02 17:07:02,652 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl:
Stopping NameNode metrics system...
2019-08-02 17:07:02,653 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl:
NameNode metrics system stopped.
2019-08-02 17:07:02,653 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl:
NameNode metrics system shutdown complete.
2019-08-02 17:07:02,653 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
Failed to start namenode.
java.io.FileNotFoundException: No valid image files found
at
org.apache.hadoop.hdfs.server.namenode.FSImageTransactionalStorageInspector.getLatestImages(FSImageTransactionalStorageInspector.java:158)
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:674)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:325)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1099)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:716)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:635)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:697)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:940)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:913)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1646)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1713)
2019-08-02 17:07:02,662 INFO org.apache.hadoop.util.ExitUtil: Exiting with
status 1: java.io.FileNotFoundException: No valid image files found
2019-08-02 17:07:02,667 INFO org.apache.hadoop.hdfs.server.namenode.NameNode:
SHUTDOWN_MSG:
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
