[
https://issues.apache.org/jira/browse/HDFS-3921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-3921:
---------------------------------
Target Version/s: 2.0.3-alpha
Affects Version/s: (was: 2.0.1-alpha)
2.0.2-alpha
Assignee: Aaron T. Myers
Summary: NN will prematurely consider blocks missing when
entering active state while still in safe mode (was: corrupt files web UI hits
error: Cannot run listCorruptFileBlocks because replication queues have not
been initialized.)
I've taken a look into this. The trouble is that the NN is always processing
its repl queues when it enters the active state even if it's still in startup
safemode, i.e. not all DNs have reported in. This causes the web UI to
prematurely indicate that blocks are missing, when in fact the NN should still
just simply be waiting for more DNs to report in order to exit startup safemode.
> NN will prematurely consider blocks missing when entering active state while
> still in safe mode
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-3921
> URL: https://issues.apache.org/jira/browse/HDFS-3921
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.0.2-alpha
> Reporter: Stephen Chu
> Assignee: Aaron T. Myers
>
> I shut down all the HDFS daemons in an Highly Available (automatic failover)
> cluster.
> Then I started one NN and it transitioned it to active. No DNs were started,
> and I saw the red warning link on the NN web UI:
> WARNING : There are 36 missing blocks. Please check the logs or run fsck in
> order to identify the missing blocks.
> I clicked this to go to the corrupt_files.jsp page, which ran into the
> following error:
> {noformat}
> HTTP ERROR 500
> Problem accessing /corrupt_files.jsp. Reason:
> Cannot run listCorruptFileBlocks because replication queues have not been
> initialized.
> Caused by:
> java.io.IOException: Cannot run listCorruptFileBlocks because replication
> queues have not been initialized.
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.listCorruptFileBlocks(FSNamesystem.java:5035)
> at
> org.apache.hadoop.hdfs.server.namenode.corrupt_005ffiles_jsp._jspService(corrupt_005ffiles_jsp.java:78)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1039)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
