[
https://issues.apache.org/jira/browse/HDFS-5568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13832431#comment-13832431
]
Uma Maheswara Rao G edited comment on HDFS-5568 at 11/26/13 9:35 AM:
---------------------------------------------------------------------
Hi Vinay, Thanks for the patch. I just reviewed the patch.
Comments:
1) if user don't have any snapshots, then it will end failing with NPE. as
getSnapshottableDirListing returns null.
{noformat}
2013-11-26 13:13:22,522 WARN namenode.NameNode (NamenodeFsck.java:fsck(254))
- Fsck on path '/' FAILED
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.fsck(NamenodeFsck.java:208)
at
org.apache.hadoop.hdfs.server.namenode.FsckServlet$1.run(FsckServlet.java:69)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1515)
at
org.apache.hadoop.hdfs.server.namenode.FsckServlet.doGet(FsckServlet.java:58)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at
org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1220)
{noformat}
2) when we take snapshot for root dir, getFIleInfo will fail saying invalid
path. That is because, you are doing path + Pathsepaator _ snapshotdir ===
//.snapshotdir ...Right now path validation does not allow '//' or ':'
{noformat}
2013-11-26 14:08:14,637 WARN namenode.NameNode (NamenodeFsck.java:fsck(254)) -
Fsck on path '/' FAILED
org.apache.hadoop.fs.InvalidPathException: Invalid path name Invalid file name:
//.snapshot
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3393)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:759)
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.check(NamenodeFsck.java:296)
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.fsck(NamenodeFsck.java:223)
at
org.apache.hadoop.hdfs.server.namenode.FsckServlet$1.run(FsckServlet.java:69)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
{noformat}
3) Test cases need to catch this kind of small issues.
I am just including the test here with the fixups above mentioned as I wrote
that while just verifying my findings.
Also please take care of above failure if still exists with this patch.
was (Author: umamaheswararao):
Hi Vinay, Thanks for the patch. I just reviewed the patch.
Comments:
1) if user don't have any snapshots, then it will end failing with NPE. as
getSnapshottableDirListing returns null.
{noformat}
2013-11-26 13:13:22,522 WARN namenode.NameNode (NamenodeFsck.java:fsck(254))
- Fsck on path '/' FAILED
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.fsck(NamenodeFsck.java:208)
at
org.apache.hadoop.hdfs.server.namenode.FsckServlet$1.run(FsckServlet.java:69)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1515)
at
org.apache.hadoop.hdfs.server.namenode.FsckServlet.doGet(FsckServlet.java:58)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at
org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1220)
{noformat}
2) when we take snapshot for root dir, getFIleInfo will fail saying invalid
path. That is because, you are doing path + Pathsepaator _ snapshotdir ===
//.snapshotdir ...Right now path validation does not allow '//' or ':'
3) Test cases need to catch this kind of small issues.
I am just including the test here with the fixups above mentioned as I wrote
that while just verifying my findings.
Also please take care of above failure if still exists with this patch.
> Support inclusion of snapshot paths in Namenode fsck
> ----------------------------------------------------
>
> Key: HDFS-5568
> URL: https://issues.apache.org/jira/browse/HDFS-5568
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: snapshots
> Reporter: Vinay
> Assignee: Vinay
> Attachments: HDFS-5568-1.patch, HDFS-5568.patch
>
>
> Support Fsck to check the snapshot paths also for inconsistency.
> Currently Fsck supports snapshot paths if path given explicitly refers to a
> snapshot path.
> We have seen safemode problems in our clusters which were due to blocks
> missing which were only present inside snapshots. But "hdfs fsck /" shows
> HEALTHY.
> So supporting snapshot paths also during fsck (may be by default or on
> demand) would be helpful in these cases instead of specifying each and every
> snapshottable directory.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
