[
https://issues.apache.org/jira/browse/HDFS-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13676503#comment-13676503
]
Hudson commented on HDFS-4850:
------------------------------
Integrated in Hadoop-trunk-Commit #3866 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/3866/])
HDFS-4850. Fix OfflineImageViewer to work on fsimages with empty files or
snapshots. Contributed by Jing Zhao. (Revision 1490080)
Result = SUCCESS
jing9 : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1490080
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodesInPath.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/ImageLoaderCurrent.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/ImageVisitor.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TextWriterImageVisitor.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/XmlImageVisitor.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshot.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
> fix OfflineImageViewer to work on fsimages with empty files or snapshots
> ------------------------------------------------------------------------
>
> Key: HDFS-4850
> URL: https://issues.apache.org/jira/browse/HDFS-4850
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tools
> Reporter: Stephen Chu
> Assignee: Jing Zhao
> Labels: snapshot
> Attachments: datadirs.tar.gz, fsimage_0000000000000000004,
> fsimage_0000000000000000008, HDFS-4850.000.patch, HDFS-4850.001.patch,
> oiv_out_1, oiv_out_2
>
>
> I deployed hadoop-trunk HDFS and created _/user/schu/_. I then forced a
> checkpoint, fetched the fsimage, and ran the default OfflineImageViewer
> successfully on the fsimage.
> {code}
> schu-mbp:~ schu$ hdfs oiv -i fsimage_0000000000000000004 -o oiv_out_1
> schu-mbp:~ schu$ cat oiv_out_1
> drwxr-xr-x - schu supergroup 0 2013-05-24 16:59 /
> drwxr-xr-x - schu supergroup 0 2013-05-24 16:59 /user
> drwxr-xr-x - schu supergroup 0 2013-05-24 16:59 /user/schu
> schu-mbp:~ schu$
> {code}
> I then touched an empty file _/user/schu/testFile1_
> {code}
> schu-mbp:~ schu$ hadoop fs -lsr /
> lsr: DEPRECATED: Please use 'ls -R' instead.
> drwxr-xr-x - schu supergroup 0 2013-05-24 16:59 /user
> drwxr-xr-x - schu supergroup 0 2013-05-24 17:00 /user/schu
> -rw-r--r-- 1 schu supergroup 0 2013-05-24 17:00
> /user/schu/testFile1
> {code}
> and forced another checkpoint, fetched the fsimage, and reran the
> OfflineImageViewer. I encountered a NegativeArraySizeException:
> {code}
> schu-mbp:~ schu$ hdfs oiv -i fsimage_0000000000000000008 -o oiv_out_2
> Input ended unexpectedly.
> 2013-05-24 17:01:13,622 ERROR [main] offlineImageViewer.OfflineImageViewer
> (OfflineImageViewer.java:go(140)) - image loading failed at offset 402
> Exception in thread "main" java.lang.NegativeArraySizeException
> at org.apache.hadoop.io.Text.readString(Text.java:458)
> at
> org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processPermission(ImageLoaderCurrent.java:370)
> at
> org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processINode(ImageLoaderCurrent.java:671)
> at
> org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processChildren(ImageLoaderCurrent.java:557)
> at
> org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processDirectoryWithSnapshot(ImageLoaderCurrent.java:464)
> at
> org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processDirectoryWithSnapshot(ImageLoaderCurrent.java:470)
> at
> org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processDirectoryWithSnapshot(ImageLoaderCurrent.java:470)
> at
> org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processLocalNameINodesWithSnapshot(ImageLoaderCurrent.java:444)
> at
> org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processINodes(ImageLoaderCurrent.java:398)
> at
> org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.loadImage(ImageLoaderCurrent.java:199)
> at
> org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewer.go(OfflineImageViewer.java:136)
> at
> org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewer.main(OfflineImageViewer.java:260)
> {code}
> This is reproducible. I've reproduced this scenario after formatting HDFS and
> restarting and touching an empty file _/testFile1_.
> Attached are the data dirs, the fsimage before creating the empty file
> (fsimage_0000000000000000004) and the fsimage afterwards
> (fsimage_0000000000000000004) and their outputs, oiv_out_1 and oiv_out_2
> respectively.
> The oiv_out_2 does not include the empty _/user/schu/testFile1_.
> I don't run into this problem using hadoop-2.0.4-alpha.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
