[
https://issues.apache.org/jira/browse/HADOOP-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12689778#action_12689778
]
Konstantin Shvachko commented on HADOOP-5467:
---------------------------------------------
This is a very useful tool.
My main concern here is that this tool introduces an alternative fsimage loader
(reader), which may easily become out of sync with the loader we have in
FSImage class.
Ideally we should have the same source code reading the fsimage file and then
using different visitors to process deserialized data. I think we can achieve
that goal by implementing a {{LoadFSImageVisitor}}, which will call
FSNamesystem methods to add inodes to the directory tree and so on, making it a
replacement to {{FSImage.loadFSImage()}}.
The {{LoadFSImageVisitor}} can be passed to {{FSImageProcessor}} same as other
visitors Jakob implemented.
We can do it in a separate Jira, but it should be done before the next release
so that we had uniform deserialization in the release.
This approach will probably also require to move {{FSImageProcessor}} code
inside {{server.namenode}} package. The {{OfflineImageViewer}} itself should
remain in tools.
Other comments:
# offlineimageviewer should be a part of hdfs shell command group rather than
hadoop.
# I would shorten it to just imageviewer.
# When I call {{hadoop offlineimageviewer}} it first prints an error: "Error
parsing options: i"
# option "-o" does not work together with "-p XML". Please check other
combination too.
# OfflineImageViewer.java warnings in line 135 about accessing static methods
in non static way.
# {{FSImageProcessorV16to19}} should be renamed to something more version
independent.
If you go to v -20 you will probably modify this class rather than implement a
new one.
I'd probably rename the interface {{FSImageProcessor}} into
{{FSImageProcessorInterface}} and then
{{FSImageProcessorV16to19}} can be renamed to {{FSImageProcessor}}.
# {{if ( p.canProcessVersion(version) )}} should not have spaces after "(" and
before ")".
# {{TextWriterFSImageProcessor}} should probably be
{{TextWriterFSImageVisitor}}.
# {{FSImageElement}} should be declared in {{FSImageVisitor}}.
# We do not want to use deprecated UTF8 class more than it is used already, so
it is better to
use {{FSImage.readBytes()}}, etc. instead of reimplementing them in
{{FSImageProcessor}}.
> Create an offline fsimage image viewer
> --------------------------------------
>
> Key: HADOOP-5467
> URL: https://issues.apache.org/jira/browse/HADOOP-5467
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Reporter: Jakob Homan
> Assignee: Jakob Homan
> Attachments: fsimage.xml, HADOOP-5467.patch, HADOOP-5467.patch,
> HADOOP-5467.patch
>
>
> It would be useful to have a tool to examine/dump the contents of the fsimage
> file to human-readable form. This would allow analysis of the namespace
> (file usage, block sizes, etc) without impacting the operation of the
> namenode. XML would be reasonable output format, as it can be easily viewed,
> compressed and manipulated via either XSLT or XQuery.
> I've started work on this and will have an initial version soon.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
