[
https://issues.apache.org/jira/browse/HDFS-5952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156091#comment-14156091
]
Hao Chen commented on HDFS-5952:
--------------------------------
I have tested for large PB-based fsimage about 8GiB which used to consume about
85GiB of memory and is just taking about 30GiB (about 30% or less) now using
this processor.
In fact, we are using this processor in production for all our clusters now
which seems to work fine aside name node without affecting its performance and
we are highly relying on it for daily hadoop storage management but not just
for temporary troubleshooting, so I am surely willing to bring it back to trunk
if it can help others too.
> Create a tool to run data analysis on the PB format fsimage
> -----------------------------------------------------------
>
> Key: HDFS-5952
> URL: https://issues.apache.org/jira/browse/HDFS-5952
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: tools
> Affects Versions: 2.6.0
> Reporter: Akira AJISAKA
> Attachments: HDFS-5952.patch
>
>
> Delimited processor in OfflineImageViewer is not supported after HDFS-5698
> was merged.
> The motivation of delimited processor is to run data analysis on the fsimage,
> therefore, there might be more values to create a tool for Hive or Pig that
> reads the PB format fsimage directly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
