[
https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290294#comment-14290294
]
Andrew Wang commented on HDFS-6673:
-----------------------------------
bq. The test is on a 3G fsimage which can easily fit in the working set of your
laptop...Multiple users are running much larger clusters, where their fsimage
can be as big as 40G
I know there are larger customers out there, but as I said above, we surveyed
the sizes of our customer's fsimages, and this was one of the larger ones. Thus
this solution will work for most production deployments. Also, even a 40GB
image easily fits in memory on server these days.
bq. but given we have HDFS-6293 as a solid solution today, I would much rather
to see this tool to be capable of handling fsimages from the real, large-scale
production runs
HDFS-6293 doesn't include metadata newer than the old format. It also requires
the NN to write out an additional old fsimage just for OIV alongside the real
one. These seem like pretty big drawbacks to me, and are addressed by this
tool. I think calling it "half-baked" is unfair considering it provides greater
functionality.
Anyway, thanks for not -1'ing, I'll commit this now. Let's continue this
discussion in a follow-on JIRA.
> Add Delimited format supports for PB OIV tool
> ---------------------------------------------
>
> Key: HDFS-6673
> URL: https://issues.apache.org/jira/browse/HDFS-6673
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 2.4.0
> Reporter: Lei (Eddy) Xu
> Assignee: Lei (Eddy) Xu
> Priority: Minor
> Attachments: HDFS-6673.000.patch, HDFS-6673.001.patch,
> HDFS-6673.002.patch, HDFS-6673.003.patch, HDFS-6673.004.patch,
> HDFS-6673.005.patch
>
>
> The new oiv tool, which is designed for Protobuf fsimage, lacks a few
> features supported in the old {{oiv}} tool.
> This task adds supports of _Delimited_ processor to the oiv tool.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
