[
https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996839#comment-13996839
]
Hadoop QA commented on HDFS-6293:
---------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12644653/HDFS-6293.trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1282 javac
compiler warnings (more than the trunk's current 1275 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6894//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Javac warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/6894//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6894//console
This message is automatically generated.
> Issues with OIV processing PB-based fsimages
> --------------------------------------------
>
> Key: HDFS-6293
> URL: https://issues.apache.org/jira/browse/HDFS-6293
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.4.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Attachments: HDFS-6293.000.patch, HDFS-6293.001.patch,
> HDFS-6293.002-save-deprecated-fsimage.patch, HDFS-6293.branch-2.patch,
> HDFS-6293.trunk.patch, HDFS-6293.trunk.patch,
> HDFS-6293_sbn_ckpt_retention.patch,
> HDFS-6293_sbn_ckpt_retention_oiv_legacy.patch, Heap Histogram.html
>
>
> There are issues with OIV when processing fsimages in protobuf.
> Due to the internal layout changes introduced by the protobuf-based fsimage,
> OIV consumes excessive amount of memory. We have tested with a fsimage with
> about 140M files/directories. The peak heap usage when processing this image
> in pre-protobuf (i.e. pre-2.4.0) format was about 350MB. After converting
> the image to the protobuf format on 2.4.0, OIV would OOM even with 80GB of
> heap (max new size was 1GB). It should be possible to process any image with
> the default heap size of 1.5GB.
> Another issue is the complete change of format/content in OIV's XML output.
> I also noticed that the secret manager section has no tokens while there were
> unexpired tokens in the original image (pre-2.4.0). I did not check whether
> they were also missing in the new pb fsimage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
