[
https://issues.apache.org/jira/browse/HDFS-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15982805#comment-15982805
]
Takanobu Asanuma commented on HDFS-10999:
-----------------------------------------
Hi, [~manojg].
Thanks for uploading the patch. I have checked almost all changed code. Change
of the interfaces accompanying the additions of new MBeans and Stats looks good
to me (non-binding). Some comments for the detailed implementations:
*BlockManagerSafeMode.java:*
* How about using {{LongAccumulator}} for {{numberOfBytesInFutureBlocks}}, too?
*CorruptReplicasMap.java:*
* Should this {{decrementBlockStat}} be included in the if statement?
{code:java}
if (datanodes.isEmpty()) {
// remove the block if there is no more corrupted replicas
corruptReplicasMap.remove(blk);
decrementBlockStat(blk);
}
{code}
* It seems package private is enough for new methods
{{getCorruptReplicatedBlocksStat}} and {{getCorruptStripedBlocksStat}}.
*InvalidateBlocks.java and LowRedundancyBlocks.java:*
Sorry, but I still need more time to review this code.
*For unit tests:*
I think it would be good if we add more unit tests for these changes in this
jira or follow-on jiras.
* Add more validations for new metrics in {{TestComputeInvalidateWork}},
{{TestCorruptReplicaInfo}} and {{TestLowRedundancyBlockQueues}}.
* {{TestUnderReplicatedBlocks}} covers only replicated files. If we use
{{DFSTestUtil#verifyClientStats}} in {{TestReconstructStripedBlocks}}, we may
be able to cover the EC case.
> Introduce separate stats for Replicated and Erasure Coded Blocks apart from
> the current Aggregated stats
> --------------------------------------------------------------------------------------------------------
>
> Key: HDFS-10999
> URL: https://issues.apache.org/jira/browse/HDFS-10999
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: erasure-coding
> Affects Versions: 3.0.0-alpha1
> Reporter: Wei-Chiu Chuang
> Assignee: Manoj Govindassamy
> Labels: hdfs-ec-3.0-nice-to-have, supportability
> Attachments: HDFS-10999.01.patch, HDFS-10999.02.patch
>
>
> Per HDFS-9857, it seems in the Hadoop 3 world, people prefer the more generic
> term "low redundancy" to the old-fashioned "under replicated". But this term
> is still being used in messages in several places, such as web ui, dfsadmin
> and fsck. We should probably change them to avoid confusion.
> File this jira to discuss it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
