[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15174741#comment-15174741
]
Zhe Zhang commented on HDFS-7866:
---------------------------------
Thanks Rui, very nice work here!
I only finished reviewing the {{INodeFile}} header encoding logic. The main
logic LGTM overall. Just some recommendations on code structure:
# The below is not very accurate. For striped blocks, the tail 11 bits do not
only represent redundancy. E.g. there could be different coders like RS, HH.
3-2 coding and 6-4 have same level of redundancy but they have different
trade-offs. I see the intention here is to unify the terminology for contiguous
and striped blocks. But I think it is pretty hard.
{code}
/**
* Number of bits used to encode block layout type.
* Different types can be replica or EC
*/
public static final int LAYOUT_BIT_WIDTH = 1;
/**
* Number of bits used to encode block redundancy.
* For replicated block, the redundancy is the replication factor;
* for erasure coded block, the redundancy is the EC policy's ID.
*/
public static final int REDUNDANCY_BIT_WIDTH = 11;
{code}
# So instead of the above, I think we can keep the {{LAYOUT_BIT_WIDTH}}, and
then explicitly parse the {{BLOCK_LAYOUT_AND_REDUNDANCY}} section for both
striped and contiguous blocks. To avoid repeating code we can add a util method
{{maskLayoutBit}}. Not sure if it's worth it, since the code is very simple.
# In the "Bit format:" Javadoc we should also explain the
{{BLOCK_LAYOUT_AND_REDUNDANCY}} section more clearly. Some ASCII art here will
be really helpful.
Also, {{SYS_POLICY1_ID}} and {{SYS_POLICY2_ID}} look a little hacky. Can we do
something similar to {{BlockStoragePolicySuite#createDefaultSuite}} and create
some constants? We can also make it a byte to save space.
> Erasure coding: NameNode manages multiple erasure coding policies
> -----------------------------------------------------------------
>
> Key: HDFS-7866
> URL: https://issues.apache.org/jira/browse/HDFS-7866
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: Rui Li
> Attachments: HDFS-7866-v1.patch, HDFS-7866-v2.patch,
> HDFS-7866-v3.patch, HDFS-7866.10.patch, HDFS-7866.4.patch, HDFS-7866.5.patch,
> HDFS-7866.6.patch, HDFS-7866.7.patch, HDFS-7866.8.patch, HDFS-7866.9.patch
>
>
> This is to extend NameNode to load, list and sync predefine EC schemas in
> authorized and controlled approach. The provided facilities will be used to
> implement DFSAdmin commands so admin can list available EC schemas, then
> could choose some of them for target EC zones.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
