[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180373#comment-15180373
]
Zhe Zhang commented on HDFS-7866:
---------------------------------
bq. We're using 11 bits to store the policy ID, which means it could excess a
byte (replication factor is also a short). What do you think?
Right, on the {{INodeFile}} level we can't save space by switching to {{byte}}.
But by constraining the ID to be a byte we are saving memory on block level. I
don't think we will possibly support more than 256 EC policies in the
foreseeable future; so maybe set it to byte now and consider changing to short
when there's a clear need? Pinging [~drankye] and [~andrew.wang] for more
insights here.
bq. I also would like to know your opinions about randomly choosing a policy in
the tests.
Very interesting idea. I think we should do a follow-on {{test}} JIRA. And yes
I think we are safe to switch back to RS-6-3 in the next rev.
bq. I think essentially it's hacky because we allow user to specify the
replication factor when creating a file but then overwrite it to store the EC
policy ID. Fixing this requires changing the API, which I think is unacceptable.
I don't think there's a compatibility issue here, because we are only changing
the {{INodeFile}} constructor with the {{isStriped}} flag. I think we can just
change the boolean to a byte, representing the EC policy ID. But given the size
of the current patch I recommend we leave it as follow-on.
> Erasure coding: NameNode manages multiple erasure coding policies
> -----------------------------------------------------------------
>
> Key: HDFS-7866
> URL: https://issues.apache.org/jira/browse/HDFS-7866
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: Rui Li
> Attachments: HDFS-7866-v1.patch, HDFS-7866-v2.patch,
> HDFS-7866-v3.patch, HDFS-7866.10.patch, HDFS-7866.11.patch,
> HDFS-7866.4.patch, HDFS-7866.5.patch, HDFS-7866.6.patch, HDFS-7866.7.patch,
> HDFS-7866.8.patch, HDFS-7866.9.patch
>
>
> This is to extend NameNode to load, list and sync predefine EC schemas in
> authorized and controlled approach. The provided facilities will be used to
> implement DFSAdmin commands so admin can list available EC schemas, then
> could choose some of them for target EC zones.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
