[
https://issues.apache.org/jira/browse/HDFS-7349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279639#comment-14279639
]
Zhe Zhang commented on HDFS-7349:
---------------------------------
Great work Vinay. Sorry I'm just catching up on this.
# It seems {{convertToEC}} and other RPC calls in {{DFSClient}} should have
{{clientName}} as a parameter.
# The {{deleteReplicas}} and {{deleteEcBlocks}} options are really cool for
users; but they are not easy to implement under striping. If we convert a file
to EC without deleting the original block replicas, that implies keeping 2
forms of the file (with and without striping). We have discussed this hybrid
layout as an advanced optimization but haven't decided whether to include it.
# The {{async}} option is easy to implement though. It's in the comment but not
in the patch. Shall we include it?
#* I agree with [~drankye] that EC-rep conversions are time consuming. I think
the sync option is to give users a similar experience as traditional space
saving operations (e.g., gzip under Linux) -- when the operation returns we
know the space is freed.
# I believe HDFS-7337 will define a {{Schema}} class, which I believe will at
least contain the following items ([~drankye] please feel free to chime in).
It's a design question whether we should ask users to configure each item
({{hdfs ec -convertToEC <path> -numDataBlocks 6 -numParityBlocks 3 -codec RS}})
or choose from a predefined list ({{hdfs ec -convertToEC <path> -s RS63}}).
#* {{int num_data_blocks}}
#* {{int num_parity_blocks}}
#* {{String codec_type}}: e.g., Reed-Solomon, XOR
# How should we handle conflicting schemas from {{convertToEC}} and
{{createZone}}? After making a decision we should document it in the comment
and help message.
# Should we always use {{createEcZone}} instead of {{createZone}} and
{{removeEcZone}} instead of {{removeZone}}? There are other zones, like
encryption zones.
> Support DFS command for the EC encoding
> ---------------------------------------
>
> Key: HDFS-7349
> URL: https://issues.apache.org/jira/browse/HDFS-7349
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Vinayakumar B
> Assignee: Vinayakumar B
> Attachments: HDFS-7349-001.patch, HDFS-7349-002.patch
>
>
> Support implementation of the following commands
> *hdfs dfs -convertToEC <path>*
> <path>: Converts all blocks under this path to EC form (if not already in
> EC form, and if can be coded).
> *hdfs dfs -convertToRep <path>*
> <path>: Converts all blocks under this path to be replicated form.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
