[
https://issues.apache.org/jira/browse/HDFS-8137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526509#comment-14526509
]
Kai Zheng commented on HDFS-8137:
---------------------------------
Thanks for your update, Uma. The patch looks good and just two minor comments:
* Regarding the following codes, I'm not very sure about the log output. Would
it be better to be:
{code}
blockLog.warn( "Failed to get the ECSchema for the file {} ",
filename_of_the_block);
...
blockLog.warn( "No EC Schema found for the file {} ", filename_of_the_block);
{code}
{code}
+ ECSchema ecSchema = null;
+ try {
+ ecSchema = namesystem.getECSchemaForPath(block
+ .getBlockCollection().getName());
+ } catch (IOException e) {
+ blockLog.warn(
+ "Failed to get the ECSchema for the blockGroup {} ", block);
+ }
+ if (ecSchema == null) {
+ blockLog.warn("No EC Schema found for the blockGroup {} , "
+ + "so ignoring the block group for EC", block);
+ // TODO: we may have to revisit later for what we can do better
to
+ // handle this case.
+ continue;
+ }
{code}
* In the codes below, might be better to compare ecSchema1 and ecSchema2 with
the system default schema used.
{code}
+ ECSchema ecSchema1 = blkECRecoveryInfo1.getECSchema();
+ ECSchema ecSchema2 = blkECRecoveryInfo2.getECSchema();
+ assertEquals(ecSchema1.getSchemaName(), ecSchema2.getSchemaName());
+ assertEquals(ecSchema1.getNumDataUnits(), ecSchema2.getNumDataUnits());
+ assertEquals(ecSchema1.getNumParityUnits(), ecSchema2.getNumParityUnits());
{code}
> Sends the EC schema to DataNode as well in EC encoding/recovering command
> -------------------------------------------------------------------------
>
> Key: HDFS-8137
> URL: https://issues.apache.org/jira/browse/HDFS-8137
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: Uma Maheswara Rao G
> Attachments: HDFS-8137-0.patch, HDFS-8137-1.patch
>
>
> Discussed with [~umamaheswararao] and [~vinayrpet], we should also send the
> EC schema to DataNode as well contained in the EC encoding/recovering
> command. The target DataNode will use it to guide the executing of the task.
> Another way would be, DataNode would just request schema actively thru a
> separate RPC call, and as an optimization consideration, DataNode may cache
> schemas to avoid repeatedly asking for the same schema twice.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
