[
https://issues.apache.org/jira/browse/HDFS-10983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-10983:
--------------------------------------
Attachment: HDFS-10983.03.patch
[~andrew.wang],
Removed the explicit replication field write for Striped files in FSImage at
{{FSImageFormatPBINode#buildPermissionStatus}}. And extended the
{{FSImageFormatPBINode#loadINodeFile}} with an assert to verify that FSImage
doesn't have replication field set for Striped files. With this new
verification {{TestFSImage#testSupportBlockGroup}} fails at FSImage load for EC
Files when the replication fields are additionally set.
{code}
@@ -332,6 +332,7 @@ private INodeFile loadINodeFile(INodeSection.INode n) {
assert n.getType() == INodeSection.INode.Type.FILE;
..
BlockType blockType = PBHelperClient.convert(f.getBlockType());
LoaderContext state = parent.getLoaderContext();
boolean isStriped = f.hasErasureCodingPolicyID();
+ assert ((!isStriped) || (isStriped && !f.hasReplication()));
Short replication = (!isStriped ? (short) f.getReplication() : null);
{code}
Currently, FSImage Loaders and Savers are not exposed properly for testing and
having an external test for verifying the fsimage inode_section image contents
seems not feasible. So, extending the {{loadINodeFile()}} with assert like few
other asserts in the same file. Please let me know if this is ok ?
> OIV tool should make an EC file explicit
> ----------------------------------------
>
> Key: HDFS-10983
> URL: https://issues.apache.org/jira/browse/HDFS-10983
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 3.0.0-alpha1
> Reporter: Wei-Chiu Chuang
> Assignee: Manoj Govindassamy
> Labels: hdfs-ec-3.0-nice-to-have
> Attachments: HDFS-10983.01.patch, HDFS-10983.02.patch,
> HDFS-10983.03.patch
>
>
> The OIV tool's webhdfs interface does not print if a file is striped or not.
> Also, it prints the file's EC policy ID as replication factor, which is
> inconsistent to the output of a typical webhdfs call to the cluster, which
> always shows replication factor of 0 for EC files.
> Not just webhdfs, but delimiter output does not print if a file is stripped
> or not either.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
