[
https://issues.apache.org/jira/browse/HDFS-13512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459782#comment-16459782
]
Hanisha Koneru commented on HDFS-13512:
---------------------------------------
Sorry I missed this earlier. The FileStatus json looks something like below.
{code:java}
{"accessTime":1525150241823,"blockSize":134217728,"childrenNum":0,"ecBit":true,
"ecPolicy":"RS-3-2-1024k",
"ecPolicyIns":{"name":"RS-3-2-1024k",
"schema":{"codecName":"rs","numDataUnits":3,"numParityUnits":2,"extraOptions":{}},
"cellSize":1048576,"id":2,"numDataUnits":3,"numParityUnits":2,"codecName":"rs","replicationPolicy":false,"systemPolicy":true},
...}
We are getting the {{numDataUnits}}, {{numParityUnits}} and {{codecName}} twice
in the json. Also, we do not need the {{replicationPolicy}} and
{{systemPolicy}} in the json. These states are dynamically calculated through
the id.
IIUC, this is happening because both {{ErasureCodingPolicy}} class and
{{ECSchema}} class are Serializable. We should keep the schema and remove the
extra 5 fields from the ListStatus json - numDataUnits, numParityUnits,
codecName, replicationPolicy and systemPolicy.
{code}
> WebHdfs getHdfsFileStatus/getFileStatus doesn't return ecPolicy name wired in
> json
> ----------------------------------------------------------------------------------
>
> Key: HDFS-13512
> URL: https://issues.apache.org/jira/browse/HDFS-13512
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Ajay Kumar
> Assignee: Ajay Kumar
> Priority: Major
> Attachments: HDFS-13512.00.patch, HDFS-13512.01.patch,
> HDFS-13512.02.patch, HDFS-13512.03.patch, HDFS-13512.04.patch
>
>
> Currently LISTSTATUS call to WebHdfs returns a json. These jsonArray elements
> do have the ecPolicy name.
> But when WebHdfsFileSystem converts it back into a FileStatus object, the
> ecPolicy is not added. This is because the json contains only the ecPolicy
> name and this name is not sufficient to decode it back to ErasureCodingPolicy
> object.
> While converting json back to HdfsFileStatus we should set ecPolicyName
> whenever it is set for give file/dir.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
