[
https://issues.apache.org/jira/browse/HDFS-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Zhao updated HDFS-5714:
----------------------------
Attachment: HDFS-5714.000.patch
Early patch for review. In general, the patch
1. Encodes the whole FileDiffList into a byte array. Instead of always keeping
the byte array in memory, currently the patch only encodes a FileDiffList to a
byte array when loading it from FSImage for the first time. And later if the
corresponding snapshot information is accessed the byte array will be decoded
to the FileDiffList and will not be encoded again (until the next time NN
restarting).
2. Remove ClientNode from FileUnderConstructionFeature and use a byte array to
represent the ClientName and ClientMachine.
> Use byte array to represent UnderConstruction feature and Snapshot feature
> for INodeFile
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-5714
> URL: https://issues.apache.org/jira/browse/HDFS-5714
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-5714.000.patch
>
>
> Currently we define specific classes to represent different INode features,
> such as FileUnderConstructionFeature and FileWithSnapshotFeature. While
> recording these feature information in memory, the internal information and
> object references can still cost a lot of memory. For example, for
> FileWithSnapshotFeature, not considering the INode's local name, the whole
> FileDiff list (with size n) can cost around 120n bytes.
> In order to decrease the memory usage, we plan to use byte array to record
> the UnderConstruction feature and Snapshot feature for INodeFile.
> Specifically, if we use protobuf's encoding, the memory usage for a
> FileWithSnapshotFeature can be less than 56n bytes.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
