Mingliang Liu created HDFS-9027:
-----------------------------------
Summary: Refactor o.a.h.hdfs.DataStreamer$isLazyPersist() method
Key: HDFS-9027
URL: https://issues.apache.org/jira/browse/HDFS-9027
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Mingliang Liu
Assignee: Mingliang Liu
In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
class checks whether the HDFS file is lazy persist. It does two things:
1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
builds an array of {{BlockStoragePolicy}} internally
2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
Code samples as following:
{code}
private static final BlockStoragePolicySuite blockStoragePolicySuite =
BlockStoragePolicySuite.createDefaultSuite();
static boolean isLazyPersist(HdfsFileStatus stat) {
final BlockStoragePolicy p = blockStoragePolicySuite.getPolicy(
HdfsConstants.MEMORY_STORAGE_POLICY_NAME);
return p != null && stat.getStoragePolicy() == p.getId();
}
{code}
This has two side effects:
1. Takes time to iterate the pre-built block storage policy array in order to
find the _same_ policy every time whose id matters only (as we need to compare
the file status policy id with lazy persist policy id)
2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
should be moved to {{hadoop-hdfs-client}} module, while the latter can stay in
{{hadoop-hdfs}} module.
Actually, we have the block storage policy IDs, which can be used to compare
with HDFS file status' policy id, as following:
{code}
static boolean isLazyPersist(HdfsFileStatus stat) {
return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
}
{code}
This way, we only need to move the block storage policies' IDs from
{{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
({{hadoop-hdfs-client}} module).
Another reason we should move those block storage policy IDs is that the block
storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
