[
https://issues.apache.org/jira/browse/HDFS-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Zhao updated HDFS-6969:
----------------------------
Attachment: HDFS-6969.000.patch
Initial patch. The main changes include:
1. Remove INode#getStoragePolicyID(int snapshotId). Instead, use the strategy
described above
2. The current replication mechanism only checks the storage policy directly
specified on the INodeFile. With the changes in #1, the storage policies
specified on ancestral directories are also checked in case that no policy is
specified on file.
3. Add several unit tests to verify the behavior when increasing replication
factor with different storage policies. But looks like currently the replicas
cannot be placed on the correct storages even if the block placement policy
returns the correct results. Will continue digging.
> Archival Storage: INode#getStoragePolicyID should always return the latest
> storage policy
> -----------------------------------------------------------------------------------------
>
> Key: HDFS-6969
> URL: https://issues.apache.org/jira/browse/HDFS-6969
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: balancer, namenode
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-6969.000.patch
>
>
> In general, every file should only provide exact one storage policy for the
> Mover, no matter its snapshot states. Suppose a file /foo/bar, and it is
> contained in snapshots s1 and s2 of the root. If /foo/bar,
> /.snapshot/s1/foo/bar and /.snapshot/s2/foo/bar have different storage
> policies, when running Mover, we have to select one of the storage policies,
> among which the latest one should be the best. And if /foo/bar is deleted, we
> should still use its storage policy before the deletion, since the file
> deletion should not trigger data migration.
> Thus maybe what we can do is:
> 1. For a file with policy directly specified on it, alway follow the latest
> 2. Otherwise follow its latest parental path to identify its storage policy
> (simply following the parent link)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
