[
https://issues.apache.org/jira/browse/HDFS-4931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jihoon Son updated HDFS-4931:
-----------------------------
Attachment: lap2.patch
> Extend the block placement policy interface to utilize the location
> information of previously stored files
> ------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4931
> URL: https://issues.apache.org/jira/browse/HDFS-4931
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Jihoon Son
> Attachments: lap2.patch
>
>
> Nowadays, I'm implementing a locality preserving block placement policy which
> stores files in a directory in the same datanode. That is to say, given a
> root directory, files under the root directory are grouped by paths of their
> parent directories. After that, files of a group are stored in the same
> datanode.
> When a new file is stored at HDFS, the block placement policy choose the
> target datanode considering locations of previously stored files.
> In the current block placement policy interface, there are some problems. The
> first problem is that there is no interface to keep the previously stored
> files when HDFS is restarted. To restore the location information of all
> files, this process should be done during the safe mode of the namenode.
> To solve the first problem, I modified the block placement policy interface
> and FSNamesystem. Before leaving the safe mode, every necessary location
> information is sent to the block placement policy.
> However, there are too much changes of access modifiers from private to
> public in my implementation. This may violate the design of the interface.
> The second problem is occurred when some blocks are moved by the balancer or
> node failures. In this case, the block placement policy should recognize the
> current status, and return a new datanode to move blocks. However, the
> current interface does not support it.
> The attached patch is to solve the first problem, but as mentioned above, it
> may violate the design of the interface.
> Do you have any good ideas?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
