[jira] [Commented] (HDFS-15549) Improve DISK/ARCHIVE movement if they are on same filesystem
[ https://issues.apache.org/jira/browse/HDFS-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261690#comment-17261690 ] Jing Zhao commented on HDFS-15549: -- Thank you for working on this, [~LeonG]! I went through the current patch and it looks good to me in general. Please feel free to upload the complete version when it's ready. > Improve DISK/ARCHIVE movement if they are on same filesystem > > > Key: HDFS-15549 > URL: https://issues.apache.org/jira/browse/HDFS-15549 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Leon Gao >Assignee: Leon Gao >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > When moving blocks between DISK/ARCHIVE, we should prefer the volume on the > same underlying filesystem and use "rename" instead of "copy" to save IO. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15549) Improve DISK/ARCHIVE movement if they are on same filesystem
[ https://issues.apache.org/jira/browse/HDFS-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261673#comment-17261673 ] Leon Gao commented on HDFS-15549: - I have put a draft patch with some tests. I tried to use hardlink to ensure durability when renaming the block files. Directly using filesystem "rename" can cause data loss when only block/metadata file is moved during datanode restart. [~hexiaoqiao] [~jingzhao] [~jianghuazhu] Could you please take a look and let me know your thoughts. > Improve DISK/ARCHIVE movement if they are on same filesystem > > > Key: HDFS-15549 > URL: https://issues.apache.org/jira/browse/HDFS-15549 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Leon Gao >Assignee: Leon Gao >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > When moving blocks between DISK/ARCHIVE, we should prefer the volume on the > same underlying filesystem and use "rename" instead of "copy" to save IO. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15549) Improve DISK/ARCHIVE movement if they are on same filesystem
[ https://issues.apache.org/jira/browse/HDFS-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192644#comment-17192644 ] Leon Gao commented on HDFS-15549: - [~jianghuazhu] I think this will depend on the implementation of HDFS-15548 > Improve DISK/ARCHIVE movement if they are on same filesystem > > > Key: HDFS-15549 > URL: https://issues.apache.org/jira/browse/HDFS-15549 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Leon Gao >Assignee: Leon Gao >Priority: Major > > When moving blocks between DISK/ARCHIVE, we should prefer the volume on the > same underlying filesystem and use "rename" instead of "copy" to save IO. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15549) Improve DISK/ARCHIVE movement if they are on same filesystem
[ https://issues.apache.org/jira/browse/HDFS-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192089#comment-17192089 ] jianghua zhu commented on HDFS-15549: - [~LeonG] , have you encountered some difficult problems? > Improve DISK/ARCHIVE movement if they are on same filesystem > > > Key: HDFS-15549 > URL: https://issues.apache.org/jira/browse/HDFS-15549 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Leon Gao >Assignee: Leon Gao >Priority: Major > > When moving blocks between DISK/ARCHIVE, we should prefer the volume on the > same underlying filesystem and use "rename" instead of "copy" to save IO. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
