[
https://issues.apache.org/jira/browse/HDFS-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liuguanghua updated HDFS-17592:
-------------------------------
Description:
FastCopy is a faster data copy tools. In federation cluster or a single
cluster , FastCopy copy blocks via hardlink. This is more much faster than
original copy.
FastCopy can support data copy via transfer in different nameservices without
federation. In theory, it could reduces one IO transfer and almost reduce halt
time.
Test Data:
blocksize 128M
1TB ECfiles + 1TB 3 replicated files
|distcp map=20|DIstcp via FastCopy(HardLink)|DistCp via
FastCopy(Transfer)|Distcp(original)|
| Time Spent|5m6.687s|22m44.094s|38m17.024s|
was:
FastCopy is a faster data copy tools. In federation cluster or a single
cluster , FastCopy copy blocks via hardlink. This is more much faster than
original copy.
FastCopy can support data copy via transfer in different nameservices without
federation. In theory, it could save almost half the time compared to origianl
copy.
> FastCopy support data copy in different nameservices without federation
> -----------------------------------------------------------------------
>
> Key: HDFS-17592
> URL: https://issues.apache.org/jira/browse/HDFS-17592
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: liuguanghua
> Assignee: liuguanghua
> Priority: Major
> Attachments: FastCopy via Transfer.jpg
>
>
> FastCopy is a faster data copy tools. In federation cluster or a single
> cluster , FastCopy copy blocks via hardlink. This is more much faster than
> original copy.
> FastCopy can support data copy via transfer in different nameservices without
> federation. In theory, it could reduces one IO transfer and almost reduce
> halt time.
>
> Test Data:
> blocksize 128M
> 1TB ECfiles + 1TB 3 replicated files
>
> |distcp map=20|DIstcp via FastCopy(HardLink)|DistCp via
> FastCopy(Transfer)|Distcp(original)|
> | Time Spent|5m6.687s|22m44.094s|38m17.024s|
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
