[
https://issues.apache.org/jira/browse/HADOOP-17998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17441639#comment-17441639
]
Chengwei Wang commented on HADOOP-17998:
----------------------------------------
Test case: 1 dir 240 files 13G
*1. Get with single thread.*
{code:java}
time hadoop fs -get /tmp/data/20211101 .
real 6m28.785s
user 0m18.844s
sys 0m44.953s
{code}
*2. Get with 10 threads.*
{code:java}
time hadoop fs -get -t 10 /tmp/data/20211101 .
real 0m58.721s
user 0m21.386s
sys 0m54.066s
{code}
> Enable get command run with multi-thread
> ----------------------------------------
>
> Key: HADOOP-17998
> URL: https://issues.apache.org/jira/browse/HADOOP-17998
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Chengwei Wang
> Priority: Major
> Attachments: HADOOP-17998.001.patch, HADOOP-17998.002.patch
>
>
> CopyFromLocal/Put is enabled to run with multi-thread with HDFS-11786 and
> HADOOP-14698， and make put dirs or multiple files faster.
> So, It's necessary to enable get and copyToLocal command run with
> multi-thread.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
