Antoine Pitrou created ARROW-10788:
--------------------------------------
Summary: [C++] Make S3 recursive calls parallel
Key: ARROW-10788
URL: https://issues.apache.org/jira/browse/ARROW-10788
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: Antoine Pitrou
Assignee: Antoine Pitrou
Doing a recursive S3 directory walk using GetFileInfo(Selector) currently lists
all encountered directories serially, waiting for the results of one directory
listing (or portion thereof) before launching the next one. Instead, we should
use the Async APIs provided by the AWS SDK to parallelize HTTP requests as much
as possible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
