[
https://issues.apache.org/jira/browse/ARROW-15035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vibhatha Lakmal Abeykoon reassigned ARROW-15035:
------------------------------------------------
Assignee: Vibhatha Lakmal Abeykoon
> [C++] Create I/O thread pools per filesystem
> --------------------------------------------
>
> Key: ARROW-15035
> URL: https://issues.apache.org/jira/browse/ARROW-15035
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Weston Pace
> Assignee: Vibhatha Lakmal Abeykoon
> Priority: Major
>
> The IOContext gets us most of the way here but we still don't do this yet
> today. One concrete advantage to this is it allows us to more intelligently
> set the number of I/O threads.
> For example, 8 threads is often too small for an S3 filesystem (ARROW-14965)
> On the other hand, in some cases, 8 threads can be too many for an HDD
> (ARROW-14354)
> I doubt we will be able to figure out the ideal size of the I/O thread pool
> for any filesystem (e.g. on an S3 filesystem it depends on how many cores you
> have and how much bandwidth the system has) but we can possibly have more
> sensible defaults.
> Furthermore, it will hopefully clarify to the user the connection between
> filesystem and I/O thread pool size.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
