[
https://issues.apache.org/jira/browse/ARROW-10117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358803#comment-17358803
]
Weston Pace commented on ARROW-10117:
-------------------------------------
Other than the minor improvement to thread pool overhead described above I saw
no real significant improvements with the work stealing thread pool. I'd like
to get ARROW-12878 merged as it will make future experiments on thread pool
strategies easier to perform. We could perhaps merge ARROW-12902 but I could
understand if there was a desire to wait for some actual macro-improvement
before doing so. I've attached a summary of my
investigation.[^Work_Stealing_Thread_Pool_Investigation.pdf]
> [C++] Implement work-stealing scheduler / multiple queues in ThreadPool
> -----------------------------------------------------------------------
>
> Key: ARROW-10117
> URL: https://issues.apache.org/jira/browse/ARROW-10117
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Wes McKinney
> Assignee: Weston Pace
> Priority: Major
> Attachments: Work_Stealing_Thread_Pool_Investigation.pdf
>
>
> This involves a change from a single task queue shared amongst all threads to
> a per-thread task queue and the ability for idle threads to take tasks from
> other threads' queues (work stealing).
> As part of this, the task submission API would need to be evolved in some
> fashion to allow for tasks related to a particular workload to end up in the
> same task queue
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
