[
https://issues.apache.org/jira/browse/ARROW-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024131#comment-17024131
]
Ben Sully commented on ARROW-6691:
----------------------------------
My hunch is also that `rayon` would be better suited to this part ('working' on
lots of things in a threadpool, rather than 'waiting' for lots of things) than
tokio + async/await, although it does sound like doing IO using gRPC might be
well suited to tokio. I'm not familiar with the Datafusion codebase at all
unfortunately, how approachable would this issue be for a first time Arrow
contributor? :D
> [Rust] [DataFusion] Use tokio and Futures instead of spawning threads
> ---------------------------------------------------------------------
>
> Key: ARROW-6691
> URL: https://issues.apache.org/jira/browse/ARROW-6691
> Project: Apache Arrow
> Issue Type: Sub-task
> Components: Rust, Rust - DataFusion
> Reporter: Andy Grove
> Priority: Major
> Fix For: 1.0.0
>
> Attachments: image-2019-12-07-17-54-57-862.png
>
>
> The current implementation of the physical query plan uses "thread::spawn"
> which is expensive. We should switch to using Futures, async!/await!, and
> tokio so that we are launching tasks in a thread pool instead and writing
> idiomatic Rust code with futures combinators to chain actions together.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
