tustvold opened a new pull request, #2226:
URL: https://github.com/apache/arrow-datafusion/pull/2226
# Which issue does this PR close?
Closes #2199.
# Rationale for this change
See ticket
# What changes are included in this PR?
Adds a new datafusion-scheduler crate that provides an implementation of a
push-based, morsel-driven scheduler based on rayon. More details, background,
and discussion can be found in the proposal document
[here](https://docs.google.com/document/d/1txX60thXn1tQO1ENNT8rwfU3cXLofa7ZccnvP4jD6AA),
please feel free to comment there.
I would describe the current implementation as an MVP, and there are
definitely things that could be improved, but I would like to propose we get
something merged and can then iterate on it further. My hope is that we can
iteratively refine the implementation, and then once we are happy with it begin
the process of refactoring the operators to better align with it.
# Are there any user-facing changes?
No, this is an entirely new crate, and does not require changes to any
others.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
