jorgecarleitao commented on a change in pull request #8664:
URL: https://github.com/apache/arrow/pull/8664#discussion_r530329221
##########
File path: rust/arrow/src/compute/kernels/aggregate.rs
##########
@@ -141,32 +146,41 @@ where
match array.data().null_buffer() {
None => {
- let sum = data.iter().fold(T::default_value(), |accumulator,
value| {
- accumulator + *value
- });
+ let total = data
+ .par_iter()
Review comment:
`rayon` is pretty smart in this respect, as its execution model starts
the first part of the iterator immediately.
I am thankful that @vertexclique introduced this, as I also believe that we
greatly benefit from multi-threading at this level.
What I am a bit unsure is whether there is any issue in nested spawning: we
may want to fork in other places that are nested with this op. But I am
confident that someone knows the answer (👀 to @vertexclique and @alamb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
