vertexclique commented on a change in pull request #8664:
URL: https://github.com/apache/arrow/pull/8664#discussion_r530343892
##########
File path: rust/arrow/src/compute/kernels/aggregate.rs
##########
@@ -141,32 +146,41 @@ where
match array.data().null_buffer() {
None => {
- let sum = data.iter().fold(T::default_value(), |accumulator,
value| {
- accumulator + *value
- });
+ let total = data
+ .par_iter()
Review comment:
Rayon is initializing global pool with 1.5 * logical cores. Since they
are all spawned but not joined threads, that will easily work from direct users
of arrow. Closures that spawn on though, are executing like fork-join. Coming
to the process forking point of view, while using this library as is, won't
cause problems. In the nested operations case, they will be queued and spawned
to any free slot, all operations are going through global producer-consumer hub
inside rayon (take a gaze into bit ops code in this pr). Sometime later if a
contributor comes and says that, "I want to configure tlp" then we can just
expose the pool config by a method. But since it configures itself on the
machine it's running we can directly skip that part.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
