Github user sethah commented on the issue:
https://github.com/apache/spark/pull/19232
Sure, we all agree there is a mechanism for avoiding overhead. However,
performance tests are very tricky things, 5% is not a huge improvement, and
hard-coding the aggregation depth to `2` limits the utility of using
`treeAggregate`. I think the change is probably fine for just the reason that
`treeAggregate` shouldn't hurt performance and might speed things up. Still, I
don't think there's enough information yet to determine under what
circumstances this actually improves the performance, if any.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
