EmilyMatt commented on issue #20728:
URL: https://github.com/apache/datafusion/issues/20728#issuecomment-4006782189
@mbutrovich
Isn't this by definition called once per *fully* sorted batch?
```
reservation
.try_grow(total_sorted_size)
.map_err(Self::err_with_oom_context)?;
```
I.e., before,
drop(reservation) was called, which only reduced it.
Now we do
reservation.free(), and then reservation.try_grow()
So I'd expect one extra call per chunked sort, which seems reasonable.
How many tasks it that machine running? perhaps there is synchronisation
happening in Spark making it slower?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
