xuchen-plus commented on issue #12136:
URL: https://github.com/apache/datafusion/issues/12136#issuecomment-2652619364
> Before sorting we have 6233344 bytes of unsorted data
> After sorting we now have 4932480 bytes of sorted data
> Spilling
> Before sorting we have 6233344 bytes of unsorted data
> After sorting we now have 4932480 bytes of sorted data
> Spilling
> Before sorting we have 6233344 bytes of unsorted data
> After sorting we now have 4932480 bytes of sorted data
> Spilling
> Before sorting we have 6233344 bytes of unsorted data
> After sorting we now have 4932480 bytes of sorted data
> Spilling
> Before sorting we have 6233344 bytes of unsorted data
> After sorting we now have 4932480 bytes of sorted data
> Failed with error:
>
> Before sorting we have 6233344 bytes of unsorted data
> After sorting we now have 4932480 bytes of sorted data
> Spilling
> Before sorting we have 6233344 bytes of unsorted data
> thread 'main' panicked at
datafusion-examples/examples/large_string_sort.rs:62:10:
> called `Result::unwrap()` on an `Err` value: ResourcesExhausted("Failed to
allocate additional 353536 bytes for ExternalSorter[1] with 7138944 bytes
already allocated for this reservation - 7489828 bytes remain available for the
total pool")
@zhuqi-lucas Sorry. I actually adjusted `sort_spill_reservation_bytes` to
32MB and the above test passed. When `sort_spill_reservation_bytes` is 16MB the
test would failed to reserve extra memory during sort.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
