geoffreyclaude opened a new pull request, #26389: URL: https://github.com/apache/datafusion/pull/26389
## Which issue does this PR close? Stacked on #26366 at `d81100982dd0837664c8cd694318adb2a4e7aa05`. This draft adds one commit implementing [Emily's review suggestion](https://github.com/apache/datafusion/pull/26366#discussion_r4285749410). [Compare only this additional commit](https://github.com/apache/datafusion/compare/d81100982dd0837664c8cd694318adb2a4e7aa05...1787cda1bd5b69810b2f6090c1fe4819952f7fbb). ## Rationale for this change Evaluate recalculating retained row-buffer memory when an input stream finishes, so its reservation is reconciled with the current cached buffers. The benchmark compares this policy against the incremental release accounting in #26366. ## What changes are included in this PR? - Clear the finished stream's cached buffer and explicitly set its recorded size to zero. - Recompute the total retained size and reconcile the reservation with `try_resize()`. - Propagate errors from release during failed-buffer-growth cleanup and end-of-stream polling. Per-batch accounting in `save()` stays incremental. ## What is the testing strategy for this PR? The exact release variant passes all eight existing row-stream tests, covering reservation ownership, buffer reuse, high-water marks, stream exhaustion, and failed-growth cleanup. Those tests cover the changed behavior, so this commit adds no tests. Request the existing `small_batches/two_keys` sort benchmark against the exact parent head above to measure the cost of this additional commit. ## Are there any user-facing changes? Query results, configuration, and public APIs are unchanged. Full recalculation adds a scan of all input slots on each release; this draft measures its effect on large merges. Developed with AI assistance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
