sandy-sachin7 opened a new pull request, #10167: URL: https://github.com/apache/arrow-rs/pull/10167
## Which issue does this PR close? Closes #10139. ## Rationale for this change The `claim()` method on `Bytes` and `MutableBuffer` creates a new pool reservation before dropping the existing one. This causes a transient period where **both** the old and new reservations exist simultaneously, resulting in the memory pool reporting up to **2x the actual memory** in use. The problematic sequence in the old code: ```rust *self.reservation.lock().unwrap() = Some(pool.reserve(self.capacity())); // Equivalent to: // let guard = self.reservation.lock().unwrap(); // old reservation alive // let new_claim = pool.reserve(self.capacity()); // BOTH old + new alive (2x count) // *guard = Some(new_claim); // old dropped, back to 1x ``` For pools with capacity limits, or when another thread reads the pool counter during this window, the pool would report double the actual memory in use. ## What changes are included in this PR? 1. **`arrow-buffer/src/bytes.rs`**: `Bytes::claim` now drops the existing reservation (`guard.take()`) before calling `pool.reserve()`. 2. **`arrow-buffer/src/buffer/mutable.rs`**: Same fix for `MutableBuffer::claim`. 3. **`arrow-buffer/src/bytes.rs`**: Added `test_claim_does_not_double_count` using a `MaxTrackerPool` wrapper that detects transient double-counting deterministically. ## Are these changes tested? Yes — the new `test_claim_does_not_double_count` test uses a custom pool that records the maximum `used()` value observed during any `reserve()` call. Before the fix, claiming the same buffer twice would produce `max_used = 2048` (double the 1024 buffer size). After the fix, `max_used = 1024`. All existing arrow-buffer tests (53) and downstream tests continue to pass. ## Are there any user-facing changes? No API changes. The behavior is identical except that there is no longer a window where the pool counter is double-counted during re-claiming. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
