CurtHagenlocher opened a new pull request, #419: URL: https://github.com/apache/arrow-dotnet/pull/419
## What's Changed `Grow` doubled the current length in a `checked` context without saturating: ```csharp int newCount = Math.Max(newElementCount, checked(Length * 2)); ``` So once a buffer passed half of the addressable maximum, its **next** grow threw `OverflowException` however small the requested increase, and even though the requested size still fit. For a `NativeBuffer<byte, …>` that is a hard ceiling near 1 GiB, with no way for a caller to work around it: asking for a smaller increment does not help, because the overflow is in the doubling rather than in the request. Growth now saturates at the largest addressable element count, so it stays amortised right up to the ceiling. A request that genuinely cannot be addressed still fails at the byte-size calculation, as it did before — behaviour is unchanged for anything that could not have worked. This is what the `TODO` those lines carried proposed: > There might be a size that's big enough to work for this case but not too big to overflow. We could > use that instead of blindly doubling. ### On testing it Reaching the boundary through `Grow` means allocating more than a gigabyte, which does not belong in a unit test. The count arithmetic is extracted to `ComputeGrowCount` so the boundary can be tested directly and exhaustively, including the per-element-size ceiling — the limit is a byte count, so a wider element type saturates at proportionally fewer elements. Verified the new tests fail against the previous arithmetic before fixing it: with `checked(Length * 2)` restored, `ComputeGrowCountSaturatesInsteadOfOverflowing` and `ComputeGrowCountSaturatesPerElementSize` both fail; the rest pass either way. `Apache.Arrow.Tests` is green on net8.0: 1870 passed, 28 skipped (the Python interop cases). ### Scope This does not change the 2 GiB ceiling on `ArrowBuffer` itself (`ReadOnlyMemory<byte>`, `int Length`) — it only stops buffers failing at half of it. Closes #418. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
