CurtHagenlocher opened a new pull request, #504:
URL: https://github.com/apache/arrow-dotnet/pull/504

   > **Draft for feedback.** This prototype is shared to get review on the 
overall design before it
   > is split into smaller PRs. Several parts are independent and could land 
separately; see
   > [Plan](#plan).
   
   ## What this does
   
   Lets callers choose how Arrow memory is allocated, and makes the behaviours 
that cause problems
   today opt-out rather than unavoidable. **Code that passes no allocator 
behaves as it does now**:
   the new behaviours are opted into by passing an allocator, so working 
applications that depend
   on today's behaviour are not affected.
   
   It addresses, fully or as an opt-in:
   
   - **#397**: spans over native buffers can outlive their memory, because 
`NativeMemoryManager`
     and `SharedMemoryHandle` release memory from finalizers.
   - **#261**: per-allocation `GC.AddMemoryPressure` calls trigger excessive 
collections.
   - **#267**: no managed allocator.
   - **#97**: inflexible builders (no capacity, extra copies on `Build`).
   - **#99**: `NativeMemory` on modern .NET.
   - **#50**: memory pressure not fully removed for non-default alignments.
   
   ## Design
   
   ### Allocators and lifetimes
   
   `MemoryAllocator.Lifetime` describes what happens to a buffer that is 
**never** disposed:
   
   | `AllocationLifetime` | Undisposed buffer | Example |
   |---|---|---|
   | `Finalized` | Released by a finalizer, even while a span is in use (#397) 
| `NativeMemoryAllocator` (the default, unchanged) |
   | `GarbageCollected` | Collected; a span keeps it alive | 
`ManagedMemoryAllocator`, `PooledManagedAllocator` |
   | `CallerOwned` | Leaked; must be disposed | `CallerOwnedNativeAllocator` |
   
   Whatever the lifetime, spans must not be used after their buffer is 
disposed. The new allocators:
   
   - **`ManagedMemoryAllocator`**: plain managed arrays, no pooling. The safe 
opt-out from #397: no
     disposal needed, and spans can never see freed or reused memory.
   - **`PooledManagedAllocator`**: reuses arrays, bounded by 
`PooledManagedAllocatorOptions`
     (default: up to 16 arrays of each power-of-two size up to 1 MiB, about 32 
MiB at most), with an
     option to poison disposed buffers to expose use after disposal.
   - **`CallerOwnedNativeAllocator`**: 64-byte-aligned native memory with no 
finalizer and no
     memory pressure (`NativeMemory.AlignedAlloc` on .NET 6+).
   
   `SharedMemoryHandle` keeps its finalizer only for `Finalized` allocators, so 
the default and
   existing custom allocators behave as before.
   
   ### Builders
   
   Builders have a *temporary* allocator for growth and a *long-term* one for 
the built buffers:
   
   - **No allocator** (all existing code): grow in managed memory, copy once 
into
     `MemoryAllocator.Default` on `Build`. This is today's behaviour, verified 
to make one native
     allocation per buffer.
   - **Allocator X**: grow in X and **hand the working memory to the built 
array** instead of
     copying it. A builder created with its final capacity allocates exactly 
once:
   
     ```csharp
     using var builder = new Int32Array.Builder(100, allocator);
     Span<int> values = builder.GetSpan(100);   // fill directly
     using Int32Array array = builder.Advance(100).Build();
     ```
   
   Also: capacity and allocator constructors (shipped constructors are 
unchanged), `GetSpan` and
   `Advance` on fixed-width builders, `IDisposable` on array builders, a 
validity bitmap that is
   created only at the first null, and child builders, `Clone`, concatenation, 
the `RecordBatch`
   column helpers and variant shredding building directly in their target 
allocator.
   
   ### Contexts for IPC and Flight
   
   Operations take an `ArrowContext` (allocator, codec factory, extension 
registry); builders take a
   `MemoryAllocator`.
   
   - **IPC writers** get context constructors. A codec factory in `IpcOptions` 
wins; the context's
     is a fallback.
   - **Flight**: `FlightClient`, `FlightServer`, `FlightSqlServer` and the 
stream readers and
     writers accept a context. It also works when the server is created by 
dependency injection, and
     when the client is created by `Grpc.Net.ClientFactory` (see 
[Questions](#questions-for-reviewers)).
   - `ExtensionTypeRegistry.Clone` is now copy-on-write, so a context's 
snapshot of the default
     registry is cheap and never sees later changes.
   
   ## Behaviour and API changes
   
   - **`IDisposable` on array builders.** Binary- and source-compatible; 
projects with CA2000
     enabled will see new warnings. Disposal is only *required* with 
caller-owned allocators.
   - **New dependency:** Apache.Arrow.Flight now references
     `Microsoft.Extensions.DependencyInjection.Abstractions` (6.0.0) for
     `[ActivatorUtilitiesConstructor]`.
   - **Bug fixes that change output:**
     - The builder factory honours a timestamp element type's unit and time 
zone; a
       `List<timestamp[us, UTC]>` builder used to produce millisecond values 
with no time zone.
     - Flight decodes schemas with the reader's extension registry instead of 
always the default.
   - **`Set` and `Swap`** on primitive builders check the index before writing; 
the exception type
     is unchanged.
   - With an explicit allocator, buffer builder capacity is rounded up to 64 
bytes.
   
   ## Bug fixes that could land on their own
   
   - **#397 writer case:** `ArrowStreamWriter` now keeps the record batch alive 
while copying its
     buffers. A Release-mode reproduction crashed the runtime without the fix 
and is clean with it.
     Debug builds extend lifetimes, so there is no unit test for it.
   - **#50**, the timestamp factory fix, Flight's extension registry, and 
copy-on-write registry
     snapshots.
   
   ## Performance
   
   Measured with `DefaultPathBuilderBenchmark`, which uses only long-standing 
APIs so it runs
   against `main` too.
   
   - **Default path (no allocator): parity or faster than `main` in every 
case.**
     - Single-threaded: 2–83% faster, except `Append(span)` on 100 values (+9%, 
~30 ns).
     - Parallel builder growth: about 25% faster.
     - Small builds allocate about 30% more managed memory, without being 
slower.
   - **Opt-in hand-over** makes a builder given its final capacity allocate 
once.
   - The remaining per-call cost of `GetSpan`/`Advance` is about 3 ns; both 
inline fully.
   - Benchmarks ending in `Build()` with the default native allocator are 
dominated by
     memory-pressure GCs (#261) and are bimodal between launches, so use 
several launches.
   
   ## Questions for reviewers
   
   1. **`IDisposable` on builders.** Is that acceptable? The alternative is a 
second set of
      builders.
   2. **The DI abstractions dependency** for Flight, versus offering only the 
existing
      `FlightClient.Create(CallInvoker, ArrowContext)` factory. Both are 
currently present. `Create`
      (#285) is unreleased and now redundant; should it be removed?
   3. **Should the temporary and long-term allocators be exposed publicly**, or 
on `ArrowContext`?
      Builders already work this way internally.
   4. **#261 batching** could be added later as an opt-in 
`INativeAllocationTracker` on
      `NativeMemoryAllocator`. Should it be in scope?
   5. **`PooledManagedAllocator` defaults** (1 MiB, 16 per size). And should 
lifetime become
      per-buffer, to allow composite allocators such as pooled small buffers 
with native large ones?
   6. **The hand-over keeps up to 50% unused capacity** rather than copying. Is 
that the right
      trade-off?
   
   ## Not yet done
   
   - Capacity constructors for Boolean, Binary/String, Decimal, FixedSizeBinary 
and nested builders
     (they have allocator constructors), and `GetSpan` on the converting 
builders (Date, Time,
     Timestamp).
   - #97's allocation without zero-filling, read-write arrays, and more 
flexible nested builders.
   - In-place `Resize` for the native allocators, and trimming the pool over 
time.
   
   ## Plan
   
   Once the shape is agreed, split along the commits here: the independent 
fixes first, then
   allocators and lifetimes, builders, IPC contexts, Flight, and benchmarks.
   
   ## Testing
   
   New tests check where every buffer comes from and that everything is 
released after disposal,
   using a counting caller-owned allocator. The key behaviours were checked by 
deliberately breaking
   them and confirming the tests fail. All suites pass on net462, net472 and 
net8.0, including the
   pyarrow C Data tests.
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   
   Co-Authored-By: Claude Opus 5.5 <[email protected]>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to