CurtHagenlocher opened a new pull request, #504:
URL: https://github.com/apache/arrow-dotnet/pull/504
> **Draft for feedback.** This prototype is shared to get review on the
overall design before it
> is split into smaller PRs. Several parts are independent and could land
separately; see
> [Plan](#plan).
## What this does
Lets callers choose how Arrow memory is allocated, and makes the behaviours
that cause problems
today opt-out rather than unavoidable. **Code that passes no allocator
behaves as it does now**:
the new behaviours are opted into by passing an allocator, so working
applications that depend
on today's behaviour are not affected.
It addresses, fully or as an opt-in:
- **#397**: spans over native buffers can outlive their memory, because
`NativeMemoryManager`
and `SharedMemoryHandle` release memory from finalizers.
- **#261**: per-allocation `GC.AddMemoryPressure` calls trigger excessive
collections.
- **#267**: no managed allocator.
- **#97**: inflexible builders (no capacity, extra copies on `Build`).
- **#99**: `NativeMemory` on modern .NET.
- **#50**: memory pressure not fully removed for non-default alignments.
## Design
### Allocators and lifetimes
`MemoryAllocator.Lifetime` describes what happens to a buffer that is
**never** disposed:
| `AllocationLifetime` | Undisposed buffer | Example |
|---|---|---|
| `Finalized` | Released by a finalizer, even while a span is in use (#397)
| `NativeMemoryAllocator` (the default, unchanged) |
| `GarbageCollected` | Collected; a span keeps it alive |
`ManagedMemoryAllocator`, `PooledManagedAllocator` |
| `CallerOwned` | Leaked; must be disposed | `CallerOwnedNativeAllocator` |
Whatever the lifetime, spans must not be used after their buffer is
disposed. The new allocators:
- **`ManagedMemoryAllocator`**: plain managed arrays, no pooling. The safe
opt-out from #397: no
disposal needed, and spans can never see freed or reused memory.
- **`PooledManagedAllocator`**: reuses arrays, bounded by
`PooledManagedAllocatorOptions`
(default: up to 16 arrays of each power-of-two size up to 1 MiB, about 32
MiB at most), with an
option to poison disposed buffers to expose use after disposal.
- **`CallerOwnedNativeAllocator`**: 64-byte-aligned native memory with no
finalizer and no
memory pressure (`NativeMemory.AlignedAlloc` on .NET 6+).
`SharedMemoryHandle` keeps its finalizer only for `Finalized` allocators, so
the default and
existing custom allocators behave as before.
### Builders
Builders have a *temporary* allocator for growth and a *long-term* one for
the built buffers:
- **No allocator** (all existing code): grow in managed memory, copy once
into
`MemoryAllocator.Default` on `Build`. This is today's behaviour, verified
to make one native
allocation per buffer.
- **Allocator X**: grow in X and **hand the working memory to the built
array** instead of
copying it. A builder created with its final capacity allocates exactly
once:
```csharp
using var builder = new Int32Array.Builder(100, allocator);
Span<int> values = builder.GetSpan(100); // fill directly
using Int32Array array = builder.Advance(100).Build();
```
Also: capacity and allocator constructors (shipped constructors are
unchanged), `GetSpan` and
`Advance` on fixed-width builders, `IDisposable` on array builders, a
validity bitmap that is
created only at the first null, and child builders, `Clone`, concatenation,
the `RecordBatch`
column helpers and variant shredding building directly in their target
allocator.
### Contexts for IPC and Flight
Operations take an `ArrowContext` (allocator, codec factory, extension
registry); builders take a
`MemoryAllocator`.
- **IPC writers** get context constructors. A codec factory in `IpcOptions`
wins; the context's
is a fallback.
- **Flight**: `FlightClient`, `FlightServer`, `FlightSqlServer` and the
stream readers and
writers accept a context. It also works when the server is created by
dependency injection, and
when the client is created by `Grpc.Net.ClientFactory` (see
[Questions](#questions-for-reviewers)).
- `ExtensionTypeRegistry.Clone` is now copy-on-write, so a context's
snapshot of the default
registry is cheap and never sees later changes.
## Behaviour and API changes
- **`IDisposable` on array builders.** Binary- and source-compatible;
projects with CA2000
enabled will see new warnings. Disposal is only *required* with
caller-owned allocators.
- **New dependency:** Apache.Arrow.Flight now references
`Microsoft.Extensions.DependencyInjection.Abstractions` (6.0.0) for
`[ActivatorUtilitiesConstructor]`.
- **Bug fixes that change output:**
- The builder factory honours a timestamp element type's unit and time
zone; a
`List<timestamp[us, UTC]>` builder used to produce millisecond values
with no time zone.
- Flight decodes schemas with the reader's extension registry instead of
always the default.
- **`Set` and `Swap`** on primitive builders check the index before writing;
the exception type
is unchanged.
- With an explicit allocator, buffer builder capacity is rounded up to 64
bytes.
## Bug fixes that could land on their own
- **#397 writer case:** `ArrowStreamWriter` now keeps the record batch alive
while copying its
buffers. A Release-mode reproduction crashed the runtime without the fix
and is clean with it.
Debug builds extend lifetimes, so there is no unit test for it.
- **#50**, the timestamp factory fix, Flight's extension registry, and
copy-on-write registry
snapshots.
## Performance
Measured with `DefaultPathBuilderBenchmark`, which uses only long-standing
APIs so it runs
against `main` too.
- **Default path (no allocator): parity or faster than `main` in every
case.**
- Single-threaded: 2–83% faster, except `Append(span)` on 100 values (+9%,
~30 ns).
- Parallel builder growth: about 25% faster.
- Small builds allocate about 30% more managed memory, without being
slower.
- **Opt-in hand-over** makes a builder given its final capacity allocate
once.
- The remaining per-call cost of `GetSpan`/`Advance` is about 3 ns; both
inline fully.
- Benchmarks ending in `Build()` with the default native allocator are
dominated by
memory-pressure GCs (#261) and are bimodal between launches, so use
several launches.
## Questions for reviewers
1. **`IDisposable` on builders.** Is that acceptable? The alternative is a
second set of
builders.
2. **The DI abstractions dependency** for Flight, versus offering only the
existing
`FlightClient.Create(CallInvoker, ArrowContext)` factory. Both are
currently present. `Create`
(#285) is unreleased and now redundant; should it be removed?
3. **Should the temporary and long-term allocators be exposed publicly**, or
on `ArrowContext`?
Builders already work this way internally.
4. **#261 batching** could be added later as an opt-in
`INativeAllocationTracker` on
`NativeMemoryAllocator`. Should it be in scope?
5. **`PooledManagedAllocator` defaults** (1 MiB, 16 per size). And should
lifetime become
per-buffer, to allow composite allocators such as pooled small buffers
with native large ones?
6. **The hand-over keeps up to 50% unused capacity** rather than copying. Is
that the right
trade-off?
## Not yet done
- Capacity constructors for Boolean, Binary/String, Decimal, FixedSizeBinary
and nested builders
(they have allocator constructors), and `GetSpan` on the converting
builders (Date, Time,
Timestamp).
- #97's allocation without zero-filling, read-write arrays, and more
flexible nested builders.
- In-place `Resize` for the native allocators, and trimming the pool over
time.
## Plan
Once the shape is agreed, split along the commits here: the independent
fixes first, then
allocators and lifetimes, builders, IPC contexts, Flight, and benchmarks.
## Testing
New tests check where every buffer comes from and that everything is
released after disposal,
using a counting caller-owned allocator. The key behaviours were checked by
deliberately breaking
them and confirming the tests fail. All suites pass on net462, net472 and
net8.0, including the
pyarrow C Data tests.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]