ump45nose opened a new pull request, #483: URL: https://github.com/apache/arrow-dotnet/pull/483
## What's Changed `CArrowArrayStreamExporter.GetNext` and `CArrowDeviceArrayStreamExporter.GetNext` read a batch from the managed stream, export it and never dispose it. The exported array takes its own references to the buffers, so releasing it on the consumer side doesn't free the batch's references, and with `NativeMemoryAllocator` the memory is only reclaimed by the finalizer. This follows the third option from the issue: the exported array takes ownership of the batch. - `ExportedAllocationOwner` gets `TakeOwnership(IDisposable)`. Owned objects are disposed in `Dispose`, after the exported buffers have been unpinned and their shared handles released, so memory managers that throw when disposed while pinned are not affected. This also covers buffers exported by pinning rather than through a shared handle. - `CArrowArrayExporter` and `CArrowDeviceArrayExporter` get internal `ExportRecordBatch` overloads with a `transferOwnership` flag. The stream exporters pass `true`, so a batch is disposed when the consumer releases the exported array (or right away if the export fails). The public `ExportRecordBatch` methods still leave the batch to the caller. Batches returned by an `IArrowArrayStream` are owned by the caller, so streams that keep using a batch after handing it out would now see it disposed once the consumer releases it. Tests: `ExportedArrayStreamDisposesBatchesOnRelease` and `ExportedDeviceStreamDisposesBatchesOnRelease` export a stream whose batch is built with `TestMemoryAllocator`, import it again, and check that disposing the imported batch returns every rented buffer. Both fail on `main` (buffers stay rented). The full `Apache.Arrow.Tests` suite passes on net8.0 locally (the pythonnet tests were skipped since Python isn't set up here). This PR was prepared with AI assistance. Closes #464. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
