Copilot commented on code in PR #504:
URL: https://github.com/apache/arrow-dotnet/pull/504#discussion_r4581130062


##########
src/Apache.Arrow/Ipc/ArrowReaderImplementation.cs:
##########
@@ -230,27 +294,39 @@ private List<IArrowArray> BuildArrays(
                 return arrays;
             }
 
-            using var bufferCreator = 
GetBufferCreator(recordBatchMessage.Compression);
-            var recordBatchEnumerator = new RecordBatchEnumerator(in 
recordBatchMessage);
-            int schemaFieldIndex = 0;
-            do
+            try
             {
-                Field field = schema.GetFieldByIndex(schemaFieldIndex++);
-                Flatbuf.FieldNode fieldNode = 
recordBatchEnumerator.CurrentNode;
+                using var bufferCreator = 
GetBufferCreator(recordBatchMessage.Compression, body);
+                var recordBatchEnumerator = new RecordBatchEnumerator(in 
recordBatchMessage);
+                int schemaFieldIndex = 0;
+                do
+                {
+                    Field field = schema.GetFieldByIndex(schemaFieldIndex++);
+                    Flatbuf.FieldNode fieldNode = 
recordBatchEnumerator.CurrentNode;
 
-                ArrayData arrayData = LoadField(version, ref 
recordBatchEnumerator, field, in fieldNode, messageBuffer, bufferCreator);
+                    ArrayData arrayData = LoadField(version, ref 
recordBatchEnumerator, field, in fieldNode, messageBuffer, bufferCreator);
 
-                arrays.Add(ArrowArrayFactory.BuildArray(arrayData));
-            } while (recordBatchEnumerator.MoveNextNode());
+                    arrays.Add(ArrowArrayFactory.BuildArray(arrayData));

Review Comment:
   If `LoadField` has already created one or more buffers in `arrowBuff` and 
then parsing a later buffer or child throws, the `BuildArrays` catch only 
disposes arrays that were already added to `arrays`; the partially built 
`ArrayData` was never added, so those buffers are leaked. This is observable 
with a caller-owned allocator (for example, malformed IPC after a decompressed 
buffer was allocated), and the new allocator path makes the leak permanent. 
Track and dispose the in-progress buffers/children when `LoadField` fails, or 
transfer ownership only after the whole field has been built.



##########
src/Apache.Arrow.Operations/Shredding/ShreddedVariantArrayBuilder.cs:
##########
@@ -83,7 +83,7 @@ public static VariantArray Build(
             // metadata column: emit the shared bytes once per row. (A 
dictionary-encoded
             // or run-end-encoded representation would compress this; 
VariantArray's reader
             // already handles those, but for simplicity we emit the plain 
binary form.)
-            BinaryArray.Builder metadataBuilder = new BinaryArray.Builder();
+            using BinaryArray.Builder metadataBuilder = new 
BinaryArray.Builder(BinaryType.Default, allocator);

Review Comment:
   The top-level validity bitmap is still constructed with `new 
ArrowBuffer.BitmapBuilder(rowCount)` rather than the supplied `allocator`, so 
an explicit allocator causes this bitmap to grow in managed scratch memory and 
then be copied during `validity.Build(allocator)`. That leaves this shredding 
path with an extra allocation/copy, unlike the allocator-backed child builders 
added below; construct the validity builder with the same allocator so the 
output can be handed over directly.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to