davidzollo opened a new pull request, #10958: URL: https://github.com/apache/seatunnel/pull/10958
## Purpose Fix `IllegalStateException: Memory was leaked by query` when using the Doris connector (and any connector backed by `ArrowToSeatunnelRowReader`). Closes #9863 ## Problem In `ArrowToSeatunnelRowReader.close()`, the previous close order was: 1. `root.close()` — close `VectorSchemaRoot` 2. `rootAllocator.close()` — close `RootAllocator` ← **wrong: arrowStreamReader still holds unreleased buffers** 3. `arrowStreamReader.close()` — too late, allocator already closed Apache Arrow's memory management requires that all child allocations are released **before** the `RootAllocator` is closed. The `ArrowStreamReader` internally allocates Arrow buffers via the `RootAllocator`. Closing the allocator while the reader still holds references causes the allocator to detect leaked memory (~64 bytes of internal metadata) and throw: ``` java.lang.IllegalStateException: Memory was leaked by query. Memory leaked: (64) Allocator(ROOT) 0/64/64/2147483647 (res/actual/peak/limit) ``` This exception propagated up through `DorisValueReader.hasNext()` → `DorisSourceReader.pollNext()` → `SourceFlowLifeCycle.collect()` causing the task to fail with state FAILED. ## Fix Corrected the close order to: 1. `arrowStreamReader.close()` — releases all Arrow buffers and internally closes `VectorSchemaRoot` 2. `rootAllocator.close()` — now safe, all allocations already released The separate `root.close()` call is removed since `ArrowStreamReader.close()` already handles it. ## Impact - **Affected connectors**: Doris, StarRocks, and any connector using `ArrowToSeatunnelRowReader` - **Trigger**: Any read operation that creates an `ArrowToSeatunnelRowReader` instance - **Backward compatible**: Yes, pure bug fix with no API or behavior changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
