DanielLeens commented on issue #10930: URL: https://github.com/apache/seatunnel/issues/10930#issuecomment-4523792008
Thanks for the detailed report, and also for opening PR #10931. I traced the current cleanup path in `ArrowToSeatunnelRowReader`, and this does look like a real bug. The issue is in the Arrow resource cleanup path, but the main problem appears to be the close order rather than the `ByteArrayInputStream` itself. Right now `readArrow()` always ends by calling `close()`, and `close()` shuts down the allocator before the `ArrowStreamReader`. That means Arrow can still have buffers associated with the reader when the allocator is already being closed, which matches the leaked-memory exception in your stack trace. So the direction of the fix is correct, and PR #10931 is in the right area. Let's continue the implementation discussion there instead of splitting it across multiple threads. Thanks again for narrowing this down and sending the fix quickly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
