pj-workspace opened a new pull request, #1290: URL: https://github.com/apache/arrow-java/pull/1290
## What's Changed When record batch serialization fails after retaining or compressing buffers, `VectorUnloader` can leave those buffers alive even after the source vectors are closed. Compression allocation failures can also leave an extra reference to the input buffer in `AbstractCompressionCodec`. Release collected batch buffers if traversal or batch construction throws, while preserving the original exception. Use try-with-resources in `AbstractCompressionCodec.compress` so the codec releases the input it owns on both success and failure. Cleanup remains in the component that owns each buffer: the unloader does not release an input already handed to a custom codec. Regression coverage checks allocator exhaustion on the first and a later compression allocation, runtime exceptions and errors, an empty input buffer, and a malformed later vector. The tests assert memory usage and reference counts return to their previous values and that the source data remains readable. Additional tests exercise allocation failure with the real LZ4 and ZSTD codecs. The memory-leak report and initial outer-cleanup proposal are from #1234; this change also handles the codec's input ownership. ## Validation - Before the fix: all 6 new vector regression cases fail; the 5 existing unload/load tests pass. - After the fix: 11 vector cases pass on each of Netty and Unsafe; 53 compression cases pass, including LZ4/ZSTD allocation failures and compressed stream/file round trips (75 test executions, no failures/errors/skips). - JDK 17 compilation and Checkstyle passed. Spotless check passed on JDK 23; `git diff --check` passed. ```sh mvn -B -ntp -pl compression -am test -Dtest=TestVectorUnloadLoad,TestVectorUnloaderFailure,TestCompressionCodec,TestArrowReaderWriterWithCompression,TestCompressionCodecServiceProvider -Dsurefire.failIfNoSpecifiedTests=false ``` - No full repository test run, native dataset/Flight integration run or performance benchmark is claimed. Closes #1234. AI assistance: OpenAI Codex was used for implementation and local validation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
