jbonofre opened a new pull request, #1457: URL: https://github.com/apache/arrow-java/pull/1457
## What's Changed `TestReservationListener#testErrorThrownFromReservationListener` closes its listenable `NativeMemoryPool` right after the scan fails. Arrow C++ can still be running the failed scan's work (Parquet file opens, read-ahead) on its thread pools, and that work keeps allocating from and freeing to the pool. Freeing the pool under it intermittently crashes the test JVM in the RC workflow's Binaries job, and surefire reports `TestAllTypes`, the next test in that JVM, as the crashed test. #1455 has the analysis. - `testErrorThrownFromReservationListener` leaves the pool open, with a comment explaining why. The assertion on the reservation after closing the pool is removed; the one before it stays. - `TestDataset#collectResultFromFactory` closes the scanner and the dataset when the scan fails, not only when it succeeds. This alone doesn't stop the crash, because closing the scanner doesn't wait for that work. I looped the test's steps up to 1,000 times per JVM on macOS arm64 with the `jni-macos-aarch_64` libraries from the RC workflow. With the pool closed, 11 of 14 runs failed (JVM crash, or a scan that should have failed succeeded), whether or not the scanner was closed first. With the pool left open, 10 of 10 runs passed. With this change, `TestReservationListener` passed in 8 of 8 runs, and the other tests that use `collectResultFromFactory` pass (`TestFileSystemDataset`, `TestAllTypes`). I skipped `TestFileSystemDataset#testBaseJsonRead`, which crashes with the macOS library for an unrelated reason (#1429). This doesn't fix the library problem that a listenable pool can be freed while native scan work still uses it. I've described that in #246. Closes #1455. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
