jbonofre opened a new pull request, #1333: URL: https://github.com/apache/arrow-java/pull/1333
## What's Changed `TestDoExchange.tearDown` is flaky (most recently in `testServerCancelLeak` on the macOS job) with: ``` java.lang.IllegalStateException: Memory was leaked by query. Memory leaked: (8192) Allocator(ROOT) 0/0/16384/2147483647 (res/actual/peak/limit) ``` This is a shutdown race rather than a real leak: the message reports 8192 leaked bytes, but the allocator state printed right after already shows 0 bytes allocated. gRPC reports the server as terminated once its transports are closed, without waiting for the calls still running on the executor. `FlightServer.close()` returned at that point, while such a call could still be parsing an incoming message (which allocates a buffer, discarded right after because the stream is already closed). Closing the allocator in that window reports a leak. `FlightServer.close()` now also waits for the executor it created, within the existing 6 seconds budget, and logs a warning if the executor does not terminate in time. Nothing changes for user-supplied executors (Flight does not own them), nor for `shutdown()` / `awaitTermination()`. Tests: - `TestServerOptions.closeWaitsForDefaultExecutor` is a new test that fails without the fix. - `TestDoExchange.testClientClose` leaked the allocator on purpose to work around the same race (`TODO(ARROW-9586)`). This workaround is removed. I reproduced the flakiness locally by looping the tests in parallel JVMs (to get CPU contention): | Scenario | Before | After | |---|---|---| | `testServerCancelLeak` | 61 failures / 12,000 runs | 0 / 12,000 | | `testClientClose` without the workaround | 1,250 failures / 7,200 runs | 0 / 7,200 | Closes #384. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
