mattcasters opened a new issue, #8768: URL: https://github.com/apache/hop/issues/8768
### Apache Hop version? 2.20.0-SNAPSHOT ### Java version? 21 (GitHub Actions, Temurin) ### Operating system Linux ### What happened? `mvn clean install` of `hop-tech-arrow` fails on GitHub Actions. It shows up on pull requests that do not touch the Arrow plugin. PR 8717 failed this way: Surefire reran `ArrowFlightDataStreamAuthenticationTest.rowsRoundTripThroughAnAuthenticatedServer` three times and the module failed. The `UNAUTHENTICATED: Invalid credentials for the Hop Arrow Flight server` stacks in the same log are not the failure. `ArrowFlightSecurity` throws that status for a wrong username or password, and Arrow's `ClientHandshakeWrapper` logs it at ERROR before rethrowing. The wrong-password tests assert on that exception, so those tests pass. Each rejected login is logged twice (`Failed on completing future`, then `Failed with unknown`). The round-trip test failed for a different reason: - Run 1: `expected: <0> but was: <null>` at the first field of the first row read back. - Runs 2 and 3: `HopException: When writing data you can't read rows from the same data stream.` from `ArrowFlightDataStream.readRow`. `MemoryMetadataSerializer` keeps the instance it was given and `load` returns that same instance. The test writes and reads through that one `ArrowFlightDataStream`. `HopFlightProducer.lookupArrowStreamBuffer` calls `initialize(..., writing=true)` on it when the server accepts the put. `acceptPut` runs on a gRPC thread, so that call races with the test: - `initialize()` replaces `rowBuffer` and `rootAllocator`. If it runs while the test is still filling the batch, the batch that is sent can contain nulls. That is run 1. - The test then calls `initialize(..., writing=false)` to read. If `acceptPut` runs after that, it sets `writing` back to true and `readRow()` refuses to read. That is runs 2 and 3. The same log shows gRPC channels for `localhost/127.0.0.1` garbage-collected without `shutdown()`. The reader builds a second `FlightClient` on the shared object and drops the writer client. Locally the server thread usually enters `acceptPut` before the test switches the object to reading, so the race does not show up. ### Issue Priority Priority: 2 ### Issue Component Component: Other -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
