damccorm opened a new pull request, #41128:
URL: https://github.com/apache/beam/pull/41128

   ### Summary
   Fixes a test race condition in 
`GrpcCommitWorkStreamTest.testCommitWorkItem_multiplePhysicalStreams_oldStreamFailsWhileNewStreamInBackoff`
 (and the identical pattern in 
`GrpcGetDataStreamTest.testRequestKeyedData_multiplePhysicalStreams_oldStreamFailsWhileNewStreamInBackoff`)
 observed in `PreCommit Java Dataflow Non-portable Worker` 
([#42633140638](https://github.com/apache/beam/actions/runs/42633140638)).
   
   ### Root Cause
   Previously, `assertTrue(triggeredExecutor.unblockNextFuture())` was called 
**before** `fakeService.setFailedStreamConnectsRemaining(1)`. If the scheduled 
handover task (`onHalfClosePhysicalStreamTimeout`) executed on 
`triggeredExecutor` before the test thread set `failedStreamConnectsRemaining = 
1`, the second physical stream (`streamInfo2`) connected immediately while 
`failedStreamConnectsRemaining` was still `0`.
   
   As a result:
   1. `fakeService.waitForFailedConnectAttempts()` blocked for 300 seconds 
(`DEFAULT_STREAM_RPC_DEADLINE_SECONDS`) until `streamInfo2`'s gRPC deadline 
expired and cancelled `streamInfo2`, triggering a reconnect that finally 
consumed `failedStreamConnectsRemaining = 1`.
   2. When `waitForConnectionAndConsumeHeader()` later popped `streamInfo2` 
from `fakeService.commitStreams`, `streamInfo2` was already cancelled by 
deadline expiration, causing `streamInfo2.responseObserver.onNext(...)` to 
throw `StatusRuntimeException: CANCELLED: call already cancelled`.
   
   ### Fix
   Call `fakeService.setFailedStreamConnectsRemaining(1)` **before** unblocking 
the handover future with `assertTrue(triggeredExecutor.unblockNextFuture())` in 
both `GrpcCommitWorkStreamTest` and `GrpcGetDataStreamTest`.
   
   ### Verification
   - Triggered `PreCommit Java Dataflow` workflow run on branch: 
[`42664407099`](https://github.com/apache/beam/actions/runs/42664407099)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to