Hi team! I've been implementing integration tests [1] for the examples under the beam/examples/complete/game folder. Most of the integration tests (the one for *GameStats* being the only outlier so far) run successfully when executed *locally* with the *Direct* runner. However, I've found that when executed against the Spark, Flink and Direct runners in jenkins, they fail with a different error depending on the runner. Direct runner on jenkins plays mostly well [2] with all the ITs; again, *GameStats* being the only one that for some reason gets stuck when trying to get a fixedWindow [3]. With Spark [4] three tests fail, two of those being executed successfully by the Direct runner. Flink [5] gives up after two hours executing, possibly due to the same *GameStats* issue. Fortunately, DataFlow executes all the ITs successfully [6], even the *GameStats* one.
I guess my question would be, what could be improved integration testing-wise? How would you pinpoint the root cause in this kind of scenario? Thanks! - Fer [1] https://github.com/apache/beam/pull/17015/files [2] https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Direct/9/testReport/ [3] https://github.com/apache/beam/blob/3e6e8a7e2afa08552849518439577eee07700f77/examples/java/src/main/java/org/apache/beam/examples/complete/game/GameStats.java#L292 [4] https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Spark/7/testReport/ [5] https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Flink/7/ [6] https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2_PR/59/testReport/ -- *This email and its contents (including any attachments) are being sent to you on the condition of confidentiality and may be protected by legal privilege. Access to this email by anyone other than the intended recipient is unauthorized. If you are not the intended recipient, please immediately notify the sender by replying to this message and delete the material immediately from your system. Any further use, dissemination, distribution or reproduction of this email is strictly prohibited. Further, no representation is made with respect to any content contained in this email.*
