Thanks for all your work to keep our infra stable! Thanks, Danny
On Mon, Nov 10, 2025 at 5:17 AM Vitaly Terentyev via dev < [email protected]> wrote: > Dear Community, > > October was a more turbulent month for Beam Infrastructure, with health > briefly dipping to ~92% in mid-month before recovering to around 97–98% > by the end of the period. The primary driver of instability was the > introduction of support for Python 3.13, which temporarily affected > multiple Python-based workflows and led to increased flakiness in > pre-commit and post-commit tests. > > Following rapid investigation and fixes across the CI infrastructure, > health levels recovered quickly and remained stable through the final week > of October. No persistent issues were observed after the adjustments were > merged, and the overall trend suggests the system adapted well to the new > Python version with only short-term disruptions. > > Please see the attached chart for October's Health Status trends. > > Key Fixes and Improvements > > During the month, we resolved a wide range of flaky workflows across > multiple areas. A full list of 30 fixes is available here > <https://github.com/apache/beam/issues?q=is%3Aissue%20state%3Aclosed%20label%3Aflaky_test%20closed%3A%3E2025-10-01%20closed%3A%3C2025-10-30%20(involves%3AAmar3tto%20OR%20involves%3AaIbrahiim%20OR%20involves%3AA1K28)> > . > > Currently failing workflows > > - > > Important Signals (1) > - > > PostCommit Python <https://github.com/apache/beam/issues/30513> > - > > PostCommit TransformService Direct > <https://github.com/apache/beam/issues/30960> > - > > Dataflow Java Tests (1) > - > > PostCommit XVR GoUsingJava Dataflow > <https://github.com/apache/beam/issues/30519> > - > > Java Runners Tests (1) > - > > PostCommit Java ValidatesRunner SparkStructuredStreaming > <https://github.com/apache/beam/issues/30516> > > Focus Areas Ahead > > - > > Sustain the current level of health and responsiveness > - > > Continue addressing flaky test issues as they appear > - > > Work toward long-term solutions for persistent infrastructure > challenges > > As always, if you encounter infrastructure-related issues, feel free to > open a GitHub issue with the “infra > <https://github.com/apache/beam/issues?q=is%3Aissue%20state%3Aopen%20label%3Ainfra>” > label, and our team will triage and address it. > > Best regards, > Vitaly Terentyev > Akvelon Inc. > Apache Beam Infrastructure Team > >
