> Could someone create an ‘infra’ component in Jira (I don’t have access)?
Could these issues go under the "testing" component? On Fri, Sep 11, 2020 at 3:11 PM Tyson Hamilton <[email protected]> wrote: > Hi All, > > Below is a summary of the improvements made to Beam’s infrastructure in > the last few months. The completion of this work is thanks to Damian > (damgad) and Tobiasz (TobKed) from Polidea. Thank you!!! > > > - > > Jenkins tests are not always triggered / Switch to new jenkins infra > - > > Beam now has a dedicated Jenkins master. > - > > Beam now has 4x the Jenkins agents which essentially eliminated > build queues. > - > > Jenkins /tmp directory periodic cleanup > - > > Moved to the inventory job for better transparency [tmp-cleanup]. > - > > Jenkins post commit tests does not clearly point to failing tests > - > > installed Test Result History Plugin [test-results-plugin, > screenshot] to start measuring flakiness. This plugin produces a similar > matrix of test results as the Grafana Chart but it is clickable and has > aggregation. > - > > BEAM-3105 Cancel Jenkins builds when an update to a PR makes a prior > build irrelevant > - > > BEAM-3105 Cancel Jenkins builds when an update to a PR makes a prior > build irrelevant > - > > BEAM-9388 Github Actions instead of Travis CI as a build framework > (Wheel Files) > - > > BEAM-9271 Fix "beam_PostCommit_Py_ValCont failure due to timeout" > - > > BEAM-4709 Javadoc build only tested during release > - > > BEAM-8665 Run 2 (one for Java, one for Python) new test suites in > Windows environment. > - > > BEAM-10281 Jenkins Load Sharing Improvements (stackdriver agents & > extend nb of executors) > - > > BEAM-10542 Investigate a possible Nexmark performance regression > around 06/16 > - > > INFRA-20649 Beam Jenkins not triggering jobs from phrases > - > > BEAM-10831 Fix broken Beam Dependency Check Report > - > > BEAM-10837 Remove unused beam_PerformanceTests_Analysis Jenkins Job > - > > BEAM-10835 Improve Github Actions cancelling duplicated runs > - > > BEAM-10807 Performance test weekly email sent to dev@ list > automatically [perf-email] > - > > Apply more strict credentials bindings to Jenkins jobs > - > > Configure seed & dependency check jobs to be triggered only by > committers > > > There are plans to continue to invest into Beam’s infra in the future. A > few examples include, > > - > > Test Flakiness tracking improvements (e.g. increase build history > retention) > - > > Reduce flakiness with cross-job caching (maven, pypi, gradle, build > cache, licenses, …) > - > > More Github Actions (java artifacts, docker images, flaky test > quarantine like Apache Ariflow) > > > If you have additional infra related tasks, or pain points, please create > a Jira issue so we can track them. Contributions are always welcome. Could > someone create an ‘infra’ component in Jira (I don’t have access)? > > > -Tyson > > [tmp-cleanup]: > https://github.com/apache/beam/commit/50366bae1c86136331be097f0f0250bd619dc7f8 > > [test-results-plugin]: https://plugins.jenkins.io/test-results-analyzer/ > > [test-results-screenshot]: https://photos.app.goo.gl/E4cGpp5WqxYhwtTAA > > [perf-email]: > https://lists.apache.org/thread.html/rc0054933a3e40c35f09641786a873391988caf6529899ec8bba67772%40%3Cdev.beam.apache.org%3E > >
