I think the big bucket makes sense and is pretty self-documenting. I think we'll hit some name conflicts between our infra component and ASF INFRA. They just took the best name :-)
I think "testing" is a worse name, really. It is documented as "Testing: General Infrastructure" but someone not reading that could put all sorts of things in the "testing" component. Since you can tag as many components as you want, I think having a bunch of fine-grained components could be fine. On the other hand, I don't want to invent a component for "email report about our dependencies". I'm happy to create an "infra" component but is there a synonym, or a small collection of other terms, we could use? Kenn On Fri, Sep 11, 2020 at 3:39 PM Tyson Hamilton <[email protected]> wrote: > > > On Fri, Sep 11, 2020 at 3:29 PM Kyle Weaver <[email protected]> wrote: > >> > Could someone create an ‘infra’ component in Jira (I don’t have access)? >> >> >> Could these issues go under the "testing" component? >> > > That is an option. There are things beyond testing as well like Github > Actions, Jenkins inventory jobs, email reports, artifact publishing, etc. > so IMO having a general 'infra' would be handy. Another option is to have > dedicated components for each of these things but the proliferation may > become a problem. > > >> >> On Fri, Sep 11, 2020 at 3:11 PM Tyson Hamilton <[email protected]> >> wrote: >> >>> Hi All, >>> >>> Below is a summary of the improvements made to Beam’s infrastructure in >>> the last few months. The completion of this work is thanks to Damian >>> (damgad) and Tobiasz (TobKed) from Polidea. Thank you!!! >>> >>> >>> - >>> >>> Jenkins tests are not always triggered / Switch to new jenkins infra >>> - >>> >>> Beam now has a dedicated Jenkins master. >>> - >>> >>> Beam now has 4x the Jenkins agents which essentially eliminated >>> build queues. >>> - >>> >>> Jenkins /tmp directory periodic cleanup >>> - >>> >>> Moved to the inventory job for better transparency [tmp-cleanup]. >>> - >>> >>> Jenkins post commit tests does not clearly point to failing tests >>> - >>> >>> installed Test Result History Plugin [test-results-plugin, >>> screenshot] to start measuring flakiness. This plugin produces a >>> similar >>> matrix of test results as the Grafana Chart but it is clickable and >>> has >>> aggregation. >>> - >>> >>> BEAM-3105 Cancel Jenkins builds when an update to a PR makes a prior >>> build irrelevant >>> - >>> >>> BEAM-3105 Cancel Jenkins builds when an update to a PR makes a prior >>> build irrelevant >>> - >>> >>> BEAM-9388 Github Actions instead of Travis CI as a build framework >>> (Wheel Files) >>> - >>> >>> BEAM-9271 Fix "beam_PostCommit_Py_ValCont failure due to timeout" >>> - >>> >>> BEAM-4709 Javadoc build only tested during release >>> - >>> >>> BEAM-8665 Run 2 (one for Java, one for Python) new test suites in >>> Windows environment. >>> - >>> >>> BEAM-10281 Jenkins Load Sharing Improvements (stackdriver agents & >>> extend nb of executors) >>> - >>> >>> BEAM-10542 Investigate a possible Nexmark performance regression >>> around 06/16 >>> - >>> >>> INFRA-20649 Beam Jenkins not triggering jobs from phrases >>> - >>> >>> BEAM-10831 Fix broken Beam Dependency Check Report >>> - >>> >>> BEAM-10837 Remove unused beam_PerformanceTests_Analysis Jenkins Job >>> - >>> >>> BEAM-10835 Improve Github Actions cancelling duplicated runs >>> - >>> >>> BEAM-10807 Performance test weekly email sent to dev@ list >>> automatically [perf-email] >>> - >>> >>> Apply more strict credentials bindings to Jenkins jobs >>> - >>> >>> Configure seed & dependency check jobs to be triggered only by >>> committers >>> >>> >>> There are plans to continue to invest into Beam’s infra in the future. A >>> few examples include, >>> >>> - >>> >>> Test Flakiness tracking improvements (e.g. increase build history >>> retention) >>> - >>> >>> Reduce flakiness with cross-job caching (maven, pypi, gradle, build >>> cache, licenses, …) >>> - >>> >>> More Github Actions (java artifacts, docker images, flaky test >>> quarantine like Apache Ariflow) >>> >>> >>> If you have additional infra related tasks, or pain points, please >>> create a Jira issue so we can track them. Contributions are always welcome. >>> Could someone create an ‘infra’ component in Jira (I don’t have access)? >>> >>> >>> -Tyson >>> >>> [tmp-cleanup]: >>> https://github.com/apache/beam/commit/50366bae1c86136331be097f0f0250bd619dc7f8 >>> >>> [test-results-plugin]: https://plugins.jenkins.io/test-results-analyzer/ >>> >>> [test-results-screenshot]: https://photos.app.goo.gl/E4cGpp5WqxYhwtTAA >>> >>> [perf-email]: >>> https://lists.apache.org/thread.html/rc0054933a3e40c35f09641786a873391988caf6529899ec8bba67772%40%3Cdev.beam.apache.org%3E >>> >>>
