There exists a few that could be used and seem adequate to cover infra, it may have been that I just didn't know about them all. Looking through the component list now in Jira some that make sense:
* build-system: Build, CI, release systems and processes (519 issues) * testing: Testing: general infrastructure (770 issues) These don't seem relevant after a further glance: * jobserver: (no description, only 3 issues) * community-metrics: Beam's community metrics dashboard (13 issues) Let's (continue to) use build-system and testing components. On Tue, Sep 15, 2020 at 9:58 AM Ahmet Altay <[email protected]> wrote: > Thank you Tyson for the update. Thank you Damian, Tobiasz for the > improvements. > > On Mon, Sep 14, 2020 at 3:29 PM Kenneth Knowles <[email protected]> wrote: > >> I think the big bucket makes sense and is pretty self-documenting. I >> think we'll hit some name conflicts between our infra component and ASF >> INFRA. They just took the best name :-) >> >> I think "testing" is a worse name, really. It is documented as "Testing: >> General Infrastructure" but someone not reading that could put all sorts of >> things in the "testing" component. >> >> Since you can tag as many components as you want, I think having a bunch >> of fine-grained components could be fine. On the other hand, I don't want >> to invent a component for "email report about our dependencies". >> >> I'm happy to create an "infra" component but is there a synonym, or a >> small collection of other terms, we could use? >> >> Kenn >> >> On Fri, Sep 11, 2020 at 3:39 PM Tyson Hamilton <[email protected]> >> wrote: >> >>> >>> >>> On Fri, Sep 11, 2020 at 3:29 PM Kyle Weaver <[email protected]> wrote: >>> >>>> > Could someone create an ‘infra’ component in Jira (I don’t have >>>> access)? >>>> >>>> >>>> Could these issues go under the "testing" component? >>>> >>> >>> That is an option. There are things beyond testing as well like Github >>> Actions, Jenkins inventory jobs, email reports, artifact publishing, etc. >>> so IMO having a general 'infra' would be handy. Another option is to have >>> dedicated components for each of these things but the proliferation may >>> become a problem. >>> >>> >>>> >>>> On Fri, Sep 11, 2020 at 3:11 PM Tyson Hamilton <[email protected]> >>>> wrote: >>>> >>>>> Hi All, >>>>> >>>>> Below is a summary of the improvements made to Beam’s infrastructure >>>>> in the last few months. The completion of this work is thanks to Damian >>>>> (damgad) and Tobiasz (TobKed) from Polidea. Thank you!!! >>>>> >>>>> >>>>> - >>>>> >>>>> Jenkins tests are not always triggered / Switch to new jenkins >>>>> infra >>>>> - >>>>> >>>>> Beam now has a dedicated Jenkins master. >>>>> - >>>>> >>>>> Beam now has 4x the Jenkins agents which essentially eliminated >>>>> build queues. >>>>> - >>>>> >>>>> Jenkins /tmp directory periodic cleanup >>>>> - >>>>> >>>>> Moved to the inventory job for better transparency >>>>> [tmp-cleanup]. >>>>> - >>>>> >>>>> Jenkins post commit tests does not clearly point to failing tests >>>>> - >>>>> >>>>> installed Test Result History Plugin [test-results-plugin, >>>>> screenshot] to start measuring flakiness. This plugin produces a >>>>> similar >>>>> matrix of test results as the Grafana Chart but it is clickable and >>>>> has >>>>> aggregation. >>>>> - >>>>> >>>>> BEAM-3105 Cancel Jenkins builds when an update to a PR makes a >>>>> prior build irrelevant >>>>> - >>>>> >>>>> BEAM-3105 Cancel Jenkins builds when an update to a PR makes a >>>>> prior build irrelevant >>>>> - >>>>> >>>>> BEAM-9388 Github Actions instead of Travis CI as a build framework >>>>> (Wheel Files) >>>>> - >>>>> >>>>> BEAM-9271 Fix "beam_PostCommit_Py_ValCont failure due to timeout" >>>>> - >>>>> >>>>> BEAM-4709 Javadoc build only tested during release >>>>> - >>>>> >>>>> BEAM-8665 Run 2 (one for Java, one for Python) new test suites in >>>>> Windows environment. >>>>> - >>>>> >>>>> BEAM-10281 Jenkins Load Sharing Improvements (stackdriver agents & >>>>> extend nb of executors) >>>>> - >>>>> >>>>> BEAM-10542 Investigate a possible Nexmark performance regression >>>>> around 06/16 >>>>> - >>>>> >>>>> INFRA-20649 Beam Jenkins not triggering jobs from phrases >>>>> - >>>>> >>>>> BEAM-10831 Fix broken Beam Dependency Check Report >>>>> - >>>>> >>>>> BEAM-10837 Remove unused beam_PerformanceTests_Analysis Jenkins Job >>>>> - >>>>> >>>>> BEAM-10835 Improve Github Actions cancelling duplicated runs >>>>> - >>>>> >>>>> BEAM-10807 Performance test weekly email sent to dev@ list >>>>> automatically [perf-email] >>>>> - >>>>> >>>>> Apply more strict credentials bindings to Jenkins jobs >>>>> - >>>>> >>>>> Configure seed & dependency check jobs to be triggered only by >>>>> committers >>>>> >>>>> >>>>> There are plans to continue to invest into Beam’s infra in the future. >>>>> A few examples include, >>>>> >>>>> - >>>>> >>>>> Test Flakiness tracking improvements (e.g. increase build history >>>>> retention) >>>>> - >>>>> >>>>> Reduce flakiness with cross-job caching (maven, pypi, gradle, >>>>> build cache, licenses, …) >>>>> - >>>>> >>>>> More Github Actions (java artifacts, docker images, flaky test >>>>> quarantine like Apache Ariflow) >>>>> >>>>> >>>>> If you have additional infra related tasks, or pain points, please >>>>> create a Jira issue so we can track them. Contributions are always >>>>> welcome. >>>>> >>>> > +1 to this request. Identifying the largest productivity blockers to the > contributors would greatly help with the future of this work track. Please > report any infra related frictions to your own workflows. > > >> Could someone create an ‘infra’ component in Jira (I don’t have access)? >>>>> >>>>> >>>>> -Tyson >>>>> >>>>> [tmp-cleanup]: >>>>> https://github.com/apache/beam/commit/50366bae1c86136331be097f0f0250bd619dc7f8 >>>>> >>>>> [test-results-plugin]: >>>>> https://plugins.jenkins.io/test-results-analyzer/ >>>>> >>>>> [test-results-screenshot]: https://photos.app.goo.gl/E4cGpp5WqxYhwtTAA >>>>> >>>>> [perf-email]: >>>>> https://lists.apache.org/thread.html/rc0054933a3e40c35f09641786a873391988caf6529899ec8bba67772%40%3Cdev.beam.apache.org%3E >>>>> >>>>>
