TobKed commented on a change in pull request #12405: URL: https://github.com/apache/beam/pull/12405#discussion_r462953052
########## File path: CI.md ########## @@ -0,0 +1,104 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Apache Beam + +## CI Environment + +Continuous Integration is important component of making Apache Beam robust and stable. + +Our execution environment for CI is mainly the Jenkins which is available at +[https://ci-beam.apache.org/](https://ci-beam.apache.org/). See +[.test-infra/jenkins/README](.test-infra/jenkins/README.md) +for trigger phrase, status and link of all Jenkins jobs. See Apache Beam Developer Guide for +[Jenkins Tips](https://cwiki.apache.org/confluence/display/BEAM/Jenkins+Tips). + +An additional execution environment for CI is [GitHub Actions](https://github.com/features/actions). GitHub Actions +(GA) are very well integrated with GitHub code and Workflow and it has evolved fast in 2019/202 to become +a fully-fledged CI environment, easy to use and develop for, so we decided to use it for building python source +distribution and wheels. + +## GitHub Actions + +### GitHub actions run types + +The following GA CI Job runs are currently run for Apache Beam, and each of the runs have different +purpose and context. + +#### Pull request run + +Those runs are results of PR from the forks made by contributors. Most builds for Apache Beam fall +into this category. They are executed in the context of the "Fork", not main +Beam Code Repository which means that they have only "read" permission to all the GitHub resources +(container registry, code repository). This is necessary as the code in those PRs (including CI job +definition) might be modified by people who are not committers for the Apache Beam Code Repository. + +The main purpose of those jobs is to check if PR builds cleanly, if the test run properly and if +the PR is ready to review and merge. + +#### Direct Push/Merge Run + +Those runs are results of direct pushes done by the committers or as result of merge of a Pull Request +by the committers. Those runs execute in the context of the Apache Beam Code Repository and have also +write permission for GitHub resources (container registry, code repository). +The main purpose for the run is to check if the code after merge still holds all the assertions - like +whether it still builds, all tests are green. + +This is needed because some of the conflicting changes from multiple PRs might cause build and test failures +after merge even if they do not fail in isolation. + +#### Scheduled runs + +Those runs are results of (nightly) triggered job - only for `master` branch. The +main purpose of the job is to check if there was no impact of external dependency changes on the Apache +Beam code (for example transitive dependencies released that fail the build). Another reason for the nightly +build is that the builds tags most recent master with `nightly-master`. + +All runs consist of the same jobs, but the jobs behave slightly differently or they are skipped in different +run categories. Here is a summary of the run categories with regards of the jobs they are running. +Those jobs often have matrix run strategy which runs several different variations of the jobs +(with different platform type / Python version to run for example) + +| Job | Description | Pull Request Run | Direct Push/Merge Run | Scheduled Run | Requires GCP Credentials | Review comment: As I mentioned in comment above, I will add them in PR which introduce them to avoid discrepancy. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
