Abacn commented on code in PR #22703:
URL: https://github.com/apache/beam/pull/22703#discussion_r951919156
##########
.github/ACTIONS.md:
##########
@@ -17,10 +17,78 @@
under the License.
-->
-> **PLEASE update this file if you add new github action or change
name/trigger phrase of a github action.**
+> **PLEASE update this file if you add new GitHub Action or change
name/trigger phrase of a GitHub Action.**
-## Beam Github Actions
+## About GitHub Actions Runners and Self-hosted Runners
+According to GitHub Docs, we can define a GitHub-hosted runner and a
self-hosted runner as the following:
+* A [GitHub-hosted
runner](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners)
is a new virtual machine (VM) hosted by GitHub with the runner application and
other tools preinstalled, and is available with Ubuntu Linux, Windows, or macOS
operating systems.
+* A [self-hosted
runner](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners)
is a system that you deploy and manage to execute jobs from GitHub Actions on
GitHub.com.
+## Apache Beam GitHub Actions
+
+Currently, we have both GitHub-hosted and self-hosted runners for running the
GitHub Actions workflows, hosted on Google Cloud Platform(GCP) Virtual Machines
and Google Kubernetes Engine(GKE). The majority of our workflows that run in
Ubuntu and Windows run in self-hosted runners, except for those that runs on
MacOS and the `Monitor Self-Hosted Runners Status` workflow that monitors our
GCP self-hosted runners.
+
+### Getting Started with self-hosted runners
+* Refer to [this README](./gh-actions-self-hosted-runners/README.md) for the
steps for creating your own self-hosted runners for testing your workflows.
+* Depending on your workflow's needs, it must specify the following `runs-on`
tags to run in the specified operating system:
+ * Ubuntu 20.04 self-hosted runner: `[self-hosted, ubuntu-20.04]`
+ * Windows Server 2019 self-hosted runner: `[self-hosted,windows-server-2019]`
+ * MacOS GitHub-hosted runner: `macos-latest`
+* Every workflow that tests the source code, needs to have the workflow
trigger `pull_request_target` instead of `pull_request`.
+* The workflow must have set read permissions for all the available scopes and
jobs: `permissions: read-all`. It must be set at the top of the `jobs`
directive.
+* For those workflows that have the `pull_request_target` trigger, in the
checkout step must be added a ref to `${{ github.event.pull_request.head.sha }}`
+``` yaml
+ - name: Checkout code
+ uses: actions/checkout@v#
+ with:
+ ref: ${{ github.event.pull_request.head.sha }}
+```
+* If your workflow runs successfully in a GitHub-hosted runner but not in the
self-hosted runner, it might need a new installation step.
+```yaml
+ - name: Setup Node
+ uses: actions/setup-node@v3
+ with:
+ node-version: 16
+```
+* You can find the GitHub-hosted runner installations in the following links:
+ *
[Ubuntu-20.04](https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md#installed-apt-packages)
+ *
[Windows-2019](https://github.com/actions/runner-images/blob/main/images/win/Windows2019-Readme.md)
+
+#### GitHub Actions Example
+```yaml
+name: GitHub Actions Example
+on:
+ pull_request_target:
+ branches: ['master']
+permissions: read-all
+jobs:
+ github-actions-example:
+ runs-on: [self-hosted, ubuntu-20.04]
+ steps:
+ - name: Check out repository code
+ uses: actions/checkout@v2
+ with:
+ ref: ${{ github.event.pull_request.head.sha }}
+ - run: echo "This job is now running on a ubuntu server hosted by Apache
Beam!"
+ - name: Setup Node
+ uses: actions/setup-node@v3
+ with:
+ node-version: 16
+ - name: Install npm dependencies
+ run: npm ci
+ working-directory: 'scripts/ci/your-path'
+ - name: Run Node.js code
+ run: npm run functionName
+ env:
+ VAR_1: my-var
+ working-directory: 'scripts/ci/your-path'
+```
+
+#### IMPORTANT for Committers
+* A **detailed review** for changes in the workflows is needed due to
important **security concerns**.
+* **DO NOT** Approve and Run changes in the workflows in the PR Conversation
tab, under "Workflow(s) awaiting approval".
Review Comment:
Is there a way to enforce this by some repo configuration or settings?
Pinning Infra committer @damccorm
##########
.github/gh-actions-self-hosted-runners/helper-functions/generateToken/example.env:
##########
@@ -0,0 +1,8 @@
+# Please create a .env file with the corresponding values of the variables
Review Comment:
Can we deduplicate the three same example.env in sub directories, or just
embed the example into documentation?
##########
.github/ACTIONS.md:
##########
@@ -17,10 +17,78 @@
under the License.
-->
-> **PLEASE update this file if you add new github action or change
name/trigger phrase of a github action.**
+> **PLEASE update this file if you add new GitHub Action or change
name/trigger phrase of a GitHub Action.**
-## Beam Github Actions
+## About GitHub Actions Runners and Self-hosted Runners
+According to GitHub Docs, we can define a GitHub-hosted runner and a
self-hosted runner as the following:
+* A [GitHub-hosted
runner](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners)
is a new virtual machine (VM) hosted by GitHub with the runner application and
other tools preinstalled, and is available with Ubuntu Linux, Windows, or macOS
operating systems.
+* A [self-hosted
runner](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners)
is a system that you deploy and manage to execute jobs from GitHub Actions on
GitHub.com.
+## Apache Beam GitHub Actions
+
+Currently, we have both GitHub-hosted and self-hosted runners for running the
GitHub Actions workflows, hosted on Google Cloud Platform(GCP) Virtual Machines
and Google Kubernetes Engine(GKE). The majority of our workflows that run in
Ubuntu and Windows run in self-hosted runners, except for those that runs on
MacOS and the `Monitor Self-Hosted Runners Status` workflow that monitors our
GCP self-hosted runners.
+
+### Getting Started with self-hosted runners
+* Refer to [this README](./gh-actions-self-hosted-runners/README.md) for the
steps for creating your own self-hosted runners for testing your workflows.
+* Depending on your workflow's needs, it must specify the following `runs-on`
tags to run in the specified operating system:
+ * Ubuntu 20.04 self-hosted runner: `[self-hosted, ubuntu-20.04]`
+ * Windows Server 2019 self-hosted runner: `[self-hosted,windows-server-2019]`
+ * MacOS GitHub-hosted runner: `macos-latest`
+* Every workflow that tests the source code, needs to have the workflow
trigger `pull_request_target` instead of `pull_request`.
Review Comment:
The GitHub documentation says "Avoid using this event if you need to build
or run code from the pull request." What makes it necessary to change
`pull_request` to `pull_request_target` trigger in existing workflows and does
it have potential security concerns? I will also do some investigation about
best practices.
##########
.github/workflows/build_wheels.yml:
##########
@@ -199,23 +202,26 @@ jobs:
run: gsutil cp -r -a public-read source/* ${{ env.GCP_PATH }}
build_wheels:
- name: Build python wheels on ${{matrix.arch}} for ${{ matrix.os_python.os
}}
+ name: Build python wheels on ${{matrix.arch}} for ${{
join(matrix.os_python.os, ', ')}}
needs: build_source
env:
CIBW_ARCHS_LINUX: ${{matrix.arch}}
runs-on: ${{ matrix.os_python.os }}
strategy:
matrix:
os_python: [
- {"os": "ubuntu-latest", "python": "cp37-* cp38-* cp39-*"},
- {"os": "macos-latest", "python": "cp37-* cp38-* cp39-*"},
- {"os": "windows-latest", "python": "cp37-* cp38-* cp39-*"},
+ {"os": [self-hosted, ubuntu-20.04], "python": "cp37-* cp38-*
cp39-*"},
+ {"os": [macos-latest], "python": "cp37-* cp38-* cp39-*"},
+ {"os": [self-hosted,windows-server-2019], "python": "cp37-* cp38-*
cp39-*"},
Review Comment:
Is there a reason that there is a white space before 'ubuntu-20.02' but no
white space before 'windows-server-2019' throughout the changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
