dannymartinm commented on code in PR #22703: URL: https://github.com/apache/beam/pull/22703#discussion_r951800794
########## .github/gh-actions-self-hosted-runners/README.md: ########## @@ -0,0 +1,106 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> +# GitHub Actions - Self-hosted Runners +The current GitHub Actions workflows are being tested on multiple operating systems, such as Ubuntu, Windows and MacOS. The way to migrate these runners from GitHub to GCP is by implementing self-hosted runners, so we implemented them in both Ubuntu and Windows environments, going with Google Kubernetes Engine and Google Cloud Compute VMs instances respectively. + +On the other hand, we will rely on GitHub-hosted runners for MacOS builds until a straightforward implementation approach comes out. + +## Ubuntu +Ubuntu Self-hosted runners are stored in Artifact Registry and implemented using Google Kubernetes Engine with the following specifications: + +#### Cluster +* Cluster: [gh-actions-linux-runners](https://console.cloud.google.com/kubernetes/clusters/details/us-central1-a/gh-actions-linux-runners/details?project=apache-beam-testing) +* Image: [linux-github-actions-runner](https://console.cloud.google.com/artifacts/docker/apache-beam-testing/us-central1/beam-github-actions/linux-github-actions-runner?project=apache-beam-testing) + +#### Pool +* Number of nodes: 5 +* Cluster Autoscaler: ON + * Minimum number of nodes: 5 + * Maximum number of nodes: 10 + +#### Node +* Machine Type: e2-custom-6-18432 +* Disk Size: 100 GB +* CPU: 6 vCPUs +* Memory : 18 GB + +#### Pod +* Image: $LOCAL_IMAGE_NAME LOCATION-docker.pkg.dev/PROJECT-ID/REPOSITORY/IMAGE:latest +* CPU: 2 +* Memory: 1028 Mi +* Volumes + * docker-sock + * gcloud-key +* Secret env variables + * Kubernetes Secrets + * github-actions-secrets + * gcloud-key + +#### AutoScaling +* Horizontal Pod Autoscaling + * 5-10 nodes (From Pool Cluster Autoscaler) + * HorizontalPodAutoscaler + * Min replicas: 10 + * Max replicas: 20 + * CPU utilization: 70% +* Vertical Pod Autoscaling + * updateMode: "Auto" + + +## Windows +Windows Virtual machines have the following specifications + +#### VM specifications +* Instance Template: _TODO: Add GCP link when final resources are in place._ +* Machine Type: n2-standard-2 +* Disk Size: 70 GB +* Disk Image: [disk-image-windows-runner](https://console.cloud.google.com/compute/imagesDetail/projects/apache-beam-testing/global/images/disk-image-windows-runner?project=apache-beam-testing) +* CPU: 2 vCPUs +* Memory : 8 GB + +#### Instance group settings +* Region: us-west1 (multizone) +* Scale-out metric: 70% of CPU Usage. +* Cooldown period: 300s + +#### Notes: +At first glance we considered implementing Windows runners using K8s, however this was not optimal because of the following reasons: + +* VS Build tools are required for certain workflows, unfortunately official images that support this dependency are huge in size, reaching 20GB easily which is not an ideal case for k8S management. +* Windows Subsystem For Linux(WSL) is a feature that allows to execute bash scripts inside Windows which removes tech debt by avoiding writing steps in powershell, but this feature is disabled with payload removed in Windows containers. Review Comment: Yes, it includes the win32 tests as well. You can find the examples in [Python tests](https://github.com/sre-apache-beam-dev/beam/runs/7957326225?check_suite_focus=true ) workflow in the `Run tests basic windows` step. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
