I'd love to help, but I am on vacation next week, just one word of warning. If you want to run GitHub Runner on your own infrastructure, that might introduce several security risks.
Basically anyone who makes a PR to your repo can compromise your runners. The dangers of compromising runners are explained here: https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions#potential-impact-of-a-compromised-runner And Github still very very strongly discourages people to use self-hosted runners for Public repositories: https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions#hardening-for-self-hosted-runners Self-hosted runners on GitHub do not have guarantees around running in ephemeral clean virtual machines, and can be persistently compromised by untrusted code in a workflow. As a result, self-hosted runners should almost never be used for public repositories on GitHub, because any user can open pull requests against the repository and compromise the environment. We've workarounded it in Airflow and we run our self-hosted runners on Amazon, however we have a patched version of runner: https://github.com/ashb/runner. We needed to patch it as the original runner cannot reject builds started from "unknown" users. Which is automatically rebased and patch re-applied whenever GitHub releases their new runner (the old versions stop working ~ few days after new version is released). We keep a list of allowed people ("committers") who can utilise such runners. We also maintain list of maintainers in our workflow that are "routed" to self-hosted runners: https://github.com/apache/airflow/blob/1bd3a5c68c88cf3840073d6276460a108f864187/.github/workflows/ci.yml#L86 Those are a bit of hacks (until GitHub implements proper support for it), but if you want to avoid people using your runners to mine bitcoin, or stealing secrets or potentially being able to modify your repository without you knowing it, similar steps should be taken. J. On Thu, Aug 5, 2021 at 7:17 PM Pablo Estrada <[email protected]> wrote: > That works for me! Maybe create a calendar invite and share it in this > thread. > Best > -P. > > On Thu, Aug 5, 2021, 8:36 AM Fernando Morales Martinez < > [email protected]> wrote: > >> Thanks for the help, Pablo! >> I'm also available most of Monday, Tuesday and Wednesday; how about we >> set the meeting for Tuesday August 10th 2pm EST? In case someone interested >> can't make it, we can adjust accordingly. >> Thanks again! >> -Fer >> >> On Wed, Aug 4, 2021 at 5:55 PM Pablo Estrada <[email protected]> wrote: >> >>> Hello Fernando! >>> The people that built the GKE cluster infrastructure have not been >>> involved with Beam for a while. >>> I think you can set a time that is convenient for you, and invite others >>> to participate - and we'll all figure it out together. >>> I'm available most of Monday, Tuesday, Wednesday of next week. I'll be >>> happy to jump on a call and be confused with you (and maybe others will >>> too). >>> Best >>> -P. >>> >>> On Tue, Aug 3, 2021 at 11:20 AM Fernando Morales Martinez < >>> [email protected]> wrote: >>> >>>> Hi everyone, >>>> As part of the work done to migrate the GitHub Actions runners over to >>>> GKE, the, not exhaustive, changes below were performed: >>>> >>>> - added a new secret to the apache-beam-testing project (this is the >>>> PAT needed by the docker image to connect to GitHub) >>>> - added a new docker image (which will execute the test flows still >>>> being ran by GitHub) >>>> >>>> and other changes which you can take a look at here: >>>> https://github.com/apache/beam/pull/15039/files >>>> Among those, the ones of notice are the following: >>>> - .github/workflows/build_wheels.yml >>>> - .github/workflows/cancel.yml >>>> - .github/workflows/java_tests.yml >>>> - .github/workflows/python_tests.yml >>>> - .test-infra/metrics/build.gradle >>>> - .test-infra/metrics/build_and_publish_containers.sh >>>> - .test-infra/metrics/docker-compose.yml >>>> - .test-infra/metrics/kubernetes/beamgrafana-deploy.yaml >>>> - .test-infra/metrics/sync/githubactions/Dockerfile >>>> - .test-infra/metrics/sync/githubactions/entrypoint.sh >>>> >>>> The docker image (.test-infra/metrics/sync/githubactions/Dockerfile) >>>> has been built and pushed to the GCP container registry already. >>>> >>>> Now, I need to deploy the new docker image to GKE. I've done that >>>> before, but in a testing cluster/namespace. I'd definitely feel more >>>> comfortable if someone knowledgeable with the Apache Beam Testing project >>>> GKE architecture watches over my shoulder. >>>> I think I have everything covered, but I could be missing some >>>> important piece. >>>> >>>> Thanks for the help! >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *This email and its contents (including any attachments) are being sent >>>> toyou on the condition of confidentiality and may be protected by >>>> legalprivilege. Access to this email by anyone other than the intended >>>> recipientis unauthorized. If you are not the intended recipient, please >>>> immediatelynotify the sender by replying to this message and delete the >>>> materialimmediately from your system. Any further use, dissemination, >>>> distributionor reproduction of this email is strictly prohibited. Further, >>>> norepresentation is made with respect to any content contained in this >>>> email.* >>> >>> >> >> >> >> >> >> >> >> *This email and its contents (including any attachments) are being sent >> toyou on the condition of confidentiality and may be protected by >> legalprivilege. Access to this email by anyone other than the intended >> recipientis unauthorized. If you are not the intended recipient, please >> immediatelynotify the sender by replying to this message and delete the >> materialimmediately from your system. Any further use, dissemination, >> distributionor reproduction of this email is strictly prohibited. Further, >> norepresentation is made with respect to any content contained in this >> email.* > > -- +48 660 796 129
