Melissa Jost has uploaded this change for review. (
https://gem5-review.googlesource.com/c/public/gem5/+/71098?usp=email )
Change subject: util: Add vagrant folder for GitHub runner configuration
......................................................................
util: Add vagrant folder for GitHub runner configuration
This adds files that can be used to configure Vagrant machines
that will be used to test running gem5 alongside Github Actions.
Change-Id: I52b0f39b6e6044c22481f02163d5fc01eab76788
---
A util/vagrant/README.md
A util/vagrant/Vagrantfile
A util/vagrant/provision_nonroot.sh
A util/vagrant/provision_root.sh
4 files changed, 218 insertions(+), 0 deletions(-)
diff --git a/util/vagrant/README.md b/util/vagrant/README.md
new file mode 100644
index 0000000..8a7cdd5
--- /dev/null
+++ b/util/vagrant/README.md
@@ -0,0 +1,29 @@
+# How to set up Vagrant VM for Github Actions
+
+First, generate a Personal Access Token, which you can create
[here](https://github.com/settings/tokens)
+Make sure to set admin permissions on this token, then paste it onto line
35 of the Vagrantfile.
+
+After this, install the relevant vagrant plugins:
+
+``` bash
+vagrant plugin install dotenv
+vagrant plugin install vagrant-libvirt
+```
+
+Lastly, make sure that the configuration for the Vagrantfile is set
properly.
+For a "runner" machine, we set the number of cpus to 1, the memory to 8GB,
and the tag in the echo command to be run.
+For a "builder" machine, we set the number of cpus to 4, the memory to
16GB, and the tag in the echo command to be build.
+You can expect each machine to take up approximatley 60GB of disk space.
+
+Once everything is set properly, you can start up the vagrant machine with
the command
+
+``` bash
+export VAGRANT_HOME=/scr/vagrant
+vagrant up --provider=libvirt
+```
+
+This should automatically create your machine, as well as configure and
start up a Github Actions runner. You can check the status of the runner
here: https://github.com/<owner>/<repo>/settings/actions/runners
+
+If the runner ever shows as offline, you can rerun the `vagrant up
--provider=libvirt` command to make sure everything is working properly.
+
+This setup follow steps from
[here](https://github.com/peaceiris/actions-self-hosted-runners)
diff --git a/util/vagrant/Vagrantfile b/util/vagrant/Vagrantfile
new file mode 100644
index 0000000..45056b6
--- /dev/null
+++ b/util/vagrant/Vagrantfile
@@ -0,0 +1,49 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+Vagrant.configure("2") do |config|
+ config.vm.box = "generic/ubuntu2204"
+ config.vm.box_check_update = true
+ config.vm.define "githubrunnermachine"
+ config.vm.hostname = "githubrunnerhost"
+ # allows us to ssh into the machine, addressing the problem below
+ #
https://www.reddit.com/r/vagrant/comments/sb7hfl/new_to_vagrant_getting_efault_warning/
+ config.ssh.username = "vagrant"
+ config.ssh.password = "vagrant"
+
+ config.vm.provider "libvirt" do |vb|
+ # Customize the amount of cpus and memory on the VM:
+ vb.cpus = "1".to_i
+ vb.memory = "8192".to_i
+ end
+
+ # sets up vm
+ config.vm.provision :root_user, type: "shell", path: "provision_root.sh"
+ config.vm.provision :vagrant_user, type: "shell", privileged: false,
path: "provision_nonroot.sh"
+
+ config.vm.provision :vagrant_user_runner, type: "shell", privileged:
false, inline: <<-SHELL
+ # Install actions/runner
+ mkdir ~/actions-runner && cd ~/actions-runner
+ curl -so actions-runner-linux-x64-2.304.0.tar.gz -L \
+
https://github.com/actions/runner/releases/download/v2.304.0/actions-runner-linux-x64-2.304.0.tar.gz
+ tar xzf ./actions-runner-linux-x64-2.304.0.tar.gz
+
+ # configure the runner
+ # echo automatically sets the name of the runner, and the tags
+ # create a personal access token with admin permission and copy it
into the curl command
+ # TODO: currently automatically references mkjost0 repository
+ echo -ne '\n\nrun\n\n' | ./config.sh --url
https://github.com/mkjost0/gem5-actions --token $(curl -L \
+ -X POST -H "Accept: application/vnd.github+json"
-H "Authorization: Bearer <PERSONAL ACCESS TOKEN>" \
+ -H "X-GitHub-Api-Version: 2022-11-28" \
+
https://api.github.com/repos/mkjost0/gem5-actions/actions/runners/registration-token
| jq -r '.token')
+ # start the runner
+ nohup ./run.sh &
+ SHELL
+
+ # always runs, so we can start up the runner again on vagrant up if it
dies at any point
+ # TODO: needs to be commented out on first vagrant up or else this runs
before previous shell script
+ config.vm.provision :vagrant_user_runner, type: "shell", privileged:
false, run: 'always', inline: <<-SHELL
+ cd actions-runner
+ nohup ./run.sh &
+ SHELL
+end
diff --git a/util/vagrant/provision_nonroot.sh
b/util/vagrant/provision_nonroot.sh
new file mode 100644
index 0000000..4465b5a
--- /dev/null
+++ b/util/vagrant/provision_nonroot.sh
@@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+
+# fail on unset variables and command errors
+set -eu -o pipefail # -x: is for debugging
+
+# Install deno
+curl -fsSL https://deno.land/x/install/install.sh | sh
+echo "export PATH=\"\${HOME}/.deno/bin:\${PATH}\"" >> ~/.profile
+echo "export PATH=\"\${HOME}/.deno/bin:\${PATH}\"" >> ~/.bash_profile
+
+# Install docker compose
+DOCKER_COMPOSE_VERSION=$(curl -s
https://api.github.com/repos/docker/compose/releases/latest | jq
-r '.tag_name')
+mkdir -p "${HOME}/.docker/cli-plugins"
+curl
-sL "https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-$(uname
-s)-$(uname -m)" -o "${HOME}/.docker/cli-plugins/docker-compose"
+chmod +x "${HOME}/.docker/cli-plugins/docker-compose"
diff --git a/util/vagrant/provision_root.sh b/util/vagrant/provision_root.sh
new file mode 100644
index 0000000..8d0dbd7
--- /dev/null
+++ b/util/vagrant/provision_root.sh
@@ -0,0 +1,125 @@
+#!/usr/bin/env bash
+
+# fail on unset variables and command errors
+set -eu -o pipefail # -x: is for debugging
+
+apt-get update
+apt-get upgrade -y
+apt-get install -y software-properties-common
+add-apt-repository --yes --update ppa:git-core/ppa
+apt-get install -y \
+ bash \
+ build-essential \
+ clang-format \
+ git \
+ git-lfs \
+ jq \
+ libffi-dev \
+ libssl-dev \
+ nkf \
+ python3 \
+ python3-dev \
+ python3-pip \
+ python3-venv \
+ shellcheck \
+ tree \
+ wget \
+ yamllint \
+ zstd
+snap install jq
+
+# Install gh
+curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg |
sudo gpg --dearmor -o /usr/share/keyrings/githubcli-archive-keyring.gpg
+echo "deb [arch=$(dpkg --print-architecture)
signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg]
https://cli.github.com/packages stable main" | sudo tee
/etc/apt/sources.list.d/github-cli.list > /dev/null
+apt-get update -y
+apt-get install -y gh
+
+# Install docker
+apt-get install -y apt-transport-https ca-certificates curl gnupg
lsb-release
+curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o
/usr/share/keyrings/docker-archive-keyring.gpg
+echo "deb [arch=amd64
signed-by=/usr/share/keyrings/docker-archive-keyring.gpg]
https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | tee
/etc/apt/sources.list.d/docker.list > /dev/null
+apt-get update -y
+apt-get install -y docker-ce docker-ce-cli containerd.io
+groupadd docker || true
+gpasswd -a vagrant docker
+newgrp docker
+systemctl restart docker
+
+# Install google-cloud-sdk (gcloud)
+sudo apt-get install -y unzip xvfb libxi6 libgconf-2-4 default-jdk
+echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg]
https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a
/etc/apt/sources.list.d/google-cloud-sdk.list
+curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo
apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -
+sudo apt-get update -y
+sudo apt-get install -y google-cloud-sdk
+
+# Install Google Chrome
+sudo curl -sS -o - https://dl-ssl.google.com/linux/linux_signing_key.pub |
apt-key add
+echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable
main" | sudo tee -a /etc/apt/sources.list.d/google-chrome.list
+sudo apt-get update -y
+sudo apt-get install -y google-chrome-stable
+
+# Install ChromeDriver
+CHROMEDRIVER_TEMPDIR=$(mktemp -d)
+wget -O "${CHROMEDRIVER_TEMPDIR}/LATEST_RELEASE"
http://chromedriver.storage.googleapis.com/LATEST_RELEASE
+CHROMEDRIVER_LATEST_VERSION=$(cat "${CHROMEDRIVER_TEMPDIR}/LATEST_RELEASE")
+wget
-O "${CHROMEDRIVER_TEMPDIR}/chromedriver.zip" "http://chromedriver.storage.googleapis.com/${CHROMEDRIVER_LATEST_VERSION}/chromedriver_linux64.zip";
+sudo unzip "${CHROMEDRIVER_TEMPDIR}/chromedriver.zip" chromedriver -d
/usr/local/bin/
+
+# Install terraform
+TERRAFORM_VERSION=$(curl -s
https://checkpoint-api.hashicorp.com/v1/check/terraform | jq
-r .current_version)
+curl
-sLO "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip";
+unzip -qq "terraform_${TERRAFORM_VERSION}_linux_amd64.zip" -d
/usr/local/bin
+rm -f "terraform_${TERRAFORM_VERSION}_linux_amd64.zip"
+
+# Install hadolint
+HADOLINT_VERSION=$(curl
-fsi "https://github.com/hadolint/hadolint/releases/latest"; | awk -F/ '/^(L|
l)ocation:/ {print $(NF)}' | tr -d '\r')
+wget
"https://github.com/hadolint/hadolint/releases/download/${HADOLINT_VERSION}/hadolint-Linux-x86_64";
+mv hadolint-Linux-x86_64 /usr/local/bin/hadolint
+chmod 755 /usr/local/bin/hadolint
+
+# Install node, npm, and yarn
+curl -fsSL https://deb.nodesource.com/setup_16.x | bash -
+apt-get install -y nodejs
+npm install -g npm
+curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | sudo
tee /usr/share/keyrings/yarnkey.gpg >/dev/null
+echo "deb [signed-by=/usr/share/keyrings/yarnkey.gpg]
https://dl.yarnpkg.com/debian stable main" | sudo tee
/etc/apt/sources.list.d/yarn.list
+apt-get update && apt-get install -y yarn
+
+# npm install -g
+npm install -g vercel netlify-cli
+
+# ansible
+apt-add-repository --yes --update ppa:ansible/ansible
+apt-get install -y ansible ansible-lint
+
+# trivy
+apt-get install -y wget apt-transport-https gnupg lsb-release
+wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo
apt-key add -
+echo "deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc)
main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
+apt-get update
+apt-get install -y trivy
+
+# golang
+add-apt-repository --yes --update ppa:longsleep/golang-backports
+apt-get install -y golang
+export GOPATH="${HOME}/go"
+export PATH="${GOPATH}/bin:${PATH}"
+echo "export GOPATH=${HOME}/go" >> "${HOME}/.bashrc"
+echo "export PATH=${GOPATH}/bin:${PATH}" >> "${HOME}/.bashrc"
+
+# golangci-lint
+curl -sSfL
https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh
| sh -s -- -b "$(go env GOPATH)/bin" v1.40.1
+
+# goreleaser
+echo 'deb [trusted=yes] https://repo.goreleaser.com/apt/ /' | sudo tee
/etc/apt/sources.list.d/goreleaser.list
+apt-get update
+apt-get install -y goreleaser
+
+# mage
+go install github.com/magefile/mage@latest
+
+# poetry
+curl -sSL https://install.python-poetry.org | python3 -
+
+# Cleanup
+apt-get autoremove -y
--
To view, visit
https://gem5-review.googlesource.com/c/public/gem5/+/71098?usp=email
To unsubscribe, or for help writing mail filters, visit
https://gem5-review.googlesource.com/settings?usp=email
Gerrit-MessageType: newchange
Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I52b0f39b6e6044c22481f02163d5fc01eab76788
Gerrit-Change-Number: 71098
Gerrit-PatchSet: 1
Gerrit-Owner: Melissa Jost <melissakj...@gmail.com>
_______________________________________________
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
