Re: [PATCH v2 01/35] scripts/ci: update gitlab-runner playbook to use latest runner

2023-01-24 Thread Richard Henderson

On 1/24/23 08:00, Alex Bennée wrote:

We were using quite and old runner on our machines and running into
issues with stalling jobs. Gitlab in the meantime now reliably provide
the latest packaged versions of the runner under a stable URL. This
update:

   - creates a per-arch subdir for builds
   - switches from binary tarballs to deb packages
   - re-uses the same binary for the secondary runner
   - updates distro check for second to 22.04

Note this script isn't fully idempotent as we end up accumulating
runners especially during testing. However we also want to be able to
run twice with different GitLab keys (e.g. project and personal) so I
think we just have to be mindful of that during testing.

Signed-off-by: Alex Bennée

---
v2
   - only register aarch32 runner, move service start post both registers
   - tested on s390x
---
  scripts/ci/setup/gitlab-runner.yml | 56 +++---
  scripts/ci/setup/vars.yml.template |  2 --
  2 files changed, 13 insertions(+), 45 deletions(-)


Acked-by: Richard Henderson 


r~



[PATCH v2 01/35] scripts/ci: update gitlab-runner playbook to use latest runner

2023-01-24 Thread Alex Bennée
We were using quite and old runner on our machines and running into
issues with stalling jobs. Gitlab in the meantime now reliably provide
the latest packaged versions of the runner under a stable URL. This
update:

  - creates a per-arch subdir for builds
  - switches from binary tarballs to deb packages
  - re-uses the same binary for the secondary runner
  - updates distro check for second to 22.04

Note this script isn't fully idempotent as we end up accumulating
runners especially during testing. However we also want to be able to
run twice with different GitLab keys (e.g. project and personal) so I
think we just have to be mindful of that during testing.

Signed-off-by: Alex Bennée 

---
v2
  - only register aarch32 runner, move service start post both registers
  - tested on s390x
---
 scripts/ci/setup/gitlab-runner.yml | 56 +++---
 scripts/ci/setup/vars.yml.template |  2 --
 2 files changed, 13 insertions(+), 45 deletions(-)

diff --git a/scripts/ci/setup/gitlab-runner.yml 
b/scripts/ci/setup/gitlab-runner.yml
index 33128be85d..95d4199c03 100644
--- a/scripts/ci/setup/gitlab-runner.yml
+++ b/scripts/ci/setup/gitlab-runner.yml
@@ -50,60 +50,30 @@
 
 - name: Download the matching gitlab-runner
   get_url:
-dest: /usr/local/bin/gitlab-runner
-url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ 
gitlab_runner_version  }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ 
gitlab_runner_arch }}"
-owner: gitlab-runner
-group: gitlab-runner
-mode: u=rwx,g=rwx,o=rx
-
-- name: Register the gitlab-runner
-  command: "/usr/local/bin/gitlab-runner register --non-interactive --url 
{{ gitlab_runner_server_url }} --registration-token {{ 
gitlab_runner_registration_token }} --executor shell --tag-list {{ 
ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower 
}}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ 
ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] 
}} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'"
-
-- name: Install the gitlab-runner service using its own functionality
-  command: /usr/local/bin/gitlab-runner install --user gitlab-runner 
--working-directory /home/gitlab-runner
-  register: gitlab_runner_install_service_result
-  failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already 
exists\" not in gitlab_runner_install_service_result.stderr"
+dest: "/root/"
+url: 
"https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_{{ 
gitlab_runner_arch }}.deb"
 
-- name: Enable the gitlab-runner service
-  service:
-name: gitlab-runner
-state: started
-enabled: yes
+- name: Install gitlab-runner via package manager
+  apt: deb="/root/gitlab-runner_{{ gitlab_runner_arch }}.deb"
 
-- name: Download secondary gitlab-runner
-  get_url:
-dest: /usr/local/bin/gitlab-runner-arm
-url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ 
gitlab_runner_version  }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-arm"
-owner: gitlab-runner
-group: gitlab-runner
-mode: u=rwx,g=rwx,o=rx
-  when:
-- ansible_facts['distribution'] == 'Ubuntu'
-- ansible_facts['architecture'] == 'aarch64'
-- ansible_facts['distribution_version'] == '20.04'
+- name: Register the gitlab-runner
+  command: "/usr/bin/gitlab-runner register --non-interactive --url {{ 
gitlab_runner_server_url }} --registration-token {{ 
gitlab_runner_registration_token }} --executor shell --tag-list {{ 
ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower 
}}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ 
ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] 
}} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'"
 
+# The secondary runner will still run under the single gitlab-runner 
service
 - name: Register secondary gitlab-runner
-  command: "/usr/local/bin/gitlab-runner-arm register --non-interactive 
--url {{ gitlab_runner_server_url }} --registration-token {{ 
gitlab_runner_registration_token }} --executor shell --tag-list aarch32,{{ 
ansible_facts[\"distribution\"]|lower }}_{{ 
ansible_facts[\"distribution_version\"] }} --description '{{ 
ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] 
}} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'"
+  command: "/usr/bin/gitlab-runner register --non-interactive --url {{ 
gitlab_runner_server_url }} --registration-token {{ 
gitlab_runner_registration_token }} --executor shell --tag-list aarch32,{{ 
ansible_facts[\"distribution\"]|lower }}_{{ 
ansible_facts[\"distribution_version\"] }} --description '{{ 
ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] 
}} {{