[M] Change in osmo-ci[master]: OBS: add jenkins job to check for new distros

2023-12-05 Thread osmith 
osmith has submitted this change. ( 
https://gerrit.osmocom.org/c/osmo-ci/+/35223?usp=email )

Change subject: OBS: add jenkins job to check for new distros
..

OBS: add jenkins job to check for new distros

Add a new jenkins job to notify us about new versions of Debian,
Raspbian and Ubuntu becoming available in the openSUSE OBS. This allows
us to consistently follow along and make the repositories available in
the Osmocom OBS. I've decided to check the openSUSE OBS instead of
checking somewhere upstream, because it takes time between a new
release of a distribution and the version becoming available in
openSUSE OBS, and we have another job that syncs the configuration with
that OBS instance.

Related: OS#6163
Change-Id: I0abc49a95197da55f7d83ed4fd1c4ebb6bd14b1e
---
M jobs/README.adoc
A jobs/osmocom-obs-check-new-distros.yml
A scripts/obs/check_new_distros.py
M scripts/obs/lib/config.py
4 files changed, 181 insertions(+), 0 deletions(-)

Approvals:
  laforge: Looks good to me, approved
  Jenkins Builder: Verified
  pespin: Looks good to me, but someone else must approve




diff --git a/jobs/README.adoc b/jobs/README.adoc
index 86fc485..c37b3bb 100644
--- a/jobs/README.adoc
+++ b/jobs/README.adoc
@@ -123,6 +123,7 @@

  18:00 - 21:00 OBS related
18:XX osmocom-obs-sync (sync Debian:12 etc. with openSUSE OBS)
+   19:XX osmocom-obs-check-new-distros
19:XX osmocom-obs-wireshark
20:XX osmocom-obs (new binary packages start building on OBS!)

diff --git a/jobs/osmocom-obs-check-new-distros.yml 
b/jobs/osmocom-obs-check-new-distros.yml
new file mode 100644
index 000..760479a
--- /dev/null
+++ b/jobs/osmocom-obs-check-new-distros.yml
@@ -0,0 +1,39 @@
+---
+- job:
+name: 'Osmocom_OBS_check_new_distros'
+project-type: freestyle
+concurrent: false
+defaults: global
+description: |
+  Check whether new distributions (Debian, Raspyberry Pi OS, Ubuntu) are
+  available in openSUSE's OBS that we need to add to the Osmocom OBS.
+properties:
+  - build-discarder:
+  days-to-keep: 30
+  num-to-keep: 30
+node: obs
+parameters:
+  - string:
+  name: CI_BRANCH
+  description: osmo-ci.git branch
+  default: 'master'
+scm:
+  - git:
+  branches:
+- '$CI_BRANCH'
+  url: https://gerrit.osmocom.org/osmo-ci
+  git-config-name: 'Jenkins Builder'
+  git-config-email: 'jenk...@osmocom.org'
+builders:
+  - shell: |
+  export PYTHONUNBUFFERED=1
+  ./scripts/obs/check_new_distros.py --docker
+triggers:
+  - timed: "H 19 * * *"
+wrappers:
+- ansicolor:
+colormap: xterm
+publishers:
+  - email:
+  notify-every-unstable-build: true
+  recipients: 'jenkins-notificati...@lists.osmocom.org'
diff --git a/scripts/obs/check_new_distros.py b/scripts/obs/check_new_distros.py
new file mode 100755
index 000..962d547
--- /dev/null
+++ b/scripts/obs/check_new_distros.py
@@ -0,0 +1,110 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright 2023 sysmocom - s.f.m.c. GmbH 
+import argparse
+import lib.docker
+import lib.osc
+import sys
+
+projects_opensuse = None
+projects_osmocom = None
+
+
+def parse_args():
+parser = argparse.ArgumentParser(description="Check for new distribution"
+" projects on the openSUSE OBS, that we want to configure in the"
+" Osmocom OBS as soon as they are available")
+parser.add_argument("-d", "--docker",
+help="run in docker to avoid installing required pkgs",
+action="store_true")
+parser.add_argument("-v", "--verbose", action="store_true",
+help="always print shell commands and their output,"
+ " instead of only printing them on error")
+
+advanced = parser.add_argument_group("advanced options")
+advanced.add_argument("-A", "--apiurl", help="source OBS API URL"
+  " (default: https://api.opensuse.org)",
+  default="https://api.opensuse.org;)
+advanced.add_argument("-p", "--prefix", default="openSUSE.org-mirror",
+  help="destination OBS prefix"
+   " (default: openSUSE.org-mirror)")
+advanced.add_argument("-t", "--to-apiurl", help="destination OBS API URL"
+  " (default: https://obs.osmocom.org)",
+  default="https://obs.osmocom.org;)
+
+args = parser.parse_args()
+lib.set_args(args)
+
+lib.osc.check_oscrc()
+
+if args.docker:
+lib.docker.run_in_docker_and_exit("check_new_distros.py", 
add_oscrc=True)
+
+
+def find_highest_distro_project(distro):
+highest = None
+for project in projects_opensuse:
+if not project.startswith(f"{distro}:"):
+continue
+
+num =

[M] Change in osmo-ci[master]: OBS: add jenkins job to check for new distros

2023-12-05 Thread laforge 
Attention is currently required from: osmith, pespin.

laforge has posted comments on this change. ( 
https://gerrit.osmocom.org/c/osmo-ci/+/35223?usp=email )

Change subject: OBS: add jenkins job to check for new distros
..


Patch Set 1: Code-Review+2


--
To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/35223?usp=email
To unsubscribe, or for help writing mail filters, visit 
https://gerrit.osmocom.org/settings

Gerrit-Project: osmo-ci
Gerrit-Branch: master
Gerrit-Change-Id: I0abc49a95197da55f7d83ed4fd1c4ebb6bd14b1e
Gerrit-Change-Number: 35223
Gerrit-PatchSet: 1
Gerrit-Owner: osmith 
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: laforge 
Gerrit-Reviewer: pespin 
Gerrit-Attention: osmith 
Gerrit-Attention: pespin 
Gerrit-Comment-Date: Tue, 05 Dec 2023 17:33:47 +
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

[M] Change in osmo-ci[master]: OBS: add jenkins job to check for new distros

2023-12-05 Thread osmith 
Attention is currently required from: pespin.

osmith has posted comments on this change. ( 
https://gerrit.osmocom.org/c/osmo-ci/+/35223?usp=email )

Change subject: OBS: add jenkins job to check for new distros
..


Patch Set 1:

(1 comment)

Patchset:

PS1:
> I wonder whether we really want to be checking this everyday, given that some 
> of them happen every 6 […]
the check itself takes about 5 seconds, I think it is fine :)



--
To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/35223?usp=email
To unsubscribe, or for help writing mail filters, visit 
https://gerrit.osmocom.org/settings

Gerrit-Project: osmo-ci
Gerrit-Branch: master
Gerrit-Change-Id: I0abc49a95197da55f7d83ed4fd1c4ebb6bd14b1e
Gerrit-Change-Number: 35223
Gerrit-PatchSet: 1
Gerrit-Owner: osmith 
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: pespin 
Gerrit-Attention: pespin 
Gerrit-Comment-Date: Tue, 05 Dec 2023 13:33:18 +
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Comment-In-Reply-To: pespin 
Gerrit-MessageType: comment

[M] Change in osmo-ci[master]: OBS: add jenkins job to check for new distros

2023-12-05 Thread pespin 
Attention is currently required from: osmith.

pespin has posted comments on this change. ( 
https://gerrit.osmocom.org/c/osmo-ci/+/35223?usp=email )

Change subject: OBS: add jenkins job to check for new distros
..


Patch Set 1: Code-Review+1

(1 comment)

Patchset:

PS1:
I wonder whether we really want to be checking this everyday, given that some 
of them happen every 6 months or so.



--
To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/35223?usp=email
To unsubscribe, or for help writing mail filters, visit 
https://gerrit.osmocom.org/settings

Gerrit-Project: osmo-ci
Gerrit-Branch: master
Gerrit-Change-Id: I0abc49a95197da55f7d83ed4fd1c4ebb6bd14b1e
Gerrit-Change-Number: 35223
Gerrit-PatchSet: 1
Gerrit-Owner: osmith 
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: pespin 
Gerrit-Attention: osmith 
Gerrit-Comment-Date: Tue, 05 Dec 2023 13:08:41 +
Gerrit-HasComments: Yes
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

[M] Change in osmo-ci[master]: OBS: add jenkins job to check for new distros

2023-12-05 Thread osmith 
osmith has uploaded this change for review. ( 
https://gerrit.osmocom.org/c/osmo-ci/+/35223?usp=email )


Change subject: OBS: add jenkins job to check for new distros
..

OBS: add jenkins job to check for new distros

Add a new jenkins job to notify us about new versions of Debian,
Raspbian and Ubuntu becoming available in the openSUSE OBS. This allows
us to consistently follow along and make the repositories available in
the Osmocom OBS. I've decided to check the openSUSE OBS instead of
checking somewhere upstream, because it takes time between a new
release of a distribution and the version becoming available in
openSUSE OBS, and we have another job that syncs the configuration with
that OBS instance.

Related: OS#6163
Change-Id: I0abc49a95197da55f7d83ed4fd1c4ebb6bd14b1e
---
M jobs/README.adoc
A jobs/osmocom-obs-check-new-distros.yml
A scripts/obs/check_new_distros.py
M scripts/obs/lib/config.py
4 files changed, 181 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.osmocom.org:29418/osmo-ci refs/changes/23/35223/1

diff --git a/jobs/README.adoc b/jobs/README.adoc
index 86fc485..c37b3bb 100644
--- a/jobs/README.adoc
+++ b/jobs/README.adoc
@@ -123,6 +123,7 @@

  18:00 - 21:00 OBS related
18:XX osmocom-obs-sync (sync Debian:12 etc. with openSUSE OBS)
+   19:XX osmocom-obs-check-new-distros
19:XX osmocom-obs-wireshark
20:XX osmocom-obs (new binary packages start building on OBS!)

diff --git a/jobs/osmocom-obs-check-new-distros.yml 
b/jobs/osmocom-obs-check-new-distros.yml
new file mode 100644
index 000..760479a
--- /dev/null
+++ b/jobs/osmocom-obs-check-new-distros.yml
@@ -0,0 +1,39 @@
+---
+- job:
+name: 'Osmocom_OBS_check_new_distros'
+project-type: freestyle
+concurrent: false
+defaults: global
+description: |
+  Check whether new distributions (Debian, Raspyberry Pi OS, Ubuntu) are
+  available in openSUSE's OBS that we need to add to the Osmocom OBS.
+properties:
+  - build-discarder:
+  days-to-keep: 30
+  num-to-keep: 30
+node: obs
+parameters:
+  - string:
+  name: CI_BRANCH
+  description: osmo-ci.git branch
+  default: 'master'
+scm:
+  - git:
+  branches:
+- '$CI_BRANCH'
+  url: https://gerrit.osmocom.org/osmo-ci
+  git-config-name: 'Jenkins Builder'
+  git-config-email: 'jenk...@osmocom.org'
+builders:
+  - shell: |
+  export PYTHONUNBUFFERED=1
+  ./scripts/obs/check_new_distros.py --docker
+triggers:
+  - timed: "H 19 * * *"
+wrappers:
+- ansicolor:
+colormap: xterm
+publishers:
+  - email:
+  notify-every-unstable-build: true
+  recipients: 'jenkins-notificati...@lists.osmocom.org'
diff --git a/scripts/obs/check_new_distros.py b/scripts/obs/check_new_distros.py
new file mode 100755
index 000..962d547
--- /dev/null
+++ b/scripts/obs/check_new_distros.py
@@ -0,0 +1,110 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright 2023 sysmocom - s.f.m.c. GmbH 
+import argparse
+import lib.docker
+import lib.osc
+import sys
+
+projects_opensuse = None
+projects_osmocom = None
+
+
+def parse_args():
+parser = argparse.ArgumentParser(description="Check for new distribution"
+" projects on the openSUSE OBS, that we want to configure in the"
+" Osmocom OBS as soon as they are available")
+parser.add_argument("-d", "--docker",
+help="run in docker to avoid installing required pkgs",
+action="store_true")
+parser.add_argument("-v", "--verbose", action="store_true",
+help="always print shell commands and their output,"
+ " instead of only printing them on error")
+
+advanced = parser.add_argument_group("advanced options")
+advanced.add_argument("-A", "--apiurl", help="source OBS API URL"
+  " (default: https://api.opensuse.org)",
+  default="https://api.opensuse.org;)
+advanced.add_argument("-p", "--prefix", default="openSUSE.org-mirror",
+  help="destination OBS prefix"
+   " (default: openSUSE.org-mirror)")
+advanced.add_argument("-t", "--to-apiurl", help="destination OBS API URL"
+  " (default: https://obs.osmocom.org)",
+  default="https://obs.osmocom.org;)
+
+args = parser.parse_args()
+lib.set_args(args)
+
+lib.osc.check_oscrc()
+
+if args.docker:
+lib.docker.run_in_docker_and_exit("check_new_distros.py", 
add_oscrc=True)
+
+
+def find_highest_distro_project(distro):
+highest = None
+for project in projects_opensuse:
+if not project.startswith(f"{distro}:"):
+continue
+
+num = project[len(distro) + 1:]
+if not