Script 'mail_helper' called by obssrc
Hello community,
here is the log from the commit of package openSUSE-release-tools for
openSUSE:Factory checked in at 2023-02-17 16:45:45
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/openSUSE-release-tools (Old)
and /work/SRC/openSUSE:Factory/.openSUSE-release-tools.new.22824 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "openSUSE-release-tools"
Fri Feb 17 16:45:45 2023 rev:459 rq:1066361 version:20230210.1cf4e6b
Changes:
--------
---
/work/SRC/openSUSE:Factory/openSUSE-release-tools/openSUSE-release-tools.changes
2023-01-17 17:35:38.465295003 +0100
+++
/work/SRC/openSUSE:Factory/.openSUSE-release-tools.new.22824/openSUSE-release-tools.changes
2023-02-17 16:46:10.651258818 +0100
@@ -1,0 +2,101 @@
+Fri Feb 10 10:11:22 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230210.1cf4e6b:
+ * leapmicro: fix the wrong Leap Micro version in RelPkgs pipeline
+
+-------------------------------------------------------------------
+Thu Feb 09 08:41:15 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230209.cdbe773:
+ * Update totestmanager.gocd.yaml.erb
+ * Add LeapMicro 5.4 pipelines
+
+-------------------------------------------------------------------
+Thu Feb 02 16:27:26 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230202.cbaa2d4:
+ * Release the multibuild container itself, that will cover all flavors
+
+-------------------------------------------------------------------
+Thu Feb 02 09:14:56 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230202.d737b9f:
+ * Tag the previous image in the cleanup script
+ * Add cleanup script running at night on botmaster
+
+-------------------------------------------------------------------
+Tue Jan 31 11:57:59 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230131.9980d01:
+ * Remove trailing whitespace in gocd/sp.target.gocd.yaml
+ * Port over more uses of Request.get_creator()
+
+-------------------------------------------------------------------
+Mon Jan 30 15:12:57 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230130.8e810cc:
+ * More gocd errors in the new SLE pipelines
+ * Fix gocd config ident - listing a stage as a job
+ * gocd: Add parametrized pipeline for automatic release
+ * gocd: Add parallel stage for totest release
+
+-------------------------------------------------------------------
+Sun Jan 29 18:09:31 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230129.a7da0b8:
+ * Don't schedule Leap 15.3 tests in O3
+
+-------------------------------------------------------------------
+Wed Jan 25 19:08:51 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230125.2b3c045:
+ * Allow specifying a devel project that has "Factory" in its name
+
+-------------------------------------------------------------------
+Wed Jan 25 18:22:19 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230125.2c0e252:
+ * Add explicit deprecation warning
+
+-------------------------------------------------------------------
+Wed Jan 25 18:04:48 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230125.be71541:
+ * Handle special DVD+Image BUILD values in SLE Micro Staging jobs
+
+-------------------------------------------------------------------
+Wed Jan 25 17:47:53 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230125.61fdf0f:
+ * accept_command: cleanup redundant ports_prjs assignment
+
+-------------------------------------------------------------------
+Wed Jan 25 17:41:50 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230125.394f8ee:
+ * use request.creator rather than request.get_creator() which is deprecated
+ * Use old function to pass tests
+ * Restore old xpathsearch for fullhistory searches
+ * Use the 15.4 container
+ * Fixes for osc-1.x
+
+-------------------------------------------------------------------
+Wed Jan 25 14:55:27 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230125.29aed3c:
+ * Make REGEX_LINE backwards compatible
+ * Fix regex for access logs parsing
+
+-------------------------------------------------------------------
+Wed Jan 25 13:07:19 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230125.4d5b976:
+ * Switch to 15.4, 15.3 has been deleted
+
+-------------------------------------------------------------------
+Tue Jan 24 12:27:48 UTC 2023 - opensuse-releaset...@opensuse.org
+
+- Update to version 20230124.c53e264:
+ * gocd/rabbit-openqa.py: Always include flavor, name and machine in the check
+
+-------------------------------------------------------------------
Old:
----
openSUSE-release-tools-20230110.5f7806f.obscpio
New:
----
openSUSE-release-tools-20230210.1cf4e6b.obscpio
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ openSUSE-release-tools.spec ++++++
--- /var/tmp/diff_new_pack.nEgZGl/_old 2023-02-17 16:46:11.459263369 +0100
+++ /var/tmp/diff_new_pack.nEgZGl/_new 2023-02-17 16:46:11.467263413 +0100
@@ -20,7 +20,7 @@
%define source_dir openSUSE-release-tools
%define announcer_filename factory-package-news
Name: openSUSE-release-tools
-Version: 20230110.5f7806f
+Version: 20230210.1cf4e6b
Release: 0
Summary: Tools to aid in staging and release work for openSUSE/SUSE
License: GPL-2.0-or-later AND MIT
++++++ _servicedata ++++++
--- /var/tmp/diff_new_pack.nEgZGl/_old 2023-02-17 16:46:11.547263864 +0100
+++ /var/tmp/diff_new_pack.nEgZGl/_new 2023-02-17 16:46:11.551263886 +0100
@@ -1,7 +1,7 @@
<servicedata>
<service name="tar_scm">
<param
name="url">https://github.com/openSUSE/openSUSE-release-tools.git</param>
- <param
name="changesrevision">5f7806f1e084217fc0dc4878e5feeeeaf21cbe24</param>
+ <param
name="changesrevision">1cf4e6bb02ea1352c8257b391e4c814aa03d8924</param>
</service>
</servicedata>
++++++ openSUSE-release-tools-20230110.5f7806f.obscpio ->
openSUSE-release-tools-20230210.1cf4e6b.obscpio ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/openSUSE-release-tools-20230110.5f7806f/ReviewBot.py
new/openSUSE-release-tools-20230210.1cf4e6b/ReviewBot.py
--- old/openSUSE-release-tools-20230110.5f7806f/ReviewBot.py 2023-01-10
08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/ReviewBot.py 2023-02-10
11:08:40.000000000 +0100
@@ -385,7 +385,7 @@
return self.devel_project_review_add(request, project, package,
message)
def devel_project_review_needed(self, request, project, package):
- author = request.get_creator()
+ author = request.creator
maintainers = set(maintainers_get(self.apiurl, project, package))
if author in maintainers:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/check_maintenance_incidents.py
new/openSUSE-release-tools-20230210.1cf4e6b/check_maintenance_incidents.py
--- old/openSUSE-release-tools-20230110.5f7806f/check_maintenance_incidents.py
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/check_maintenance_incidents.py
2023-02-10 11:08:40.000000000 +0100
@@ -55,7 +55,7 @@
# check if pkgname was submitted by the correct maintainer. If not, set
# self.needs_maintainer_review
def _check_maintainer_review_needed(self, req, a):
- author = req.get_creator()
+ author = req.creator
if a.type == 'maintenance_incident':
# check if there is a link and use that or the real package
# name as src_packge may end with something like
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/check_source.py
new/openSUSE-release-tools-20230210.1cf4e6b/check_source.py
--- old/openSUSE-release-tools-20230110.5f7806f/check_source.py 2023-01-10
08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/check_source.py 2023-02-10
11:08:40.000000000 +0100
@@ -267,7 +267,7 @@
if self.add_devel_project_review:
devel_project, devel_package = devel_project_fallback(self.apiurl,
target_project, target_package)
if devel_project and devel_package:
- submitter = self.request.get_creator()
+ submitter = self.request.creator
maintainers = set(maintainers_get(self.apiurl, devel_project,
devel_package))
known_maintainer = False
if maintainers:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/check_tags_in_requests.py
new/openSUSE-release-tools-20230210.1cf4e6b/check_tags_in_requests.py
--- old/openSUSE-release-tools-20230110.5f7806f/check_tags_in_requests.py
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/check_tags_in_requests.py
2023-02-10 11:08:40.000000000 +0100
@@ -4,6 +4,8 @@
import osc.conf
import osc.core
+
+from osclib.core import get_request_list_with_history
from lxml import etree as ET
from urllib.error import HTTPError, URLError
@@ -72,7 +74,7 @@
project = project[len(prefix):]
sr = srprefix
break
- requests = osc.core.get_request_list(apiurl, project, package,
None, ['new', 'review'], 'submit')
+ requests = get_request_list_with_history(apiurl, project, package,
None, ['new', 'review'], 'submit')
except (HTTPError, URLError):
self.logger.error("caught exception while checking %s/%s",
project, package)
return None
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/dashboard/generate.py
new/openSUSE-release-tools-20230210.1cf4e6b/dashboard/generate.py
--- old/openSUSE-release-tools-20230110.5f7806f/dashboard/generate.py
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/dashboard/generate.py
2023-02-10 11:08:40.000000000 +0100
@@ -173,11 +173,11 @@
download_url='http://download.opensuse.org/ports/zsystems/tumbleweed/iso/',
openqa_group='openSUSE Tumbleweed s390x',
openqa_version='Tumbleweed', openqa_groupid=34)
else:
- fetcher.add('openSUSE:Leap:Micro:5.3', nick='Leap:Micro:5.3',
-
download_url='https://download.opensuse.org/distribution/leap-micro/5.3/product/iso',
- openqa_group='openSUSE Leap Micro 5.3',
openqa_version='5.3', openqa_groupid=102)
- fetcher.add('openSUSE:Leap:Micro:5.3:Images',
nick='Leap:Micro:5.3:Images',
- openqa_group='openSUSE Leap Micro 5.3 Image',
openqa_version='5.3', openqa_groupid=101)
+ fetcher.add('openSUSE:Leap:Micro:5.4', nick='Leap:Micro:5.4',
+
download_url='https://download.opensuse.org/distribution/leap-micro/5.4/product/iso',
+ openqa_group='openSUSE Leap Micro 5.4',
openqa_version='5.4', openqa_groupid=102)
+ fetcher.add('openSUSE:Leap:Micro:5.4:Images',
nick='Leap:Micro:5.4:Images',
+ openqa_group='openSUSE Leap Micro 5.4 Image',
openqa_version='5.4', openqa_groupid=101)
fetcher.add('openSUSE:Leap:15.5', nick='Leap:15.5',
download_url='https://download.opensuse.org/distribution/leap/15.5/iso',
openqa_group='openSUSE Leap 15', openqa_version='15.5',
openqa_groupid=50)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/data/apimap.json
new/openSUSE-release-tools-20230210.1cf4e6b/data/apimap.json
--- old/openSUSE-release-tools-20230110.5f7806f/data/apimap.json
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/data/apimap.json
2023-02-10 11:08:40.000000000 +0100
@@ -1,14 +1,4 @@
{
- "openSUSE:Leap:15.3:Update" : {
- "version": "15.3",
- "flavor": "DVD-Incidents",
- "distri": "opensuse"
- },
- "openSUSE:Backports:SLE-15-SP3:Update" : {
- "version": "15.3",
- "flavor": "DVD-Backports-Incidents",
- "distri": "opensuse"
- },
"openSUSE:Leap:15.4:Update" : {
"version": "15.4",
"flavor": "DVD-Incidents",
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/data/incidents.json
new/openSUSE-release-tools-20230210.1cf4e6b/data/incidents.json
--- old/openSUSE-release-tools-20230110.5f7806f/data/incidents.json
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/data/incidents.json
2023-02-10 11:08:40.000000000 +0100
@@ -1,16 +1,4 @@
{
- "openSUSE:Leap:15.3:Update": {
- "DISTRI": "opensuse",
- "FLAVOR": "DVD-Incidents",
- "VERSION": "15.3",
- "ARCH": "x86_64"
- },
- "openSUSE:Backports:SLE-15-SP3:Update": {
- "DISTRI": "opensuse",
- "FLAVOR": "DVD-Backports-Incidents",
- "VERSION": "15.3",
- "ARCH": "x86_64"
- },
"openSUSE:Leap:15.4:Update": {
"DISTRI": "opensuse",
"FLAVOR": "DVD-Incidents",
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/data/repos.json
new/openSUSE-release-tools-20230210.1cf4e6b/data/repos.json
--- old/openSUSE-release-tools-20230110.5f7806f/data/repos.json 2023-01-10
08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/data/repos.json 2023-02-10
11:08:40.000000000 +0100
@@ -1,35 +1,5 @@
{
"https://openqa.opensuse.org": {
- "openSUSE:Leap:15.3:Update": {
- "settings": {
- "OS_TEST_ISSUES": "",
- "FLAVOR": "DVD-Updates",
- "DISTRI": "opensuse",
- "VERSION": "15.3",
- "ARCH": "x86_64"
- },
- "test": "kde",
- "repos": [
- "http://download.opensuse.org/update/leap/15.3/oss/";,
- "http://download.opensuse.org/update/leap/15.3/non-oss/";,
- "http://download.opensuse.org/update/leap/15.3/backports/";,
- "http://download.opensuse.org/update/leap/15.3/sle/";
- ]
- },
- "openSUSE:Backports:SLE-15-SP3:Update": {
- "settings": {
- "OS_TEST_ISSUES": "",
- "FLAVOR": "DVD-Backports-Incidents",
- "DISTRI": "opensuse",
- "VERSION": "15.3",
- "ARCH": "x86_64"
- },
- "test": "kde",
- "repos": [
- "http://download.opensuse.org/update/leap/15.3/backports/";,
- "http://download.opensuse.org/update/leap/15.3/sle/";
- ]
- },
"openSUSE:Leap:15.4:Update": {
"settings": {
"OS_TEST_ISSUES": "",
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/devel-project.py
new/openSUSE-release-tools-20230210.1cf4e6b/devel-project.py
--- old/openSUSE-release-tools-20230110.5f7806f/devel-project.py
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/devel-project.py
2023-02-10 11:08:40.000000000 +0100
@@ -7,7 +7,6 @@
import osc.conf
from osc.core import HTTPError
-from osc.core import get_request_list
from osc.core import get_review_list
from osc.core import show_package_meta
from osc.core import show_project_meta
@@ -15,6 +14,7 @@
from osclib.conf import Config
from osclib.core import devel_project_fallback
from osclib.core import entity_email
+from osclib.core import get_request_list_with_history
from osclib.core import package_list_kind_filtered
from osclib.core import request_age
from osclib.stagingapi import StagingAPI
@@ -163,10 +163,9 @@
# Disable including source project in get_request_list() query.
osc.conf.config['include_request_from_project'] = False
for devel_project in devel_projects:
- requests = get_request_list(apiurl, devel_project,
- req_state=('new', 'review'),
- req_type='submit',
- withfullhistory=True)
+ requests = get_request_list_with_history(
+ apiurl, devel_project, req_state=('new', 'review'),
+ req_type='submit')
for request in requests:
action = request.actions[0]
age = request_age(request).days
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/dist/ci/docker-compose.yml
new/openSUSE-release-tools-20230210.1cf4e6b/dist/ci/docker-compose.yml
--- old/openSUSE-release-tools-20230110.5f7806f/dist/ci/docker-compose.yml
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/dist/ci/docker-compose.yml
2023-02-10 11:08:40.000000000 +0100
@@ -1,7 +1,7 @@
version: "3.4"
x-obs: &obs
- image:
registry.opensuse.org/opensuse/tools/images/containers153/osrt_miniobs:latest
+ image:
registry.opensuse.org/opensuse/tools/images/containers154/osrt_miniobs:latest
x-testenv: &testenv
image:
registry.opensuse.org/opensuse/tools/images/containers_tumbleweed/osrt_testenv_tumbleweed:latest
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/dist/ci/miniobs/Dockerfile
new/openSUSE-release-tools-20230210.1cf4e6b/dist/ci/miniobs/Dockerfile
--- old/openSUSE-release-tools-20230110.5f7806f/dist/ci/miniobs/Dockerfile
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/dist/ci/miniobs/Dockerfile
2023-02-10 11:08:40.000000000 +0100
@@ -1,7 +1,7 @@
#!BuildTag: osrt_miniobs
-FROM opensuse/leap:15.3
+FROM opensuse/leap:15.4
-RUN zypper ar
http://download.opensuse.org/repositories/OBS:/Server:/Unstable/15.3/ 'O:S:U'; \
+RUN zypper ar
http://download.opensuse.org/repositories/OBS:/Server:/Unstable/15.4/ 'O:S:U'; \
zypper --gpg-auto-import-keys refresh
RUN zypper install -y obs-api obs-worker obs-server \
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/gocd/admin/cleanup.py
new/openSUSE-release-tools-20230210.1cf4e6b/gocd/admin/cleanup.py
--- old/openSUSE-release-tools-20230110.5f7806f/gocd/admin/cleanup.py
1970-01-01 01:00:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/gocd/admin/cleanup.py
2023-02-10 11:08:40.000000000 +0100
@@ -0,0 +1,225 @@
+import requests
+import sys
+import time
+import subprocess
+import shutil
+import glob
+import os
+import psycopg2
+
+
+token = open("gocd_api.token").read().strip()
+api_prefix = "http://localhost:8153/go/api";
+container_image =
"registry.opensuse.org/opensuse/tools/images/containers_tumbleweed/gocd-agent-release-tools"
+# api_prefix = 'https://botmaster.suse.de/go/api'
+
+
+def botmaster_headers(version):
+ headers = {"Authorization": f"Bearer {token}"}
+ headers["X-GoCD-Confirm"] = "true"
+ headers["Accept"] = f"application/vnd.go.cd.v{version}+json"
+ headers["Content-Type"] = "application/json"
+ return headers
+
+
+def botmaster_get(url, version):
+ url = api_prefix + url
+ return requests.get(url, headers=botmaster_headers(version))
+
+
+def botmaster_post(url, version):
+ url = api_prefix + url
+ return requests.post(url, headers=botmaster_headers(version))
+
+
+def botmaster_delete(url, version):
+ url = api_prefix + url
+ return requests.delete(url, headers=botmaster_headers(version))
+
+
+def botmaster_patch(url, data, version):
+ url = api_prefix + url
+ return requests.patch(url, data, headers=botmaster_headers(version))
+
+
+def delete_agents(only_disable=False):
+ x = botmaster_get("/agents", version=7)
+ if x.status_code not in [200]:
+ print("Can't retrieve agent list")
+ sys.exit(1)
+ agents = x.json()["_embedded"]["agents"]
+ for agent in agents:
+ url = f'/agents/{agent["uuid"]}'
+ # first needs to be disabled
+ if agent["agent_config_state"] != "Disabled":
+ x = botmaster_patch(url, '{"agent_config_state": "Disabled"}',
version=7)
+ if x.status_code != 200:
+ print("Can't disable agent", url, x, x.content)
+ continue
+ if not only_disable:
+ botmaster_delete(url, version=7)
+
+
+def cleanup_cache():
+ for file in glob.glob("/srv/go-repository-cache/repo-*solv*"):
+ os.unlink(file)
+ for suffix in ["openSUSE:Maintenance:*", "*:Staging:adi:*", "repo-*"]:
+ for dir in glob.glob(f"/srv/go-repository-cache/{suffix}"):
+ shutil.rmtree(dir)
+ for root, dir, files in os.walk("/srv/go-repository-cache/"):
+ if root.endswith("/.cache"):
+ for file in files:
+ os.unlink(os.path.join(root, file))
+
+
+def remove_old_runs(cur, pipeline):
+ cur.execute(
+ """SELECT distinct p.id,p.label from pipelines p join stages s on
s.pipelineid = p.id
+ and s.createdtime < current_date - interval '5' day and
p.name=%s""",
+ (pipeline,),
+ )
+ ids = []
+ for row in cur.fetchall():
+ id, label = row
+ print("Remove", pipeline, label)
+ path = f"/var/lib/go-server/artifacts/pipelines/{pipeline}/{label}"
+ try:
+ shutil.rmtree(path)
+ except FileNotFoundError:
+ pass
+ ids.append(id)
+ cur.execute(
+ "delete from BUILDSTATETRANSITIONS where STAGEID in (select id from
stages where pipelineid = ANY(%s))",
+ (ids,),
+ )
+ cur.execute(
+ "delete from PIPELINEMATERIALREVISIONS where pipelineid = ANY(%s)",
(ids,)
+ )
+ cur.execute(
+ "delete from builds where stageid in (select id from stages where
pipelineid = ANY(%s))",
+ (ids,),
+ )
+ cur.execute("delete from stages where pipelineid = ANY(%s)", (ids,))
+ cur.execute("delete from pipelines where id = ANY(%s)", (ids,))
+ cur.execute("update pipelines set NATURALORDER=0 where name = %s",
(pipeline,))
+
+
+def cleanup_old_pipelines():
+ # very safe password - only reachable from localhost
+ conn = psycopg2.connect(
+ "dbname=gocd user=gocd_database_user password=gocd_database_password
host=localhost"
+ )
+ with conn.cursor() as cur:
+ cur.execute("SELECT distinct name from pipelines")
+ pipelines = sorted([row[0] for row in cur.fetchall()])
+ for pipeline in pipelines:
+ remove_old_runs(cur, pipeline)
+ conn.commit()
+ cur.execute(
+ """delete from modifiedfiles where modificationid in
+ (select id from modifications where modifiedtime < current_date
- interval '10' day)"""
+ )
+ # cur.execute("delete from modifications where modifiedtime <
current_date - interval '10' day")
+ conn.close()
+
+
+def main():
+ # Make a tag of the preivous image
+ subprocess.run(
+ [
+ "docker",
+ "tag",
+ container_image + ":latest",
+ container_image + ":previous",
+ ],
+ check=True,
+ )
+
+ # pull new image - if registry is down, better stop here
+ subprocess.run(
+ [
+ "docker",
+ "pull",
+ container_image + ":latest",
+ ],
+ check=True,
+ )
+
+ # disable all agents
+ delete_agents(only_disable=True)
+
+ # putting the server into maintenance mode
+ x = botmaster_post("/admin/maintenance_mode/enable", version=1)
+ if x.status_code not in [204, 409]:
+ print("Failed to enable maintenance mode", x, x.content)
+ sys.exit(1)
+
+ # wait for all jobs to finish - cancel the monitors manually
+ while True:
+ info = botmaster_get("/admin/maintenance_mode/info", version=1)
+ if info.status_code not in [200]:
+ print("Failed to retrieve maintenance mode info", info,
info.content)
+ sys.exit(1)
+ info = info.json()
+ if not info["is_maintenance_mode"]:
+ print("Failed to enable maintenance mode", x, x.content)
+ sys.exit(1)
+
+ running_systems =
info["attributes"]["running_systems"]["building_jobs"]
+ if len(running_systems) == 0:
+ break
+ pipelines = []
+ for job in running_systems:
+ if job["pipeline_name"] in [
+ "SUSE.Repo.Monitor",
+ "SUSE.openQA",
+ "openSUSE.Repo.Monitor",
+ "openSUSE.openQA",
+ ]:
+ url =
f"/stages/{job['pipeline_name']}/{job['pipeline_counter']}/{job['stage_name']}/{job['stage_counter']}/cancel"
+ x = botmaster_post(url, version=3)
+ if x.status_code not in [200]:
+ print(f"Can't cancel {job['pipline_name']}")
+ print(x, x.content)
+ sys.exit(1)
+ else:
+ pipelines.append(job["pipeline_name"])
+ print("Waiting 2 min for the jobs to finish", sorted(pipelines))
+ time.sleep(120)
+
+ # stop all agents
+ subprocess.run(["systemctl", "stop", "go-agent-*"], check=True)
+
+ # cleanup
+ proc = subprocess.Popen(["docker", "system", "prune"],
stdin=subprocess.PIPE)
+ proc.communicate(input=b"y\n")
+
+ # stop the server
+ subprocess.run(["systemctl", "stop", "go-server.service"])
+
+ cleanup_old_pipelines()
+
+ # start the server again
+ subprocess.run(["systemctl", "start", "go-server.service"])
+
+ while True:
+ try:
+ info = botmaster_get("/admin/maintenance_mode/info", version=1)
+ print(info, info.content)
+ if info.status_code in [503]:
+ # is starting
+ time.sleep(5)
+ continue
+ break
+ except requests.exceptions.ConnectionError:
+ time.sleep(5)
+ continue
+
+ delete_agents()
+ cleanup_cache()
+
+ # start the agents on new image
+ subprocess.run(["systemctl", "start", "--all", "go-agent-*"], check=True)
+
+
+main()
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/gocd/admin/remove-old-logs.service
new/openSUSE-release-tools-20230210.1cf4e6b/gocd/admin/remove-old-logs.service
---
old/openSUSE-release-tools-20230110.5f7806f/gocd/admin/remove-old-logs.service
1970-01-01 01:00:00.000000000 +0100
+++
new/openSUSE-release-tools-20230210.1cf4e6b/gocd/admin/remove-old-logs.service
2023-02-10 11:08:40.000000000 +0100
@@ -0,0 +1,8 @@
+[Unit]
+Description=Remove old gocd logs
+After=local-fs.target
+
+[Service]
+Type=oneshot
+WorkingDirectory=/root/admin
+ExecStart=/usr/bin/python3 cleanup.py
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/gocd/admin/remove-old-logs.timer
new/openSUSE-release-tools-20230210.1cf4e6b/gocd/admin/remove-old-logs.timer
---
old/openSUSE-release-tools-20230110.5f7806f/gocd/admin/remove-old-logs.timer
1970-01-01 01:00:00.000000000 +0100
+++
new/openSUSE-release-tools-20230210.1cf4e6b/gocd/admin/remove-old-logs.timer
2023-02-10 11:08:40.000000000 +0100
@@ -0,0 +1,9 @@
+[Unit]
+Description=Remove Old gocd logs
+After=local-fs.target
+
+[Timer]
+OnCalendar=*-*-* 01:07:00
+
+[Install]
+WantedBy=timers.target
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/gocd/leapmicro.target.gocd.yaml
new/openSUSE-release-tools-20230210.1cf4e6b/gocd/leapmicro.target.gocd.yaml
--- old/openSUSE-release-tools-20230110.5f7806f/gocd/leapmicro.target.gocd.yaml
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/gocd/leapmicro.target.gocd.yaml
2023-02-10 11:08:40.000000000 +0100
@@ -17,26 +17,7 @@
resources:
- repo-checker
tasks:
- - script: ./pkglistgen.py -A https://api.opensuse.org
update_and_solve -p openSUSE:Leap:Micro:5.2 -s target --only-release-packages
-
- LeapMicro53.RelPkgs:
- group: Leap
- lock_behavior: unlockWhenFinished
- timer:
- spec: 0 10 * ? * *
- only_on_changes: false
- materials:
- git:
- git: https://github.com/openSUSE/openSUSE-release-tools.git
- environment_variables:
- OSC_CONFIG: /home/go/config/oscrc-staging-bot
- stages:
- - Create.Release.Packages:
- approval: manual
- resources:
- - repo-checker
- tasks:
- - script: ./pkglistgen.py -A https://api.opensuse.org
update_and_solve -p openSUSE:Leap:Micro:5.3 -s target --only-release-packages
+ - script: ./pkglistgen.py -A https://api.opensuse.org
update_and_solve -p openSUSE:Leap:Micro:5.4 -s target --only-release-packages
LeapMicro.Package.Lists:
group: Leap
@@ -54,23 +35,4 @@
resources:
- repo-checker
tasks:
- - script: ./pkglistgen.py -A https://api.opensuse.org --debug
update_and_solve -p openSUSE:Leap:Micro:5.2 -s target
-
- LeapMicro53.Package.Lists:
- group: Leap
- lock_behavior: unlockWhenFinished
- environment_variables:
- OSC_CONFIG: /home/go/config/oscrc-staging-bot
- timer:
- spec: 0 40 * ? * *
- only_on_changes: false
- materials:
- git:
- git: https://github.com/openSUSE/openSUSE-release-tools.git
- stages:
- - Update.000product:
- resources:
- - repo-checker
- tasks:
- - script: ./pkglistgen.py -A https://api.opensuse.org --debug
update_and_solve -p openSUSE:Leap:Micro:5.3 -s target
-
+ - script: ./pkglistgen.py -A https://api.opensuse.org --debug
update_and_solve -p openSUSE:Leap:Micro:5.4 -s target
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/gocd/rabbit-openqa.py
new/openSUSE-release-tools-20230210.1cf4e6b/gocd/rabbit-openqa.py
--- old/openSUSE-release-tools-20230110.5f7806f/gocd/rabbit-openqa.py
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/gocd/rabbit-openqa.py
2023-02-10 11:08:40.000000000 +0100
@@ -11,6 +11,7 @@
from osclib.stagingapi import StagingAPI
from lxml import etree as ET
from openqa_client.client import OpenQA_Client
+from packaging import version
from urllib.error import HTTPError
from urllib.parse import quote_plus
@@ -100,11 +101,39 @@
# collect job infos to pick names
for job in openqa:
print(staging, iso, job['id'], job['state'], job['result'],
- job['settings']['MACHINE'], job['settings']['TEST'])
+ job['settings']['FLAVOR'], job['settings']['TEST'],
job['settings']['MACHINE'])
openqa_infos[job['id']] = {'url': self.listener.test_url(job)}
openqa_infos[job['id']]['state'] = self.map_openqa_result(job)
- openqa_infos[job['id']]['name'] = job['settings']['TEST']
- openqa_infos[job['id']]['machine'] = job['settings']['MACHINE']
+ openqa_infos[job['id']]['build'] = job['settings']['BUILD']
+ openqa_infos[job['id']]['name'] =
f"{job['settings']['FLAVOR']}-{job['settings']['TEST']}@{job['settings']['MACHINE']}"
+
+ def compare_simple_builds(build1, build2):
+ """Simple build number comparison"""
+ ver1 = version.parse(build1)
+ ver2 = version.parse(build2)
+ if ver1 < ver2:
+ return -1
+ if ver1 > ver2:
+ return 1
+ return 0
+
+ def compare_composite_builds(build1, build2):
+ """Compare BUILD numbers consisting of multiple _-separated
components."""
+ components1 = build1.split('_')
+ components2 = build2.split('_')
+ if len(components1) != len(components2):
+ raise Exception(f'Failed to compare {build1} and {build2}:
Different format')
+
+ component_cmps = [Project.compare_simple_builds(components1[i],
components2[i]) for i in range(0, len(components1))]
+ less = -1 in component_cmps
+ greater = 1 in component_cmps
+ if less and greater:
+ raise Exception(f'Failed to compare {build1} and {build2}: Not
ordered')
+ if less:
+ return -1
+ if greater:
+ return 1
+ return 0
def update_staging_status(self, staging):
openqa_infos = dict()
@@ -119,18 +148,36 @@
url = self.api.makeurl(['status_reports', 'published', staging,
'images', 'reports', buildid])
# make sure the names are unique
+ obsolete_jobs = []
taken_names = dict()
for id in openqa_infos:
name = openqa_infos[id]['name']
if name in taken_names:
- openqa_infos[id]['name'] = openqa_infos[id]['name'] + "@" +
openqa_infos[id]['machine']
- # the other id
- oid = taken_names[name]
- openqa_infos[oid]['name'] = openqa_infos[oid]['name'] + "@" +
openqa_infos[oid]['machine']
- if openqa_infos[id]['name'] == openqa_infos[oid]['name']:
- raise Exception(f'Names of job #{id} and #{oid} collide')
+ # There are multiple jobs with that specific
FLAVOR-TEST@MACHINE.
+ # In SLE Micro, jobs currently use
BUILD=(dvdbuild)_(image_build),
+ # so if the dvd is rebuilt, new image jobs are triggered for
the
+ # same binary. The openQA ?latest=1 filter doesn't look at
that,
+ # so we have to figure out which of those is the most recent
one.
+ build1 = openqa_infos[taken_names[name]]['build']
+ build2 = openqa_infos[id]['build']
+ if '_' in build1 and '_' in build2 and build1 != build2:
+ # Use the more recent build
+ buildcmp = Project.compare_composite_builds(build1, build2)
+ self.logger.info(f'Multiple builds for {name}, {build1}
and {build2}. Comparison: {buildcmp}')
+ if buildcmp < 0: # Drop the previous one
+ obsolete_jobs.append(taken_names[name])
+ taken_names[name] = id
+ continue
+ elif buildcmp > 0: # Drop this one
+ obsolete_jobs.append(id)
+ continue
+
+ raise Exception(f'Names of job #{id} and #{taken_names[name]}
collide: {name}')
taken_names[name] = id
+ for id in obsolete_jobs:
+ del openqa_infos[id]
+
for info in openqa_infos.values():
xml = self.openqa_check_xml(info['url'], info['state'], 'openqa:'
+ info['name'])
try:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/gocd/sp.target.gocd.yaml
new/openSUSE-release-tools-20230210.1cf4e6b/gocd/sp.target.gocd.yaml
--- old/openSUSE-release-tools-20230110.5f7806f/gocd/sp.target.gocd.yaml
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/gocd/sp.target.gocd.yaml
2023-02-10 11:08:40.000000000 +0100
@@ -45,7 +45,7 @@
osc -A https://api.suse.de release
--target-project=SUSE:SLE-15-SP5:GA:TEST --target-repository=containers -r
containerfile SUSE:SLE-15-SP5:GA $container
done
osc -A https://api.suse.de release
--target-project=SUSE:SLE-15-SP5:GA:TEST --target-repository=containers -r
images SUSE:SLE-15-SP5:GA sles15-image
- PRODUCTS=$(osc -A https://api.suse.de ls SUSE:SLE-15-SP5:GA | grep
"^SLES15-SP5" | grep -v release ||:)
+ PRODUCTS=$(osc -A https://api.suse.de ls SUSE:SLE-15-SP5:GA | grep
"^SLES15-SP5" | grep -v release | grep -v : ||:)
if [ -z "${PRODUCTS}" ]; then
echo "[WARNNING] There is no cloud image to be released"
fi
@@ -64,6 +64,20 @@
done
osc -A https://api.suse.de/ api
"/build/SUSE:SLE-15-SP5:GA:TEST/_result?view=summary&repository=images" | grep
"result project" | grep 'code="published" state="published">' && echo PUBLISHED
+ - Release.Images.To.Test.New:
+ approval: manual
+ roles:
+ - SLE
+ environment_variables:
+ OSC_CONFIG: /home/go/config/oscrc-totest-manager
+ JIRA_PAT: /home/go/config/jira-pat
+ JIRA_INSTANCE: /home/go/config/jira-instance
+ resources:
+ - staging-bot
+ tasks:
+ - script: |-
+ sle-prjmgr-tools --jira-instance $JIRA_INSTANCE --osc-config
$OSC_CONFIG ibs_to_jira --jira-pat $JIRA_PAT -p SUSE:SLE-15-SP5:GA
+
- Release.Images.To.Publish:
approval: manual
roles:
@@ -151,3 +165,51 @@
- staging-bot
tasks:
- script: osc -A https://api.suse.de release SUSE:SLE-15-SP5:GA:TEST
+
+ SLE15.SP5.ReleaseScc:
+ group: SLE15.Target
+ parameters:
+ product: sle15sp5
+ target: MyTarget
+ build: "0.0"
+ materials:
+ scripts:
+ auto_update: true
+ git: https://github.com/openSUSE/openSUSE-release-tools.git
+ whitelist:
+ - DO_NOT_TRIGGER
+ destination: scripts
+ stages:
+ # "ratt" is an alias for the internal IP of the API, only available on
botmaster.
+ - Release.Images.Update:
+ approval: manual
+ roles:
+ - SLE
+ resources:
+ - staging-bot
+ tasks:
+ - script: curl --verbose --max-time 3600
"http://ratt:5000/update?product=#{product}&target=#{target}&build=#{build}";
+ - Release.Images.Rename:
+ approval: manual
+ roles:
+ - SLE
+ resources:
+ - staging-bot
+ tasks:
+ - script: curl --verbose --max-time 3600
"http://ratt:5000/rename?product=#{product}&target=#{target}";
+ - Release.Images.Sync:
+ approval: manual
+ roles:
+ - SLE
+ resources:
+ - staging-bot
+ tasks:
+ - script: curl --verbose --max-time 3600
"http://ratt:5000/sync?product=#{product}&target=#{target}";
+ - Release.Images.Release:
+ approval: manual
+ roles:
+ - SLE
+ resources:
+ - staging-bot
+ tasks:
+ - script: curl --verbose --max-time 3600
"http://ratt:5000/release?product=#{product}";
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/gocd/totestmanager.gocd.yaml
new/openSUSE-release-tools-20230210.1cf4e6b/gocd/totestmanager.gocd.yaml
--- old/openSUSE-release-tools-20230110.5f7806f/gocd/totestmanager.gocd.yaml
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/gocd/totestmanager.gocd.yaml
2023-02-10 11:08:40.000000000 +0100
@@ -252,7 +252,7 @@
- script: |-
install -D /home/go/config/openqa-client.conf
/home/go/.config/openqa/client.conf
scripts/totest-manager.py -A https://api.opensuse.org --debug run
openSUSE:Leap:15.5:ARM:Images
- TTM.Leap_Micro_5.2:
+ TTM.Leap_Micro_5.4:
group: openSUSE.Checkers
lock_behavior: unlockWhenFinished
environment_variables:
@@ -272,8 +272,8 @@
tasks:
- script: |-
install -D /home/go/config/openqa-client.conf
/home/go/.config/openqa/client.conf
- scripts/totest-manager.py -A https://api.opensuse.org --debug run
openSUSE:Leap:Micro:5.2
- TTM.Leap_Micro_5.2_Images:
+ scripts/totest-manager.py -A https://api.opensuse.org --debug run
openSUSE:Leap:Micro:5.4
+ TTM.Leap_Micro_5.4_Images:
group: openSUSE.Checkers
lock_behavior: unlockWhenFinished
environment_variables:
@@ -293,46 +293,4 @@
tasks:
- script: |-
install -D /home/go/config/openqa-client.conf
/home/go/.config/openqa/client.conf
- scripts/totest-manager.py -A https://api.opensuse.org --debug run
openSUSE:Leap:Micro:5.2:Images
- TTM.Leap_Micro_5.3:
- group: openSUSE.Checkers
- lock_behavior: unlockWhenFinished
- environment_variables:
- OSC_CONFIG: /home/go/config/oscrc-totest-manager
- materials:
- script:
- git: https://github.com/openSUSE/openSUSE-release-tools.git
- destination: scripts
- timer:
- spec: 0 */15 * ? * *
- only_on_changes: false
- stages:
- - Run:
- approval: manual
- resources:
- - staging-bot
- tasks:
- - script: |-
- install -D /home/go/config/openqa-client.conf
/home/go/.config/openqa/client.conf
- scripts/totest-manager.py -A https://api.opensuse.org --debug run
openSUSE:Leap:Micro:5.3
- TTM.Leap_Micro_5.3_Images:
- group: openSUSE.Checkers
- lock_behavior: unlockWhenFinished
- environment_variables:
- OSC_CONFIG: /home/go/config/oscrc-totest-manager
- materials:
- script:
- git: https://github.com/openSUSE/openSUSE-release-tools.git
- destination: scripts
- timer:
- spec: 0 */15 * ? * *
- only_on_changes: false
- stages:
- - Run:
- approval: manual
- resources:
- - staging-bot
- tasks:
- - script: |-
- install -D /home/go/config/openqa-client.conf
/home/go/.config/openqa/client.conf
- scripts/totest-manager.py -A https://api.opensuse.org --debug run
openSUSE:Leap:Micro:5.3:Images
+ scripts/totest-manager.py -A https://api.opensuse.org --debug run
openSUSE:Leap:Micro:5.4:Images
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/gocd/totestmanager.gocd.yaml.erb
new/openSUSE-release-tools-20230210.1cf4e6b/gocd/totestmanager.gocd.yaml.erb
---
old/openSUSE-release-tools-20230110.5f7806f/gocd/totestmanager.gocd.yaml.erb
2023-01-10 08:02:00.000000000 +0100
+++
new/openSUSE-release-tools-20230210.1cf4e6b/gocd/totestmanager.gocd.yaml.erb
2023-02-10 11:08:40.000000000 +0100
@@ -13,10 +13,8 @@
openSUSE:Leap:15.5:Images
openSUSE:Leap:15.5:ARM
openSUSE:Leap:15.5:ARM:Images
- openSUSE:Leap:Micro:5.2
- openSUSE:Leap:Micro:5.2:Images
- openSUSE:Leap:Micro:5.3
- openSUSE:Leap:Micro:5.3:Images
+ openSUSE:Leap:Micro:5.4
+ openSUSE:Leap:Micro:5.4:Images
) -%>
TTM.<%= project.gsub('openSUSE:', '').gsub(':', '_') %>:
group: openSUSE.Checkers
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/metrics/access/ingest.php
new/openSUSE-release-tools-20230210.1cf4e6b/metrics/access/ingest.php
--- old/openSUSE-release-tools-20230110.5f7806f/metrics/access/ingest.php
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/metrics/access/ingest.php
2023-02-10 11:08:40.000000000 +0100
@@ -1,7 +1,7 @@
#!/usr/bin/php
<?php
-const REGEX_LINE = '/\S+ \S+ \S+ \[([^:]+:\d+:\d+:\d+ [^\]]+)\] "(\S+)(?:
(\S+) \S+)?" (\S+) (\S+) "[^"]*" "[^"]*" .* size:(\S+) \S+(?:
+"?(\S+-\S+-\S+-\S+-[^\s"]+|-)"?
"?(dvd|ftp|mini|usb-[^"]*|livecd-[^"]*|appliance-?[^"]*|-)"?)?/';
+const REGEX_LINE = '/\S+ \S+ \S+ \[([^:]+:\d+:\d+:\d+ [^\]]+)\] "(\S+)(?:
(\S+) \S+)?" (\S+) (\S+) "[^"]*" "[^"]*" .* (?:size:|want:- give:- \d+ )(\S+)
\S+(?: +"?(\S+-\S+-\S+-\S+-[^\s"]+|-)"?
"?(dvd|ftp|mini|usb-[^"]*|livecd-[^"]*|appliance-?[^"]*|-)"?)?/';
const REGEX_PRODUCT =
'#/(?:(tumbleweed)|distribution/(?:leap/)?(\d+\.\d+)|openSUSE(?:_|:/)(?:leap(?:_|:/))?(factory|tumbleweed|\d+\.\d+))#i';
const REGEX_IMAGE =
'#(?:/(?:iso|live)/[^/]+-(DVD|NET|GNOME-Live|KDE-Live|Rescue-CD|Kubic-DVD)-[^/]+\.iso(?:\.torrent)?|/jeos/[^/]+-(JeOS)\.[^/]+\.(?:qcow2|vhdx|vmdk|vmx)$)#';
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/openSUSE-release-tools-20230110.5f7806f/metrics.py
new/openSUSE-release-tools-20230210.1cf4e6b/metrics.py
--- old/openSUSE-release-tools-20230110.5f7806f/metrics.py 2023-01-10
08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/metrics.py 2023-02-10
11:08:40.000000000 +0100
@@ -19,6 +19,7 @@
import osclib.conf
from osclib.cache import Cache
from osclib.conf import Config
+from osclib.core import get_request_list_with_history
from osclib.core import project_pseudometa_package
from osclib.stagingapi import StagingAPI
@@ -102,9 +103,8 @@
def ingest_requests(api, project):
- requests = get_request_list(api.apiurl, project,
- req_state=('accepted', 'revoked',
'superseded'),
- withfullhistory=True)
+ requests = get_request_list_with_history(
+ api.apiurl, project, req_state=('accepted', 'revoked', 'superseded'))
for request in requests:
if request.find('action').get('type') not in ('submit', 'delete'):
# TODO Handle non-stageable requests via different flow.
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/osc-staging.py
new/openSUSE-release-tools-20230210.1cf4e6b/osc-staging.py
--- old/openSUSE-release-tools-20230110.5f7806f/osc-staging.py 2023-01-10
08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/osc-staging.py 2023-02-10
11:08:40.000000000 +0100
@@ -53,14 +53,15 @@
if project.startswith(('openSUSE', 'SUSE')):
return project
- if 'Factory' in project or 'openSUSE' in project:
+ if project.startswith('Factory'):
return 'openSUSE:%s' % project
- if 'SLE' in project:
+ if project.startswith('SLE') or project.startswith('ALP'):
return 'SUSE:%s' % project
# If we can't guess, raise a Warning
- warnings.warn('%s project not recognized.' % project)
+ if (':' not in project):
+ warnings.warn('%s project not recognized.' % project)
return project
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/osclib/accept_command.py
new/openSUSE-release-tools-20230210.1cf4e6b/osclib/accept_command.py
--- old/openSUSE-release-tools-20230110.5f7806f/osclib/accept_command.py
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/osclib/accept_command.py
2023-02-10 11:08:40.000000000 +0100
@@ -275,7 +275,6 @@
self.update_version_attribute(project, curr_version)
- ports_prjs = ['PowerPC', 'ARM', 'zSystems', 'RISCV']
ports_prjs = ['ARM', 'LegacyX86', 'PowerPC', 'RISCV', 'zSystems']
for ports in ports_prjs:
project = self.api.project + ':' + ports
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/osclib/core.py
new/openSUSE-release-tools-20230210.1cf4e6b/osclib/core.py
--- old/openSUSE-release-tools-20230110.5f7806f/osclib/core.py 2023-01-10
08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/osclib/core.py 2023-02-10
11:08:40.000000000 +0100
@@ -12,13 +12,13 @@
from osc.core import get_binarylist
from osc.core import get_commitlog
from osc.core import get_dependson
-from osc.core import get_request_list
from osc.core import http_DELETE
from osc.core import http_GET
from osc.core import http_POST
from osc.core import http_PUT
from osc.core import makeurl
from osc.core import owner
+from osc.core import search as osc_core_search
from osc.core import Request
from osc.core import Action
from osc.core import show_package_meta
@@ -56,10 +56,60 @@
return members
-# osc uses xml.etree while we rely on lxml
+
+def get_request_list_with_history(
+ apiurl, project='', package='', req_who='', req_state=('new',
'review', 'declined'),
+ req_type=None, exclude_target_projects=[]):
+ """using an xpath search to get full request history. deprecated copy of
old code from osc 0.x."""
+
+ import warnings
+ warnings.warn(
+ "get_request_list_with_history() uses an xpath search, which is slow.
consider porting to"
+ "use osc.core.get_request_collection() instead.",
+ DeprecationWarning
+ )
+
+ xpath = ''
+ if 'all' not in req_state:
+ for state in req_state:
+ xpath = xpath_join(xpath, 'state/@name=\'%s\'' % state, inner=True)
+ if req_who:
+ xpath = xpath_join(xpath, '(state/@who=\'%(who)s\' or
history/@who=\'%(who)s\')' % {'who': req_who}, op='and')
+
+ # XXX: we cannot use the '|' in the xpath expression because it is not
supported
+ # in the backend
+ todo = {}
+ if project:
+ todo['project'] = project
+ if package:
+ todo['package'] = package
+ for kind, val in todo.items():
+ xpath_base = 'action/target/@%(kind)s=\'%(val)s\''
+ if conf.config['include_request_from_project']:
+ xpath_base = xpath_join(xpath_base,
'action/source/@%(kind)s=\'%(val)s\'', op='or', inner=True)
+ xpath = xpath_join(xpath, xpath_base % {'kind': kind, 'val': val},
op='and', nexpr_parentheses=True)
+
+ if req_type:
+ xpath = xpath_join(xpath, 'action/@type=\'%s\'' % req_type, op='and')
+ for i in exclude_target_projects:
+ xpath = xpath_join(xpath, '(not(action/target/@project=\'%(prj)s\'))'
% {'prj': i}, op='and')
+
+ if conf.config['verbose'] > 1:
+ print('[ %s ]' % xpath)
+ queries = {}
+ queries['request'] = {'withfullhistory': '1'}
+ res = osc_core_search(apiurl, queries=queries, request=xpath)
+ collection = res['request']
+ requests = []
+ for root in collection.findall('request'):
+ r = Request()
+ r.read(root)
+ requests.append(r)
+ return requests
def convert_from_osc_et(xml):
+ """osc uses xml.etree while we rely on lxml."""
from xml.etree import ElementTree as oscET
return ET.fromstring(oscET.tostring(xml))
@@ -1085,7 +1135,7 @@
# Disable including source project in get_request_list() query.
before = conf.config['include_request_from_project']
conf.config['include_request_from_project'] = False
- requests = get_request_list(apiurl, project, package, None, states,
request_type, withfullhistory=True)
+ requests = get_request_list_with_history(apiurl, project, package, None,
states, request_type)
conf.config['include_request_from_project'] = before
for request in requests:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/publish_distro_conf/publish_leap-micro54.config
new/openSUSE-release-tools-20230210.1cf4e6b/publish_distro_conf/publish_leap-micro54.config
---
old/openSUSE-release-tools-20230110.5f7806f/publish_distro_conf/publish_leap-micro54.config
1970-01-01 01:00:00.000000000 +0100
+++
new/openSUSE-release-tools-20230210.1cf4e6b/publish_distro_conf/publish_leap-micro54.config
2023-02-10 11:08:40.000000000 +0100
@@ -0,0 +1,50 @@
+# vim:syntax=sh
+
+micro_version=5.4
+qu="" # empty string for GA, QuarterlyUpdate repins have -N in version
+logfile_base=~/publish_logs/$micro_version/$(date -d "$date" '+%Y/%m/%d/%H%M')
+synclog="${logfile_base}.log"
+deletelog="${logfile_base}-deletes.log"
+path="/distribution/leap-micro/$micro_version/product"
+flavors=(DVD-x86_64 DVD-aarch64)
+
+get_version() {
+ # get expected version from first flavor
+ if [ -z "$version" ]; then
+ version=`echo
$stage/iso/openSUSE-Leap-Micro-$micro_version$qu-$flavor-Build[0123456789]*.[0123456789]*-Media.iso`
+ version=${version##*Build}
+ version=${version%-*}
+ if [ -z "$version" ]; then
+ echo "no version found, exit." | tee -a $synclog
+ exit 1
+ fi
+ fi
+}
+
+get_iso()
+{
+
iso="openSUSE-Leap-Micro-$micro_version$qu-$flavor-Build$version-Media.iso"
+}
+
+get_iso_link()
+{
+ link="$stage/iso/openSUSE-Leap-$micro_version$qu-$flavor-Current.iso"
+}
+
+get_diff_url()
+{
+ url="$diff_url_base/$micro_version/diff/$version"
+}
+
+get_mark_published_url()
+{
+ url="$diff_url_base/$micro_version/current"
+}
+
+get_changes_filename()
+{
+ # ChangeLog files from obsgendiff are used instead
+ #changes="$changes_dir_base/jump/$jump_version/Changes.$version.txt"
+ :
+# changes="$changes_dir_base/leap/$micro_version/Changes.$version.txt"
+}
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/publish_distro_conf/publish_leap-micro54_appliances.config
new/openSUSE-release-tools-20230210.1cf4e6b/publish_distro_conf/publish_leap-micro54_appliances.config
---
old/openSUSE-release-tools-20230110.5f7806f/publish_distro_conf/publish_leap-micro54_appliances.config
1970-01-01 01:00:00.000000000 +0100
+++
new/openSUSE-release-tools-20230210.1cf4e6b/publish_distro_conf/publish_leap-micro54_appliances.config
2023-02-10 11:08:40.000000000 +0100
@@ -0,0 +1,58 @@
+# vim:syntax=sh
+
+micro_version=5.4
+release=0
+logfile_base=~/publish_logs/$micro_version-appliances/$(date -d "$date"
'+%Y/%m/%d/%H%M')
+synclog="${logfile_base}.log"
+deletelog="${logfile_base}-deletes.log"
+path="/distribution/leap-micro/$micro_version/appliances"
+flavors=(Default Default-RT Default-SelfInstall)
+repos=()
+extra_repos=()
+isodir=""
+
+get_version() {
+ # get expected version from first flavor
+ if [ -z "$version" ]; then
+ version=`echo
$stage/openSUSE-Leap-Micro.x86_64-$micro_version.$release-$flavor-Build*.raw.xz`
+ version=${version##*Build}
+ version=${version%.*}
+ if [ -z "$version" ]; then
+ echo "no version found, exit." | tee -a $synclog
+ exit 1
+ fi
+ fi
+}
+
+_get_iso()
+{
+ local snapshot="$1"
+ local suffix=xz
+# echo
"openSUSE-Leap-Micro-$micro_version-JeOS.x86_64-$micro_version.0-$flavor-$snapshot.$suffix"
+ echo
"openSUSE-Leap-Micro.x86_64-$micro_version.$release-$flavor-$snapshot.$suffix"
+}
+
+get_iso()
+{
+ iso=`_get_iso "Build$version"`
+}
+
+get_iso_link()
+{
+ link="$stage/`_get_iso Current`"
+}
+
+get_diff_url()
+{
+ url=""
+}
+
+get_mark_published_url()
+{
+ url=""
+}
+
+get_changes_filename()
+{
+ changes=""
+}
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/openSUSE-release-tools-20230110.5f7806f/tests/OBSLocal.py
new/openSUSE-release-tools-20230210.1cf4e6b/tests/OBSLocal.py
--- old/openSUSE-release-tools-20230110.5f7806f/tests/OBSLocal.py
2023-01-10 08:02:00.000000000 +0100
+++ new/openSUSE-release-tools-20230210.1cf4e6b/tests/OBSLocal.py
2023-02-10 11:08:40.000000000 +0100
@@ -71,14 +71,18 @@
with open(OSCRC, 'w+') as f:
f.write('\n'.join([
'[general]',
- 'apiurl = http://api:3000',
+ 'apiurl = {}'.format(APIURL),
'http_debug = false',
'debug = false',
'cookiejar = {}'.format(OSCCOOKIEJAR),
- '[http://api:3000]',
+ '[{}]'.format(APIURL),
'user = {}'.format(userid),
'pass = opensuse',
'email = {}@example.com'.format(userid),
+ # allow plain http even if it is insecure; we're testing after
all
+ 'allow_http = 1',
+ # disable cert checking to allow self-signed certs
+ 'sslcertck = 0',
'',
]))
@@ -87,8 +91,7 @@
self.users.append(userid)
self.oscrc(userid)
- # Rather than modify userid and email, just re-parse entire config and
- # reset authentication by clearing opener to avoid edge-cases.
+ # Rather than modify userid and email, just re-parse entire config
self.oscParse()
def osc_user_pop(self):
@@ -96,16 +99,12 @@
self.osc_user(self.users.pop())
def oscParse(self):
- # Otherwise, will stick to first user for a given apiurl.
- conf._build_opener.last_opener = (None, None)
-
# Otherwise, will not re-parse same config file.
if 'cp' in conf.get_configParser.__dict__:
del conf.get_configParser.cp
conf.get_config(override_conffile=OSCRC,
- override_no_keyring=True,
- override_no_gnome_keyring=True)
+ override_no_keyring=True)
os.environ['OSC_CONFIG'] = OSCRC
os.environ['OSRT_DISABLE_CACHE'] = 'true'
@@ -310,8 +309,7 @@
memoize_session_reset()
osc.core.conf.get_config(override_conffile=oscrc,
- override_no_keyring=True,
- override_no_gnome_keyring=True)
+ override_no_keyring=True)
os.environ['OSC_CONFIG'] = oscrc
if os.environ.get('OSC_DEBUG'):
++++++ openSUSE-release-tools.obsinfo ++++++
--- /var/tmp/diff_new_pack.nEgZGl/_old 2023-02-17 16:46:12.659270126 +0100
+++ /var/tmp/diff_new_pack.nEgZGl/_new 2023-02-17 16:46:12.667270172 +0100
@@ -1,5 +1,5 @@
name: openSUSE-release-tools
-version: 20230110.5f7806f
-mtime: 1673334120
-commit: 5f7806f1e084217fc0dc4878e5feeeeaf21cbe24
+version: 20230210.1cf4e6b
+mtime: 1676023720
+commit: 1cf4e6bb02ea1352c8257b391e4c814aa03d8924
