Den fre. 23. okt. 2026 kl. 12.37 skrev Sven Püschel <[email protected]>: > > Hi Bruno, > > On 10/23/26 11:28, Bruno Thomsen wrote: > > Den fre. 23. okt. 2026 kl. 11.01 skrev Sven Püschel > > <[email protected]>: > >> Add an update_check python script as an easy check which PTXdist > >> rules can be bumped in their versions. It uses the report file generated > >> from an existing PTXdist BSP as a basis to determine the package > >> versions. Then it compares these package versions with the package > >> versions against the newest release versions reported by > >> release-monitoring.org . > >> > >> release-monitoring.org is a deployment of an open-source release > >> monitoring project named Anitya [1]. It allows to define a Project > >> which is some software you want to keep track of updates. A given > >> software can have multiple Projects. An example is GTK+, where you want > >> to have separate projects for GTK+ 3 and GTK+ 4. A Project can then > >> define a mapping to a package name in a Distribution. > >> > >> I've created the PTXdist distribution [2] and created Project mappings > >> for a good chunk of the PTXdist rules. I've excluded certain rules with > >> user-defined versions (e.g. the linux kernel) and projects where it was > >> unclear if PTXdist should track the latest major version or use the > >> latest version of the current major version. Also I've ignored rules, > >> where no Project existed yet. Furthermore I've skipped the `python-` > >> rules, as they can probably be added in an automated fashion (given the > >> pypy name can be used for matching purposes). > > Nice that you have added PTXdist to release-monitoring.org, you can > > also see it in Repology[1] where it has lists of outdated packages[2], > > potentially vulnerable packages[3], and packages with problems[4] like > > dead URLs or permanent redirects. Information is tracking the master branch. > > Yeah, Michael also mentioned your PTXDist patches towards repology and > that he'd prefer some solution based on a report file. >
Latest versions of the patches were not posted to the mailing list and we continued development in a separate repo[1] which also contains regular updates to the JSON output that repology consumes. > Personally I didn't quite understand how Repology actually works. It > looks like fuzzy matching of package names and getting the newest > versions based on the comparison with other distro packages. > Release-Monitoring looked dead-simple: Each Project has a dedicated > definition where to look for new versions and a explicit mapping to a > distro package name. Therefore I'd just went with it (with the side > effect of also making PTXdist a bit present there). I'm not too > invested, which Backend the script uses to actually get the necessary > data (or it could also use multiple backends, etc.). Yes, packages are fuzzy matched by download URL + name, and new versions are detected when 2 or more distributions have a newer version. I like Release-Monitoring as well, so I get a RHBZ ping to update a RPM spec :) They are good for two slightly different things, but I hope you get the patch series complete. /Bruno [1] https://github.com/baxeno/ptxdist-repology > > Sincerely > Sven > > > /Bruno > > > > [1] https://repology.org/repository/ptxdist > > [2] > > https://repology.org/projects/?inrepo=ptxdist&outdated=1&families_newest=2- > > [3] https://repology.org/projects/?inrepo=ptxdist&vulnerable=1 > > [4] https://repology.org/repository/ptxdist/problems > > > >> This python script uses the unauthenticated API to get all current > >> versions for the Projects with a mapped PTXdist name. It then caches > >> these versions locally (updating only if the cache is older then 24h) > >> and compares the versions against the versions in the report file. > >> It displays all rules that can be updated and also counts the number of > >> up-to-date rules and rules missing a mapping. With a command line flag > >> the latter two can also be displayed. > >> > >> Note that this isn't perfect and just provides an approximation to the > >> actual truth. There are a number of imperfections: > >> - PTXdist using non-stable versions (e.g. iftop 1.0pre4 instead of 0.17) > >> - Versions with different separators (dashes/underscores vs. dots) > >> - Rules not intended to be tracked (e.g. cross-dummy-strip) > >> > >> Also there are some pitfalls in this design: > >> - Version numbers are based on the BSP > >> - Updated rules might be already present in a newer PTXdist release > >> - Rules might be overwritten with an older version locally > >> - BSP only rules (custom applications) will show up as untracked > >> - Versions may lag behind a bit (Anitya cron job and rate-limits) > >> > >> [1] https://github.com/fedora-infra/anitya > >> [2] https://release-monitoring.org/distro/PTXdist/ > >> > >> Signed-off-by: Sven Püschel <[email protected]> > >> --- > >> scripts/update_check.py | 132 ++++++++++++++++++++++++++++++++++++++++ > >> 1 file changed, 132 insertions(+) > >> create mode 100755 scripts/update_check.py > >> > >> diff --git a/scripts/update_check.py b/scripts/update_check.py > >> new file mode 100755 > >> index 000000000..94ffe83bc > >> --- /dev/null > >> +++ b/scripts/update_check.py > >> @@ -0,0 +1,132 @@ > >> +#!/usr/bin/env python3 > >> +# > >> +# Copyright (C) 2026 by Sven Püschel <[email protected]> > >> +# > >> +# For further information about the PTXdist project and license conditions > >> +# see the README file. > >> +# > >> + > >> +import argparse > >> +import itertools > >> +import json > >> +import os > >> +import time > >> +from urllib import request > >> + > >> +import yaml > >> + > >> +API_ENDPOINT = 'https://release-monitoring.org/api/v2' > >> +ITEMS_PER_PAGE = 25 > >> + > >> +XDG_CACHE_DIR = os.getenv('XDG_CACHE_HOME') > >> +if not XDG_CACHE_DIR: > >> + XDG_CACHE_DIR = os.path.join(os.getenv('HOME'), '.cache') > >> + > >> +VERSION_CACHE_FILENAME = 'upstream_pkg_versions.json' > >> +VERSION_CACHE_EXPIRE_SEC = 24 * 60 * 60 > >> +VERSION_CACHE_DIR = os.path.join(XDG_CACHE_DIR, 'ptxdist') > >> +VERSION_CACHE_FILE = os.path.join(VERSION_CACHE_DIR, > >> VERSION_CACHE_FILENAME) > >> + > >> +def download_pkg_version_list(): > >> + versions = [] > >> + > >> + for page in itertools.count(1): > >> + with request.urlopen(request.Request( > >> + > >> f'{API_ENDPOINT}/packages/?distribution=PTXdist&page={page}&items_per_page={ITEMS_PER_PAGE}', > >> + headers={ 'User-Agent': 'PTXdist update_check script' } > >> + )) as url: > >> + data = json.load(url) > >> + assert data['page'] == page > >> + assert data['items_per_page'] == ITEMS_PER_PAGE > >> + total = data['total_items'] > >> + versions.extend(data['items']) > >> + print(f'\rDownloading current package versions > >> ({len(versions)}/{total})', end='') > >> + if len(versions) >= total: > >> + break > >> + print() > >> + return versions > >> + > >> +parser = > >> argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter) > >> +parser.add_argument('--show-untracked', help='Show rules where no updated > >> version is known', action='store_true') > >> +parser.add_argument('--show-up-to-date', help='Show rules which are > >> already at the newest versions', action='store_true') > >> +parser.add_argument('report_file', help='report file generated by > >> `ptxdist fast-bsp-report`') > >> + > >> +args = parser.parse_args() > >> + > >> +versions = None > >> +try: > >> + if os.stat(VERSION_CACHE_FILE).st_mtime > time.time() - > >> VERSION_CACHE_EXPIRE_SEC: > >> + print('Using cached version list') > >> + with open(VERSION_CACHE_FILE) as f: > >> + versions = json.load(f) > >> +except FileNotFoundError: > >> + # No cache exists, just download it > >> + pass > >> +except Exception as e: > >> + print(f'Unexpected error trying to read the version cache: {e}') > >> + > >> +if not versions: > >> + versions = download_pkg_version_list() > >> + try: > >> + os.makedirs(VERSION_CACHE_DIR, exist_ok=True) > >> + with open(VERSION_CACHE_FILE, 'w') as f: > >> + json.dump(versions, f, indent=1) > >> + f.write("\n") > >> + except Exception as e: > >> + print(f"Couldn't write package version cache: {e}") > >> +print() > >> + > >> +stable_versions = {} > >> +for package in versions: > >> + stable_versions[package['name']] = package['stable_version'] > >> + > >> +with open(args.report_file) as f: > >> + report = yaml.safe_load(f) > >> + > >> +latest_version = [] > >> +needs_update = [] > >> +untracked = [] > >> + > >> +max_pkgname_len = 0 > >> +max_version_len = 0 > >> + > >> +for pkg in report['packages'].values(): > >> + if not 'rulefile' in pkg or not 'version' in pkg: > >> + continue > >> + > >> + max_pkgname_len = max(max_pkgname_len, len(pkg['name'])) > >> + max_version_len = max(max_version_len, len(pkg['version'])) > >> + > >> + version = stable_versions.get(pkg['name']) > >> + if version is None: > >> + version = stable_versions.get(pkg['name'].removeprefix('host-')) > >> + > >> + if version is None: > >> + untracked.append((pkg['name'], pkg['version'])) > >> + else: > >> + if version != pkg['version']: > >> + needs_update.append((pkg['name'], pkg['version'], version)) > >> + else: > >> + latest_version.append((pkg['name'], pkg['version'])) > >> + > >> +if args.show_up_to_date: > >> + print('Up-to-date rules') > >> + print('================') > >> + for pkg in latest_version: > >> + print(f"{pkg[0].ljust(max_pkgname_len)} > >> {pkg[1].rjust(max_version_len)}") > >> + print() > >> + > >> +print('Updateable rules') > >> +print('================') > >> +for pkg in needs_update: > >> + print(f"{pkg[0].ljust(max_pkgname_len)} > >> {pkg[1].rjust(max_version_len)} -> {pkg[2]}") > >> +print() > >> + > >> +if args.show_untracked: > >> + print('Untracked rules') > >> + print('===============') > >> + for pkg in untracked: > >> + print(f"{pkg[0].ljust(max_pkgname_len)} > >> {pkg[1].rjust(max_version_len)}") > >> + print() > >> + > >> +print(f"Updateable: {len(needs_update)} Up-to-date: {len(latest_version)} > >> Untracked: {len(untracked)}") > >> -- > >> 2.47.3 > >> > >>
