Package: src:python-loky Version: 3.5.6-1 User: [email protected] Usertags: python3.15 Tags: patch, ftbfs, forky, sid Severity: important
Hi! While rebuilding the python related packages against the Python 3.15rc2 version we found that python-loky fails to build from source [1]. Almost all the resource tracker tests fail, and this can be fixed by applying the upstream resource_tracker fix (5c28e6c) [2]. I've applied the upstream fix in the sandbox [3] to verify that it builds successfully, please consider applying the patch to support the upcoming 3.15 version. Setting the severity to important for now. Once Python 3.15 is released, it will be added to python3-defaults and this bug will become release critical. Happy hacking, [1]: https://debusine.debian.net/debian/r-python-python3.15/artifact/4495806/ [2]: https://github.com/joblib/loky/commit/5c28e6c1f6be265f9337630efc1b8349b0360d94 [3]: https://debusine.debian.net/debian/r-python-python3.15/ -- "Can you imagine what I would do if I could do all I can?" -- Sun Tzu Saludos /\/\ /\ >< `/
From: Loïc Estève <[email protected]> Subject: MNT Vendor CPython resource tracker code for easier maintainability (#472) Co-authored-by: Eric Larson <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> Origin: backport, https://github.com/joblib/loky/commit/5c28e6c1f6be265f9337630efc1b8349b0360d94 Bug: https://github.com/joblib/loky/pull/472 Index: python-loky/loky/backend/resource_tracker.py =================================================================== --- python-loky.orig/loky/backend/resource_tracker.py +++ python-loky/loky/backend/resource_tracker.py @@ -38,6 +38,8 @@ # resources once all processes connected to the resource tracker have exited. +import base64 +import json import os import shutil import sys @@ -91,6 +93,30 @@ if os.name == "posix": VERBOSE = False +def _decode_message(line): + """Decode a message sent by a resource tracker client. + + Python 3.15's multiprocessing.resource_tracker._send() always encodes + messages as base64-JSON instead of the legacy colon-separated format. + """ + if line.startswith(b"{"): + obj = json.loads(line.decode("ascii")) + cmd, rtype = obj["cmd"], obj["rtype"] + name = base64.urlsafe_b64decode(obj.get("base64_name", "")).decode( + "utf-8", "surrogateescape" + ) + else: + splitted = line.strip().decode("ascii").split(":") + # name can potentially contain separator symbols (for + # instance folders on Windows) + cmd, name, rtype = ( + splitted[0], + ":".join(splitted[1:-1]), + splitted[-1], + ) + return cmd, name, rtype + + class ResourceTracker(_ResourceTracker): """Resource tracker with refcounting scheme. @@ -284,14 +310,7 @@ def main(fd, verbose=0): with open(fd, "rb") as f: for line in f: try: - splitted = line.strip().decode("ascii").split(":") - # name can potentially contain separator symbols (for - # instance folders on Windows) - cmd, name, rtype = ( - splitted[0], - ":".join(splitted[1:-1]), - splitted[-1], - ) + cmd, name, rtype = _decode_message(line) if rtype not in _CLEANUP_FUNCS: raise ValueError(
