On Fri, Jan 26, 2024 at 12:14 PM Böszörményi Zoltán <[email protected]> wrote:
> 2024. 01. 26. 11:53 keltezéssel, Martin Jansa írta: > > > I copied vercmp_string_op() and all others called by it from > bitbake/lib/bb/utils.py > > into a new python script. > > > > It's not an issue in vercmp_string_op itself, > > I already know this. > Apparently you haven't read my mail fully for the conclusion. > I did and you said that you were testing vercmp_string_op in jack, instead of looking at waf --version. > but the version value it gets from waf --version isn't just the version > itself but also > > SyntaxWarning from waf. > > That is not a problem in itself as the preceding call only collects the > stdout: > > result = subprocess.check_output([python, wafbin, '--version'], > cwd=subsrcdir, > stderr=subprocess.STDOUT) > > SyntaxWarning goes (or should go) to stderr. > Which goes to stdout as configured in this call. After applying https://gitlab.com/ita1024/waf/-/commit/d2060dfd8af4edb5824153ff24e207b39ecd67a2.patch to fix: | DEBUG: Executing shell function do_configure | Traceback (most recent call last): | File "/OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git/./waf", line 166, in <module> | from waflib import Scripting | File "/OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git/waflib/Scripting.py", line 10, in <module> | from waflib import Utils, Configure, Logs, Options, ConfigSet, Context, Errors, Build, Node | File "/OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git/waflib/Configure.py", line 16, in <module> | from waflib import ConfigSet, Utils, Options, Logs, Context, Build, Errors | File "/OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git/waflib/Options.py", line 14, in <module> | from waflib import Logs, Utils, Context, Errors | File "/OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git/waflib/Context.py", line 9, in <module> | import os, re, imp, sys | ModuleNotFoundError: No module named 'imp' martin@jama /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git $ git diff diff --git a/waflib/Context.py b/waflib/Context.py index 761b521f..38ab03f1 100644 --- a/waflib/Context.py +++ b/waflib/Context.py @@ -6,10 +6,17 @@ Classes and functions enabling the command system """ -import os, re, imp, sys +import os, re, sys from waflib import Utils, Errors, Logs import waflib.Node +if sys.hexversion > 0x3040000: + import types + class imp(object): + new_module = lambda x: types.ModuleType(x) +else: + import imp + # the following 3 constants are updated on each new release (do not touch) HEXVERSION=0x2000c00 """Constant updated on new releases""" The SyntaxWarning is easily reproducible: martin@jama /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git $ python3 waf --version /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git/waflib/Context.py:617: SyntaxWarning: invalid escape sequence '\_' """ /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git/waflib/Build.py:107: SyntaxWarning: invalid escape sequence '\*' """List of targets to build (default: \*)""" /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git/waflib/Task.py:1047: SyntaxWarning: invalid escape sequence '\w' re_cond = re.compile('(?P<var>\w+)|(?P<or>\|)|(?P<and>&)') /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git/waflib/TaskGen.py:730: SyntaxWarning: invalid escape sequence '\w' re_m4 = re.compile('@(\w+)@', re.M) waf 2.0.12 (54841218840ffa34fddf834680a5a17db69caa12) martin@jama /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/jack/1.9.22/git $ python3 --version Python 3.12.1 When this part is moved into a minimal test case, there is no error at all. > > The error only happens inside bitbake and there was no such problem > with Python 3.11.x or earlier. > > Ergo, the problem must be a Python 3.12 issue. > The actual version here is python3-3.12.1-2.fc39.x86_64 for me. > > > So you need to update waf in jack not to trigger SyntaxWarning when waf > --version is called. > > > > On Fri, Jan 26, 2024 at 10:17 AM Zoltan Boszormenyi <[email protected]> > wrote: > > > > Hi, > > > > Just a FYI. > > > > After the bitbake-server getting stuck after bitbake exited > > with Python 3.12 issue was fixed, here's another one. > > > > This time it's semi-reliably triggered by the jack recipe > > in meta-openembedded: > > > > ERROR: jack-1.19.22-r0 do_configure: Error executing a python > function in > > exec_func_python() autogenerated: > > > > The stack trace of python calls that resulted in this > exception/failure was: > > File: 'exec_func_python() autogenerated', lineno: 2, function: > <module> > > 0001: > > *** 0002:waf_preconfigure(d) > > 0003: > > File: > > > > '/home/zozo/test-yocto-4.3-gh/conf/../layers/openembedded-core/meta/classes-recipe/waf.bbclass', > > > > lineno: 58, function: waf_preconfigure > > 0054: wafbin = os.path.join(subsrcdir, 'waf') > > 0055: try: > > 0056: result = subprocess.check_output([python, wafbin, > '--version'], > > cwd=subsrcdir, stderr=subprocess.STDOUT) > > 0057: version = result.decode('utf-8').split()[1] > > *** 0058: if bb.utils.vercmp_string_op(version, "1.8.7", > ">="): > > 0059: d.setVar("WAF_EXTRA_CONF", "--bindir=${bindir} > > --libdir=${libdir}") > > 0060: except subprocess.CalledProcessError as e: > > 0061: bb.warn("Unable to execute waf --version, exit > code %d. Assuming waf > > version without bindir/libdir support." % e.returncode) > > 0062: except FileNotFoundError: > > File: '/home/zozo/test-yocto-4.3-gh/bitbake/lib/bb/utils.py', > lineno: 150, function: > > vercmp_string_op > > 0146: Compare two versions and check if the specified > comparison operator > > matches > > the result of the comparison. > > 0147: This function is fairly liberal about what operators > it will accept > > since > > there are a variety of styles > > 0148: depending on the context. > > 0149: """ > > *** 0150: res = vercmp_string(a, b) > > 0151: if op in ('=', '=='): > > 0152: return res == 0 > > 0153: elif op == '<=': > > 0154: return res <= 0 > > File: '/home/zozo/test-yocto-4.3-gh/bitbake/lib/bb/utils.py', > lineno: 140, function: > > vercmp_string > > 0136: return r > > 0137: > > 0138:def vercmp_string(a, b): > > 0139: """ Split version strings and compare them """ > > *** 0140: ta = split_version(a) > > 0141: tb = split_version(b) > > 0142: return vercmp(ta, tb) > > 0143: > > 0144:def vercmp_string_op(a, b, op): > > File: '/home/zozo/test-yocto-4.3-gh/bitbake/lib/bb/utils.py', > lineno: 91, function: > > split_version > > 0087: """Split a version string into its constituent parts > (PE, PV, PR)""" > > 0088: s = s.strip(" <>=") > > 0089: e = 0 > > 0090: if s.count(':'): > > *** 0091: e = int(s.split(":")[0]) > > 0092: s = s.split(":")[1] > > 0093: r = "" > > 0094: if s.count('-'): > > 0095: r = s.rsplit("-", 1)[1] > > Exception: ValueError: invalid literal for int() with base 10: > 'SyntaxWarning' > > > > ERROR: Logfile of failure stored in: > > > > /home/zozo/test-yocto-4.3-gh/tmp-sicom-glibc/work/corei7-64-oe-linux/jack/1.19.22/temp/log.do_configure.1709417 > > > > "Semi-reliably", i.e. about 50-50 or 60-40 percent failure vs > success rate. > > Usually the second run succeeds after a failure. > > > > I copied vercmp_string_op() and all others called by it > > from bitbake/lib/bb/utils.py into a new python script as a > > minimum test case. Running this script inside a locally > > cloned jack git repository has 100% success rate. > > > > It looks like an internal Python "flake" issue. > > > > Best regards, > > Zoltán Böszörményi > > > > > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#194354): https://lists.openembedded.org/g/openembedded-core/message/194354 Mute This Topic: https://lists.openembedded.org/mt/103973063/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
