On Mon, Sep 26, 2022, at 12:23 PM, Karl Berry wrote:
> Is anyone up for debugging some Python-related test failures on
> RHEL-based systems?
I have access to a RHEL7 system, I know Python, and this sounds much
less unpleasant than everything else I'm supposed to be doing today.
> I have a suspicion the problem is that on RHEL systems, "python" invokes
> "python2" (because that's what the Python maintainers recommend, as I
> understand it), but the tests are assuming it invokes "python3".
Indeed:
$ lsb_release -d
Description: Red Hat Enterprise Linux Workstation release 7.9 (Maipo)
$ python --version
Python 2.7.5
$ python3 --version
Python 3.6.8
I have very strong opinions on this topic, having had to deal with a
lot of badly-written Python 2 code that, in the worst case, silently
corrupts data when run under Python 3. IMNSHO, the bare command name
"python" and the absolute pathname "/usr/bin/python" MUST[rfc2119] be
*permanently reserved* for the Python 2 interpreter. It's fine for
them not to exist at all, but if they exist and they invoke Python 3,
that is a *bug* in the distribution.
(Yes, there's a PEP that says something different -- IMO it is wrong,
its authors probably hadn't encountered the *really* troublesome
Python 2 code that I have.)
---
RHEL7, unsurprisingly, ships with Autoconf 2.69. Before attempting to
build Automake from Git, I built Autoconf 2.71 from the tarball
release, installed it in a subdirectory of my homedir, and made that
directory come first on PATH. There were no unexpected test failures
in Autoconf's testsuite. Other relevant tool versions include:
$ perl --version
This is perl 5, version 16, subversion 3 (v5.16.3) built for
x86_64-linux-thread-multi
(with 44 registered patches, see perl -V for more detail)
$ m4 --version
m4 (GNU M4) 1.4.16
$ /bin/sh --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
$ make --version
GNU Make 3.82
$ sort --version
sort (GNU coreutils) 8.22
$ gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
$ libtool --version
libtool (GNU libtool) 2.4.2
I see these Automake testsuite failures:
Testsuite summary for GNU Automake 1.16i
========================================
# TOTAL: 2988
# PASS: 2877
# SKIP: 54
# XFAIL: 39
# FAIL: 18
# XPASS: 0
# ERROR: 0
FAIL: t/canon7
FAIL: t/fort5
FAIL: t/instmany-python
FAIL: t/ltinit
FAIL: t/nobase-python
FAIL: t/pr401b
FAIL: t/py-compile-basedir
FAIL: t/py-compile-basic
FAIL: t/py-compile-destdir
FAIL: t/py-compile-option-terminate
FAIL: t/python-pr10995
FAIL: t/python-prefix
FAIL: t/python-vars
FAIL: t/python10
FAIL: t/python12
FAIL: t/python3
FAIL: t/subobj9
FAIL: t/suffix5
That's _most_ of the failures you got, plus some more:
+FAIL: t/canon7
+FAIL: t/fort5
+FAIL: t/ltinit
+FAIL: t/pr401b
+FAIL: t/python10
+FAIL: t/subobj9
+FAIL: t/suffix5
-FAIL: t/subobj-pr13928-more-langs
There are only two failure modes:
canon7, fort5, ltinit, pr401b, subobj9, suffix5
ERROR: files left in build directory after distclean:
./.__afsxxxx
I did the build in an AFS volume; these are the AFS equivalent of
the NFS "silly rename" hack (apparently it can happen with SMB as well).
"make distcleancheck" should ignore these.
instmany-python, nobase-python, py-compile-basic, py-compile-destdir,
py-compile-option-terminate, pyghon-pr10995, python-prefix,
python-vars, python10, python12, python3
AttributeError: 'module' object has no attribute 'implementation'
This is coming from this code in lib/py-compile (there are three
copies of it):
if hasattr(sys.implementation, 'cache_tag'):
py_compile.compile(filepath,
importlib.util.cache_from_source(filepath), path)
else:
py_compile.compile(filepath, filepath + 'c', path)
sys.implementation was added in Python 3.3 and
importlib.util.cache_from_source was added in 3.4.
Python 2.7's importlib _only_ has 'import_module'.
The appended patch should address both issues. Note that I have only
tested it with Python 2.7 and 3.6. It _may_ not be correct for Python
in the 3.0--3.3 (inclusive) range; I cannot conveniently test that.
zw
* lib/am/distdir.am (distcleancheck_listfiles): Filter out "silly rename"
files, unavoidably created by deleting files that are still open in some
process on various network file systems.
* lib/py-compile: Test directly for availability of
importlib.util.cache_from_source. Untangle logic for when to generate
-O and -OO bytecode. Reformat embedded Python fragments.
diff --git a/lib/am/distdir.am b/lib/am/distdir.am
index 0f591266d..b8d90e572 100644
--- a/lib/am/distdir.am
+++ b/lib/am/distdir.am
@@ -568,7 +568,9 @@ distuninstallcheck:
## Define distcleancheck_listfiles and distcleancheck separately
## from distcheck, so that they can be overridden by the user.
.PHONY: distcleancheck
-distcleancheck_listfiles = find . -type f -print
+distcleancheck_listfiles = \
+ find . \( -type f -a \! \
+ \( -name .nfs* -o -name .smb* -o -name .__afs* \) \) -print
distcleancheck: distclean
@if test '$(srcdir)' = . ; then \
echo "ERROR: distcleancheck can only run from a VPATH build" ; \
diff --git a/lib/py-compile b/lib/py-compile
index 24a8a1a54..de0f02e32 100755
--- a/lib/py-compile
+++ b/lib/py-compile
@@ -1,7 +1,7 @@
#!/bin/sh
# py-compile - Compile a Python program
-scriptversion=2022-02-06.07; # UTC
+scriptversion=2022-09-26.22; # UTC
# Copyright (C) 2000-2022 Free Software Foundation, Inc.
@@ -141,88 +141,83 @@ python_minor=`$PYTHON -c 'import sys;
print(sys.version_info[1])'`
$PYTHON -c "
import sys, os, py_compile, importlib
+# importlib.util.cache_from_source was added in 3.4
+if (
+ hasattr(importlib, 'util')
+ and hasattr(importlib.util, 'cache_from_source')
+):
+ destpath = importlib.util.cache_from_source
+else:
+ destpath = lambda filepath: filepath + 'c'
+
sys.stdout.write('Byte-compiling python modules...\n')
for file in sys.argv[1:]:
$pathtrans
$filetrans
- if not os.path.exists(filepath) or not (len(filepath) >= 3
- and filepath[-3:] == '.py'):
- continue
+ if (
+ not os.path.exists(filepath)
+ or not (len(filepath) >= 3 and filepath[-3:] == '.py')
+ ):
+ continue
sys.stdout.write(file + ' ')
sys.stdout.flush()
- if hasattr(sys.implementation, 'cache_tag'):
- py_compile.compile(filepath,
importlib.util.cache_from_source(filepath), path)
- else:
- py_compile.compile(filepath, filepath + 'c', path)
+ py_compile.compile(filepath, destpath(filepath), path)
sys.stdout.write('\n')" "$@" || exit $?
# Then byte compile w/optimization all the modules.
-case $python_major.$python_minor in
-2.*)
- $PYTHON -O -c "
+$PYTHON -O -c "
import sys, os, py_compile, importlib
-# pypy does not use .pyo optimization
-if hasattr(sys, 'pypy_translation_info'):
+# importlib.util.cache_from_source was added in 3.4
+if (
+ hasattr(importlib, 'util')
+ and hasattr(importlib.util, 'cache_from_source')
+):
+ destpath = importlib.util.cache_from_source
+else:
+ destpath = lambda filepath: filepath + 'o'
+
+# pypy2 does not use .pyo optimization
+if sys.version_info.major <= 2 and hasattr(sys, 'pypy_translation_info'):
sys.exit(0)
sys.stdout.write('Byte-compiling python modules (optimized versions) ...\n')
for file in sys.argv[1:]:
$pathtrans
$filetrans
- if not os.path.exists(filepath) or not (len(filepath) >= 3
- and filepath[-3:] == '.py'):
- continue
+ if (
+ not os.path.exists(filepath)
+ or not (len(filepath) >= 3 and filepath[-3:] == '.py')
+ ):
+ continue
sys.stdout.write(file + ' ')
sys.stdout.flush()
- if hasattr(sys.implementation, 'cache_tag'):
- py_compile.compile(filepath,
importlib.util.cache_from_source(filepath), path)
- else:
- py_compile.compile(filepath, filepath + 'o', path)
+ py_compile.compile(filepath, destpath(filepath), path)
sys.stdout.write('\n')" "$@" 2>/dev/null || exit $?
- ;;
-*) # Python 3+
- $PYTHON -O -c "
-import sys, os, py_compile, importlib
-
-print('Byte-compiling python modules (optimized versions) ...')
-for file in sys.argv[1:]:
- $pathtrans
- $filetrans
- if not os.path.exists(filepath) or not (len(filepath) >= 3
- and filepath[-3:] == '.py'):
- continue
- print(file, end=' ', flush=True)
- if hasattr(sys.implementation, 'cache_tag'):
- py_compile.compile(filepath,
importlib.util.cache_from_source(filepath), path)
- else:
- py_compile.compile(filepath, filepath + 'o', path)
-print()" "$@" 2>/dev/null || exit $?
- ;;
-esac
# Then byte compile w/more optimization.
+# Only do this for Python 3.5+, see https://bugs.gnu.org/38043 for background.
case $python_major.$python_minor in
2.*|3.[0-4])
;;
-*) # Python 3.5+
- # See https://bugs.gnu.org/38043 for background.
+*)
$PYTHON -OO -c "
-import sys, os, py_compile, imp
+import sys, os, py_compile, importlib
-print('Byte-compiling python modules (-OO version) ...')
+sys.stdout.write('Byte-compiling python modules (more optimized versions)'
+ ' ...\n')
for file in sys.argv[1:]:
$pathtrans
$filetrans
- if not os.path.exists(filepath) or not (len(filepath) >= 3
- and filepath[-3:] == '.py'):
+ if (
+ not os.path.exists(filepath)
+ or not (len(filepath) >= 3 and filepath[-3:] == '.py')
+ ):
continue
- print(file, end=' ', flush=True)
- if hasattr(imp, 'get_tag'):
- py_compile.compile(filepath, imp.cache_from_source(filepath), path)
- else:
- py_compile.compile(filepath, filepath + 'o', path)
-print()" "$@" 2>/dev/null || exit $?
+ sys.stdout.write(file + ' ')
+ sys.stdout.flush()
+ py_compile.compile(filepath, importlib.util.cache_from_source(filepath),
path)
+sys.stdout.write('\n')" "$@" 2>/dev/null || exit $?
;;
esac
--end--
