Re: python debugging on trunk

2022-09-28 Thread Karl Berry
The appended patch should address both issues.  

Thanks Zack!! I installed the changes (separately).

Note that I have only tested it with Python 2.7 and 3.6.  It _may_
not be correct for Python in the 3.0--3.3 (inclusive) range

I didn't test any of those versions either. Certainly your change is a
vast improvement in any case.

The test suite now passes on my Alma Linux 8 system (I had one unrelated
failure which I could now fix). For the record, this has Python 2.7.18
and 3.9.7. Thanks again! -k

Re: python debugging on trunk

2022-09-26 Thread Zack Weinberg
On Mon, Sep 26, 2022, at 12:23 PM, Karl Berry wrote:
> Is anyone up for debugging some Python-related test failures on
> RHEL-based systems?

I have access to a RHEL7 system, I know Python, and this sounds much
less unpleasant than everything else I'm supposed to be doing today.

> I have a suspicion the problem is that on RHEL systems, "python" invokes
> "python2" (because that's what the Python maintainers recommend, as I
> understand it), but the tests are assuming it invokes "python3".


$ lsb_release -d
Description: Red Hat Enterprise Linux Workstation release 7.9 (Maipo)
$ python --version
Python 2.7.5
$ python3 --version
Python 3.6.8

I have very strong opinions on this topic, having had to deal with a
lot of badly-written Python 2 code that, in the worst case, silently
corrupts data when run under Python 3.  IMNSHO, the bare command name
"python" and the absolute pathname "/usr/bin/python" MUST[rfc2119] be
*permanently reserved* for the Python 2 interpreter.  It's fine for
them not to exist at all, but if they exist and they invoke Python 3,
that is a *bug* in the distribution.

(Yes, there's a PEP that says something different -- IMO it is wrong,
its authors probably hadn't encountered the *really* troublesome
Python 2 code that I have.)

RHEL7, unsurprisingly, ships with Autoconf 2.69.  Before attempting to
build Automake from Git, I built Autoconf 2.71 from the tarball
release, installed it in a subdirectory of my homedir, and made that
directory come first on PATH.  There were no unexpected test failures
in Autoconf's testsuite. Other relevant tool versions include:

$ perl --version
This is perl 5, version 16, subversion 3 (v5.16.3) built for 
(with 44 registered patches, see perl -V for more detail)

$ m4 --version
m4 (GNU M4) 1.4.16

$ /bin/sh --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)

$ make --version
GNU Make 3.82

$ sort --version
sort (GNU coreutils) 8.22

$ gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)

$ libtool --version
libtool (GNU libtool) 2.4.2

I see these Automake testsuite failures:

Testsuite summary for GNU Automake 1.16i

# TOTAL: 2988
# PASS:  2877
# SKIP:  54
# XFAIL: 39
# FAIL:  18
# XPASS: 0
# ERROR: 0

FAIL: t/canon7
FAIL: t/fort5
FAIL: t/instmany-python
FAIL: t/ltinit
FAIL: t/nobase-python
FAIL: t/pr401b
FAIL: t/py-compile-basedir
FAIL: t/py-compile-basic
FAIL: t/py-compile-destdir
FAIL: t/py-compile-option-terminate
FAIL: t/python-pr10995
FAIL: t/python-prefix
FAIL: t/python-vars
FAIL: t/python10
FAIL: t/python12
FAIL: t/python3
FAIL: t/subobj9
FAIL: t/suffix5

That's _most_ of the failures you got, plus some more:

+FAIL: t/canon7
+FAIL: t/fort5
+FAIL: t/ltinit
+FAIL: t/pr401b
+FAIL: t/python10
+FAIL: t/subobj9
+FAIL: t/suffix5
-FAIL: t/subobj-pr13928-more-langs

There are only two failure modes:

canon7, fort5, ltinit, pr401b, subobj9, suffix5

ERROR: files left in build directory after distclean:

I did the build in an AFS volume; these are the AFS equivalent of
the NFS "silly rename" hack (apparently it can happen with SMB as well).
"make distcleancheck" should ignore these.

instmany-python, nobase-python, py-compile-basic, py-compile-destdir,
py-compile-option-terminate, pyghon-pr10995, python-prefix,
python-vars, python10, python12, python3

AttributeError: 'module' object has no attribute 'implementation'

This is coming from this code in lib/py-compile (there are three
copies of it):

if hasattr(sys.implementation, 'cache_tag'):
importlib.util.cache_from_source(filepath), path)
py_compile.compile(filepath, filepath + 'c', path)

sys.implementation was added in Python 3.3 and
importlib.util.cache_from_source was added in 3.4.
Python 2.7's importlib _only_ has 'import_module'.

The appended patch should address both issues.  Note that I have only
tested it with Python 2.7 and 3.6.  It _may_ not be correct for Python
in the 3.0--3.3 (inclusive) range; I cannot conveniently test that.


  * lib/am/ (distcleancheck_listfiles): Filter out "silly rename"
  files, unavoidably created by deleting files that are still open in some
  process on various network file systems.

  * lib/py-compile: Test directly for availability of
  importlib.util.cache_from_source.  Untangle logic for when to generate
  -O and -OO bytecode.  Reformat embedded Python fragments.

diff --git a/lib/am/ b/lib/am/
index 0f591266d..b8d90e572 100644
--- a/lib/am/
+++ b/lib/am/
@@ -568,7 +568,9 @@ distuninstallcheck:
 ## Define distcleancheck_listfiles and distcleancheck separately
 ## from distcheck, so that they can be overridden by the user.
 .PHONY: distcleancheck
-distcleancheck_listfiles = find . -type f -print
+distcleancheck_listfiles = \
+  find . \( -type f -a \! \
+  \( -name .nfs* -o -name .smb* -o -name .__afs* \) \) -print