Hi all (esp. doko and Colin),
(pls Cc: me when replying, thanks)
(mail resent, now with debian-python ML in Cc:, please reply to this
mail, so discussion ends on the list, thanks)
as part of my work for the UBports Foundation, I am currently working
on getting software components of Ubuntu Touch to Debian. One of those
comment is the src:pkg click (one of Colin's former upstream projects).
We see various test failures recently when running unit tests of click
against Python 3.10. It seems that something regarding file descriptor
inheritance in subprocess calls has changed between Python 3.9 and
Python 3.10 which I fail to pinpoint exactly.
At the end of the mail you find a build log excerpt where you can see
that the unit test script loops over python3.9 and python3 (aka
python3.10 in unstable) when running the unit tests. Whereas tests
under python3.9 succeed (first block of test results), tests under
python3(.10) fail (second block of test results).
The reported errors all resemble this pattern:
```
test_install (click_package.tests.test_install.TestClickInstaller) ...
b'ERROR:root:[\'dpkg\', \'--force-not-root\', \'--force-bad-path\',
\'--force-architecture\', \'--instdir\',
\'/tmp/click9a_gafth/root/test-package/1.0\', \'--admindir\',
\'/tmp/click9a_gafth/root/test-package/1.0/.click\',
\'--path-exclude\', \'*/.click/*\', \'--log\',
\'/tmp/click9a_gafth/root/.click/log\', \'--no-triggers\',
\'--install\', \'/tmp/click9a_gafth/fake-package.click\'] failed with
exit_code 1:\ndpkg-split: error: failed to read archive
\'/tmp/click9a_gafth/fake-package.click\': Bad file descriptor\ndpkg:
error processing archive /tmp/click9a_gafth/fake-package.click
(--install):\n subprocess dpkg-split returned error exit status
2\nErrors were encountered while processing:\n
/tmp/click9a_gafth/fake-package.click\n\nTraceback (most recent call
last):\n File "/<>/click_package/tests/gimock.py", line
497, in run_in_subprocess\nyield partial(helper, lib_path,
rpreloads), preloads\n File
"/<>/click_package/tests/test_install.py", line 476, in
test_install\ninstaller.install(path)\n File
"/<>/click_package/install.py", line 462, in install\n
package_name, package_version, old_version = self._unpack(\n File
"/<>/click_package/install.py", line 416, in _unpack\n
fn(command,\n File "/usr/lib/python3.10/subprocess.py", line 420, in
check_output\nreturn run(*popenargs, stdout=PIPE, timeout=timeout,
check=True,\n File "/usr/lib/python3.10/subprocess.py", line 524, in
run\nraise CalledProcessError(retcode,
process.args,\nsubprocess.CalledProcessError: Command \'[\'dpkg\',
\'--force-not-root\', \'--force-bad-path\', \'--force-architecture\',
\'--instdir\', \'/tmp/click9a_gafth/root/test-package/1.0\',
\'--admindir\', \'/tmp/click9a_gafth/root/test-package/1.0/.click\',
\'--path-exclude\', \'*/.click/*\', \'--log\',
\'/tmp/click9a_gafth/root/.click/log\', \'--no-triggers\',
\'--install\', \'/tmp/click9a_gafth/fake-package.click\']\' returned
non-zero exit status 1.\n'
ERROR
```
The core information in above error output is:
```
dpkg-split: error: failed to read archive
'/tmp/click9a_gafth/fake-package.click': Bad file descriptor
dpkg: error processing archive /tmp/click9a_gafth/fake-package.click
(--install):
subprocess dpkg-split returned error exit status 2
```
I am suspecting some FD handover in click's install.py for being the
point where something fails, but I am unsure. And why does fail in
Python 3.10, but not in 3.9?
https://gitlab.com/ubports/core/click/-/blob/main/click_package/install.py#L397
Also I found a comment by Colin that might be related (but why only
with Python 3.10??):
https://gitlab.com/ubports/core/click/-/blob/main/preload/clickpreload.c#L298
Does anyone have a spontaneous idea what might be causing the observed
test failures under Python 3.10? Maybe someone can point me to a
commit in Python upstream where things in subprocess might have
started to change? Any help is greatly appreciated!!!
Thanks,
Mike
```
make check-local
make[3]: Entering directory '/<>'
set -e; for python in python3.9 python3; do \
$python setup.py test; \
done
/usr/lib/python3/dist-packages/setuptools/dist.py:493: UserWarning:
Normalizing '0.5.0-6' to '0.5.0.post6'
warnings.warn(tmpl.format(**locals()))
running test
WARNING: Testing via this command is deprecated and will be removed in
a future version. Users looking for a generic test entry point
independent of test runner are encouraged to use tox.
running egg_info
creating click.egg-info
writing click.egg-info/PKG-INFO
writing dependency_links to click.egg-info/dependency_links.txt
writing top-level names to click.egg-info/top_level.txt
writing manifest file 'click.egg-info/SOURCES.txt'
reading manifest file 'click.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writin
