Bug#1043543: autopkgtest: /usr/share/autopkgtest/lib/__pycache__ is not managed

2023-08-17 Thread Antonio Terceiro 
On Sat, Aug 12, 2023 at 09:07:44PM +0200, Alexandre Detiste wrote:
> Le sam. 12 août 2023 à 20:16, Paul Gevers  a écrit :
> >
> > Call me innocent, but I'm not even seeing cache files on my system for
> > the recent python. When are these files created?
> >
> > Paul
> 
> I think I just ran autopkgtest once, as root, on this day at 2am...
> I think I should sleep more.
> 
> So they are created anytime a user with write access to
> /usr/share/autopkgtest/lib
> run autopkgtest, which means root.
> 
> dh_python3 would add the calls to py3compile/py3clean,
> but might have other side effects for your package that have to be
> backportable to oldstable(?not sure) so you might consider calling
> py3{compile/clean} directly.

I think the correct fix for this is to turn autopkgtest into a "proper"
Python package, so that we can just use dh-python instead of the custom
build system, and all of this will be taken care of.

I have thought a few times about starting this, but I haven't had the
spoons to actually do it yet.


signature.asc
Description: PGP signature

Bug#1043543: autopkgtest: /usr/share/autopkgtest/lib/__pycache__ is not managed

2023-08-12 Thread Alexandre Detiste 
Le sam. 12 août 2023 à 20:16, Paul Gevers  a écrit :
>
> Call me innocent, but I'm not even seeing cache files on my system for
> the recent python. When are these files created?
>
> Paul

I think I just ran autopkgtest once, as root, on this day at 2am...
I think I should sleep more.

So they are created anytime a user with write access to
/usr/share/autopkgtest/lib
run autopkgtest, which means root.

dh_python3 would add the calls to py3compile/py3clean,
but might have other side effects for your package that have to be
backportable to oldstable(?not sure) so you might consider calling
py3{compile/clean} directly.

Not having the .pyc files at all induce a tiny performance drop.

Another attempt right now:

$ ls -l /usr/share/autopkgtest/lib/__pycache__/*310*.pyc --full-time
-rw-r--r-- 1 root root  4276 2022-11-03 02:34:17.512478073 +0100
/usr/share/autopkgtest/lib/__pycache__/adt_binaries.cpython-310.pyc
-rw-r--r-- 1 root root  2940 2022-11-03 02:34:17.412477486 +0100
/usr/share/autopkgtest/lib/__pycache__/adtlog.cpython-310.pyc
-rw-r--r-- 1 root root 40030 2022-11-03 02:34:17.488477932 +0100
/usr/share/autopkgtest/lib/__pycache__/adt_testbed.cpython-310.pyc
-rw-r--r-- 1 root root 12270 2022-11-03 02:34:17.520478120 +0100
/usr/share/autopkgtest/lib/__pycache__/autopkgtest_args.cpython-310.pyc
-rw-r--r-- 1 root root 17628 2022-11-03 02:34:17.428477580 +0100
/usr/share/autopkgtest/lib/__pycache__/testdesc.cpython-310.pyc
-rw-r--r-- 1 root root 21171 2022-11-03 02:34:17.508478050 +0100
/usr/share/autopkgtest/lib/__pycache__/VirtSubproc.cpython-310.pyc

Original timestamps, that don't match any release date or migration to testing
or the time of day when unattended-upgrades run:

tchet@brix ~/git/cruft-ng $ ls -l
/usr/share/autopkgtest/lib/__pycache__/*311*.pyc --full-time
-rw-r--r-- 1 root root  7748 2023-08-12 20:52:27.131803347 +0200
/usr/share/autopkgtest/lib/__pycache__/adt_binaries.cpython-311.pyc
-rw-r--r-- 1 root root  5248 2023-08-12 20:52:27.067803064 +0200
/usr/share/autopkgtest/lib/__pycache__/adtlog.cpython-311.pyc
-rw-r--r-- 1 root root 79617 2023-08-12 20:52:27.179803560 +0200
/usr/share/autopkgtest/lib/__pycache__/adt_testbed.cpython-311.pyc
-rw-r--r-- 1 root root 21495 2023-08-12 20:52:27.215803720 +0200
/usr/share/autopkgtest/lib/__pycache__/autopkgtest_args.cpython-311.pyc
-rw-r--r-- 1 root root 31364 2023-08-12 20:52:27.087803152 +0200
/usr/share/autopkgtest/lib/__pycache__/testdesc.cpython-311.pyc
-rw-r--r-- 1 root root 41735 2023-08-12 20:52:27.203803667 +0200
/usr/share/autopkgtest/lib/__pycache__/VirtSubproc.cpython-311.pyc

Some minimal package that do the right thing:

$ cat /var/lib/dpkg/info/python3-six.postinst
# Automatically added by dh_python3
if command -v py3compile >/dev/null 2>&1; then
py3compile -p python3-six
fi
# End automatically added section

$ cat /var/lib/dpkg/info/python3-six.prerm
# Automatically added by dh_python3
if command -v py3clean >/dev/null 2>&1; then
py3clean -p python3-six
fi
# End automatically added section

Bug#1043543: autopkgtest: /usr/share/autopkgtest/lib/__pycache__ is not managed

2023-08-12 Thread Paul Gevers 

Hi,

On 12-08-2023 19:42, Alexandre Detiste wrote:

I have got old, stale, python3.10 cache files from autopkgtest on my system.


Call me innocent, but I'm not even seeing cache files on my system for 
the recent python. When are these files created?


Paul

Bug#1043543: autopkgtest: /usr/share/autopkgtest/lib/__pycache__ is not managed

2023-08-12 Thread Alexandre Detiste 
Package: autopkgtest
Version: 5.30
Severity: minor
User: cruft...@packages.debian.org
Usertags: cruft

Hi,

I have got old, stale, python3.10 cache files from autopkgtest on my system.


I think that simply using some modern DebHelper plugin
would ensure that the .pyc files are correctly generated on postinst
and correctly removed in post rm.

To clean-up the past I suggest a
"rm -f /usr/share/autopkgtest/lib/__pycache__/*-cpython-310.pyc"
in preinst/postinst.

Greetings,



$ ls /usr/share/autopkgtest/lib/__pycache__ -l
total 108
-rw-r--r-- 1 root root  4276  3 nov  2022 adt_binaries.cpython-310.pyc
-rw-r--r-- 1 root root  2940  3 nov  2022 adtlog.cpython-310.pyc
-rw-r--r-- 1 root root 40030  3 nov  2022 adt_testbed.cpython-310.pyc
-rw-r--r-- 1 root root 12270  3 nov  2022 autopkgtest_args.cpython-310.pyc
-rw-r--r-- 1 root root 17628  3 nov  2022 testdesc.cpython-310.pyc
-rw-r--r-- 1 root root 21171  3 nov  2022 VirtSubproc.cpython-310.pyc
$ py3versions -s
python3.11



https://www.debian.org/doc/packaging-manuals/python-policy

> 4.7. Modules Byte-Compilation
>
> If a binary package provides any binary-independent modules (foo.py files),
> the corresponding byte-compiled modules (foo.pyc files)
> must not ship in the package.
> Instead, they should be generated in the package’s post-install script,
> and removed in the package’s pre-remove script.


-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (501, 'testing'), (450, 'unstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 6.4.0-1-amd64 (SMP w/2 CPU threads; PREEMPT)
Locale: LANG=fr_BE.UTF-8, LC_CTYPE=fr_BE.UTF-8 (charmap=UTF-8), 
LANGUAGE=fr_BE:fr
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages autopkgtest depends on:
ii  apt-utils   2.6.1
ii  libdpkg-perl1.21.22
ii  mawk1.3.4.20230730-1
ii  procps  2:4.0.3-1
ii  python3 3.11.4-5
ii  python3-debian  0.1.49

Versions of packages autopkgtest recommends:
pn  autodep8  
ii  fakeroot  1.32.1-1

Versions of packages autopkgtest suggests:
pn  docker.io
pn  fakemachine  
pn  lxc  
pn  lxd  
pn  ovmf 
pn  ovmf-ia32
pn  podman   
ii  python3-distro-info  1.5
pn  qemu-efi-aarch64 
pn  qemu-efi-arm 
pn  qemu-system  
pn  qemu-utils   
ii  schroot  1.6.13-3+b2
ii  util-linux   2.39.1-3
pn  vmdb2
pn  zerofree 

-- no debconf information