Re: Python module packages that don't bytecompile on installation?
Paul Wise writes: > On Sat, Nov 2, 2019 at 6:04 PM Matthias Klose wrote: >> I'd say, there are currently more pressing issues than that, like the Python2 >> removal, or the introduction of Python 3.8. 3.8 also offers a central >> directory >> for bc files, so that's maybe another thing to look at, but not a priority >> now >> from my point of view. > > Agreed re Python 2 etc. Eventually switching to using a central > directory in /var would be nice, right now the .pyc files are dumped > cruftily into /usr subdirs. But /var is for variable state data, i.e. data that is modified during regular system operation. This isn't the case for *.pyc which follow the same rules as the *.py included directly in the packages. So these files should stay in /usr. (I even think /var/lib/dpkg/info should really be in /usr for similar reasons.) See also http://0pointer.net/blog/projects/stateless.html for some other reasons why not using /var might be an advantage. Ansgar
Re: Python module packages that don't bytecompile on installation?
On 11/3/19 10:27 AM, Matthias Klose wrote: > On 03.11.19 02:20, Paul Wise wrote: >> On Sat, Nov 2, 2019 at 6:04 PM Matthias Klose wrote: >>> Python2 >>> removal, or the introduction of Python 3.8. 3.8 also offers a >>> central directory >>> for bc files, so that's maybe another thing to look at, but not a >>> priority now >>> from my point of view. >> >> Agreed re Python 2 etc. Eventually switching to using a central >> directory in /var would be nice, right now the .pyc files are dumped >> cruftily into /usr subdirs. > > yes, but that's something when 3.7 is removed. > > Matthias Maybe that's a newbie question, sorry for this, but... What happens when we upgrade to a minor Python 3 version? Do we get everything recompiled? Is there a userland command to rebuild everything? Cheers, Thomas Goirand (zigo)
Re: Python module packages that don't bytecompile on installation?
On 03.11.19 02:20, Paul Wise wrote: On Sat, Nov 2, 2019 at 6:04 PM Matthias Klose wrote: At this point I'd ignore any Python2 related package, and concentrate on Python3 stuff only. Yes, I was referring only to python3-* module packages. Byte compilation is an optimization, speeding up a program start if byte-compiled files are ready. Packages are smaller without bc, take a bit longer to install. I think nobody is asking the question if .py files should be dropped in favor of .pyc files. As I understand it, the .pyc files are interpreter version specific and doing a transition for each major Python update was deemed to be too much work, so instead we offload building the .pyc files onto user's systems. yes, that's another reason. Not having to touch binary indep packages during a transition. I'd say, there are currently more pressing issues than that, like the Python2 removal, or the introduction of Python 3.8. 3.8 also offers a central directory for bc files, so that's maybe another thing to look at, but not a priority now from my point of view. Agreed re Python 2 etc. Eventually switching to using a central directory in /var would be nice, right now the .pyc files are dumped cruftily into /usr subdirs. yes, but that's something when 3.7 is removed. Matthias
Re: Python module packages that don't bytecompile on installation?
On Sat, Nov 2, 2019 at 6:04 PM Matthias Klose wrote: > At this point I'd ignore any Python2 related package, and concentrate on > Python3 > stuff only. Yes, I was referring only to python3-* module packages. > Byte compilation is an optimization, speeding up a program start if > byte-compiled files are ready. Packages are smaller without bc, take a bit > longer to install. I think nobody is asking the question if .py files should > be > dropped in favor of .pyc files. As I understand it, the .pyc files are interpreter version specific and doing a transition for each major Python update was deemed to be too much work, so instead we offload building the .pyc files onto user's systems. > I'd say, there are currently more pressing issues than that, like the Python2 > removal, or the introduction of Python 3.8. 3.8 also offers a central > directory > for bc files, so that's maybe another thing to look at, but not a priority now > from my point of view. Agreed re Python 2 etc. Eventually switching to using a central directory in /var would be nice, right now the .pyc files are dumped cruftily into /usr subdirs. -- bye, pabs https://wiki.debian.org/PaulWise https://bonedaddy.net/pabs3/
Re: Python module packages that don't bytecompile on installation?
On 02.11.19 04:22, Paul Wise wrote: Hi all, I run adequate on my system, which means I notice when Python module packages don't bytecompile when they are installed. So far I've just been ignoring the warnings that adequate prints. https://salsa.debian.org/debian/adequate In addition I noticed: * some Python modules on my system weren't bytecompiled but did bytecompile when I reinstalled them (blinker for example) * many of the Python modules on my system don't bytecompile their test directories (django for example) * relatively few packages I have installed don't bytecompile when I ignore test paths (lib2to3, tk, uno, distutils) Some questions: Should all module packages bytecompile? At this point I'd ignore any Python2 related package, and concentrate on Python3 stuff only. Should all module packages bytecompile all their directories? What are the downsides when module packages don't bytecompile? What are the downsides when module packages do bytecompile? Byte compilation is an optimization, speeding up a program start if byte-compiled files are ready. Packages are smaller without bc, take a bit longer to install. I think nobody is asking the question if .py files should be dropped in favor of .pyc files. Do we need a lintian complaint about this issue? Do we need any improvements in how module packages bytecompile? For example using triggers instead of postinst scriptlets. Any other thoughts? I'd say, there are currently more pressing issues than that, like the Python2 removal, or the introduction of Python 3.8. 3.8 also offers a central directory for bc files, so that's maybe another thing to look at, but not a priority now from my point of view. Matthias
