Does this proposal mean that .pyc files would need to be regenerated after installing from a wheel (because the timestamp inside the .pyc file would no longer match the .py file from which it came)? I could be wrong, but it seems like it’s a desirable feature that .pyc files be generated and up to date after installation.
I notice in your problem statement, you only describe that it is a problem for .h and .pxd files not to get new timestamps. —Chris On Mon, Aug 6, 2018 at 3:55 AM Jeroen Demeyer <j.deme...@ugent.be> wrote: > Proposal: change timestamps of files installed from a wheel to the time > of installation, except for .py files. > > **Existing situation** > > Let's consider two ways of installing a Python package: installation > from a source tarball and installation from a wheel, in both cases using > "pip install". > > The installation roughly works as follows: > > (A) Create a temporary directory for pip to work in, with a filename > like /tmp/pip-6N_2Kx-build > > (B) Extract the archive (sources or wheel) somewhere in the temporary > directory > > (C) Build the package in the temporary directory (in both cases, this > involves creating .pyc files for the .py files) > > (D) Copy the installation from the temporary directory to the final > installation directory (typically somewhere in site-packages) > > (E) Delete the temporary directory > > Both (B) and (D) involve a copy operation of some sort, so the question > is: what to do with timestamps? In both cases, (D) currently preserves > the timestamp of the files. For (B), timestamps are preserved when > extracting a tarball but not when extracting a wheel. > > **Proposal** > > Change step (D) for wheel installs to NOT preserve timestamps. It might > be safer to make sure that all installed files have the same timestamp, > so this would mean: give all installed files the timestamp of a fixed > time, namely the time when step (D) was started in the installation > process. > > Important exception: for .py files, do preserve the timestamp. > > I think that (B) should preserve timestamps. However, that is a > different issue which should be fixed independently. > > **The problem** > > For a source installation, the timestamps are preserved in both (B) and > (D), so the timestamp of the installed files can be much older than the > time of installation. > > This is a problem for dependency checking: suppose that I am the > developer of a package MYPKG which has a dependency on the C API of a > package OTHERPKG. The C API of OTHERPKG is defined by .h files which are > installed by OTHERPKG. The package MYPKG correctly declares this > dependency using the "depends" keyword of the Extension objects. Now, > when I install a different version of OTHERPKG, I want that distutils > notices that the dependency has been changed and automatically rebuilds > the Extensions with that dependency. > > This requires that the timestamp of the installed .h files is more > recent than the timestamp of the compiled extension. In other words, the > timestamp of installed files should be the time of installation. > > A similar situation happens for a Cython API exposed by installed .pxd > (and possibly also .h) files. > > **Comparison with automake** > > Automake (which is probably the most widely used build system for open > source/free software) installs files with timestamp the time of > installation. So, my proposal is doing what automake already does. > > **The strange case of .pyc files** > > As noted in https://github.com/pypa/pip/issues/5648 it is important that > the timestamp of .py files are preserved after the .pyc file is created. > This is because Python checks the exact timestamp of the .py file to > determine whether it is up to date. > > This explains the exception for .py files in the proposal. > > **Backwards compatibility** > > In other words: won't this break packages which rely on timestamps to be > preserved? > > When installing a wheel, timestamps from the wheel are already not > preserved in step (B). So any package which would rely on preserving > timestamps would already be broken when installed as wheel. So this > proposal can only break stuff which depends on timestamps from step (C), > which happens to be the case for .py files. > > In my proposal, I'm only suggesting to make changes for wheel installs. > I do think that the change makes sense for installs from source, but > that will be irrelevant after PEP 517 is implemented anyway. > > > Jeroen. > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/SOBBI2YA7TVEUI3EVXJX2Z4EDJOU4WYY/ >
-- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/FNNHIJX7PEW4P7LSBWRQRDJ3HM2JLSPC/
