On Fri, 8 Feb 2019 at 20:46, Min RK <benjami...@gmail.com> wrote: > This was opened as an issue (https://github.com/pypa/pip/issues/4032) some > years ago, but it was more recently recommended to open a thread here and > that issue was closed due to inactivity, so here goes. > > pip currently rewrites sys.executable in the shebang line of scripts to be > installed from wheels. This is necessary for portability, envs, etc. On the > far opposite end of the spectrum, conda finds and rewrites the install prefix > in any file anywhere in the package (including binaries). In IPython/Jupyter > (specifically ipykernel), we have a file that we want to install that is a > data_file in `share/jupyter/kernels/python3/kernel.json`. Like scripts, this > file contains a reference to sys.executable, and like scripts, this should be > rewritten at install time, since the wheel doesn't know the right value at > build time. However, the sys.executable rewrite from pip is not configurable > or accessible, so only scripts are allowed to have this modification. It > would be useful to us to be able to opt-in to this modification for > data_files as well. > > Our current choices are: > > 1. disable wheels for our package because we need to know sys.executable to > create these files, and thus must rely on arbitrary code execution to get it > right > 2. use `python` and add special runtime-handling (this is what we do, and it > can be wrong, e.g. when sys.executable at runtime is not actually the > sys.executable used for installation, exactly the problem script shebang > rewrite solves for scripts) > 3. don't install the kernelspec with the package and tell users that proper > installation is a two step process: `pip install ipykernel; ipython kernel > install --sys-prefix`. We used to do this, and it caused lots of problems due > to uninstall/upgrade causing the package and kernelspec to fall out of sync > since part of the package is not managed by the package manager. > > A "works for us" version of this could be to allow specifying that a given > data_file is a script (or entrypoint) and should be treated as one. This > would allow us to place a script next to our json file in a way that it is > found by our kernelspec mechanism. This isn't as general a solution, but it > works for us without needing to generalize sys.executable (or sys.prefix) > handling as much to locations in files other than the shebang, it only needs > generalizing the installation location of scripts.
Unfortunately, the challenge you would face with either of those approaches is that there would then be a large set of package installer versions that will *appear* to install ipykernel correctly, but will in fact miss rewriting the sys.executable reference in the relevant file. If the new semantics are instead specified in a way that forces old installers to fail to install them, that's problematic for a completely different reason: you have to choose between allowing users to continue using old installers and using the new mechanism that you want to use. While it's a pretty gross hack, would it be feasible for ipykernel to change the "python" marker logic to look up its own RECORD file to find a rewritten script file and extract the shebang line from that file? Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/XPBJXXL7LJ2OVZLAJSMPV6KSYVSQQSOI/
