On 16Jul2016 1254, Paul Moore wrote:
On 15 July 2016 at 23:20, Steve Dower <steve.do...@python.org> wrote:
Hi all
I'd like to get this PEP approved (status changed to Active, IIUC).
Some comments below.
Awesome, thanks! Posted a pull request at
https://github.com/python/peps/pull/59 for ease of diff reading, and
some commentary below (with aggressive snipping).
Motivation
==========
When installed on Windows, the official Python installer creates a registry
key for discovery and detection by other applications. This allows tools such
as installers or IDEs to automatically detect and display a user's Python
installations.
The PEP seems quite strongly focused on GUI tools ... I'd like to avoid tool
developers reading this section and thinking "it only applies to GUI tools or
OS integration, not to me".
Agreed. I tried to avoid any console/GUI-specific terms, but I can
probably be more explicit about it being useful to both.
For example, virtualenv introspects the available Python installations
- see https://github.com/pypa/virtualenv/blob/master/virtualenv.py#L86
- to support the "-p <interpreter>" flag. To handle this well, it
would be useful to allow distributions to register a "short tag", so
that as well as "-p 3.5" or "-p 2", Virtualenv could support (say) "-p
conda3.4" or "-p pypy2". (The short tag should be at the Company
level, so "conda" or "pypy", and the version gets added to that).
Another place where this might be useful is the py.exe launcher (it's
not in scope for this PEP, but having the data needed to allow the
launcher to invoke any available installation could be useful for
future enhancements).
virtualenv would be a great example to use. My thinking was that the Tag
should be appropriate here (perhaps with the Company to disambiguate
when necessary), and that is now explicit.
Anaconda currently has "Anaconda_4.1.1_64-bit" as their tag, which would
not be convenient, so an explicit suggestion here would help ensure this
is useful.
Another key motivation for me would be to define clearly what
information tools can rely on being able to get from the available
registry entries describing what's installed. Whenever I've needed to
scan the registry, the things I've needed to find out are where I find
the Python interpreter, what Python version it is, and whether it's
32-bit or 64-bit. The first so that I can run Python, and the latter
two so that I can tell if this is a version I support *without*
needing to run the interpreter. For me, everything else in this PEP is
about UI, but those 3 items plus the "short tag" idea are more about
what capabilities I can provide.
Good points. I discussed architecture with a colleague at one point and
I'm not entirely sure it's universally useful (what architecture is
IronPython when built for Any CPU? what architecture is Jython?), but
maybe something like the contents of importlib.machinery.IMPORT_SUFFIXES
would be?
On 64-bit Windows, ``HKEY_LOCAL_MACHINE\Software\Wow6432Node`` is a special
key that 32-bit processes transparently read and write to rather than
accessing the ``Software`` key directly.
It might be worth being more explicit here that 32-bit and 64-bit
processes see the registry keys slightly differently. More on this
below.
I considered this and thought I had a link to the official docs about
it. I don't want this PEP to be mistaken for documentation on how
registry redirection works :)
Backwards Compatibility
-----------------------
Also, Python 3.5 doesn't appear to include the architecture in
sys.winver either.
...
(Unless it adds -32 for 32-bit, and reserves the bare version for
64-bit. I've skimmed the CPython source but can't confirm that). The
documentation of sys.winver makes no mention of whether it
distinguishes 32- and 64-bit builds. In fact, it states "The value is
normally the first three characters of version". If we're relying on
sys.winver being unique by version/architecture, the docs need to say
so (so that future changes don't accidentally violate that).
I'll update the docs, but your guess is correct. I changed sys.winver on
32-bit to be "3.5-32" since that matches what py.exe was already using
to refer to it. I didn't want to invent
yet-another-way-to-tag-architectures. (I also updated py.exe to match
tags directly under PythonCore, so 3.5-32 is matched without scanning
the binary type.)
Also, sys.winver is defined in PCBuild/python.props, which is how we
accidentally backported the suffix to 2.7.11 :(
It is not possible to detect side-by-side installations of both 64-bit and
32-bit versions of Python prior to 3.5 when they have been installed for the
current user. Python 3.5 and later always uses different Tags for 64-bit and
32-bit versions.
From what I can see, this latter isn't true. I presume that 64-bit
uses no suffix, but 32-bit uses a "-32" suffix? This should probably
be made explicit. At a minimum, if I were writing a tool to list all
installed Python versions, with only what I have available to go on
(the PEP and a 64-bit Python 3.5) I wouldn't be able to write correct
code, as I don't have all the information I need.
Also, if we expect to be able to distinguish 32 and 64 bit
implementations in this way, that's putting a new restriction on
sys.winver, that it returns a different value for 32-bit and 64-bit
builds. If that's the case, I'd rather see that explicitly documented,
both here and in the sys.winver documentation.
I'd actually prefer a more explicit mechanism going forward, but as
this is a "backward compatibility" section I'll save that for later.
I don't want to lock in the actual scheme of the tags used by CPython.
Granted, without SysArchitecture in the key you can't identify the
architecture from the information you have, but Tags are supposed to be
treated as opaque with the exception of those that were released prior
to 3.5 (also not explicit, so I'll fix that).
The Company part of the key is intended to group related environments and to
ensure that Tags are namespaced appropriately. The key name should be
alphanumeric without spaces and likely to be unique. For example, a
trademarked
name, a UUID, or a hostname would be appropriate::
HKEY_CURRENT_USER\Software\Python\ExampleCorp
HKEY_CURRENT_USER\Software\Python\6C465E66-5A8C-4942-9E6A-D29159480C60
HKEY_CURRENT_USER\Software\Python\www.example.com
I'd suggest adding "Human-readable Company values are preferred".
UUIDs seem like a horrible idea in practice.
Maybe, but name-squatting seems equally bad (and I've seen enough
collaborative registration systems like this to see the value in
guaranteed uniqueness). I'll recommend trademarked names though.
It's also worth noting that "Display Name" isn't actually as useful as
it sounds, in practice. A tool that relies on it would report the
python.org installers as being provided by "PythonCore", which isn't
particularly user friendly. Maybe we need something in the "Backward
Compatibility" section going into a bit more detail as to how tools
should deal with that, and maybe we need to add a "DisplayName" in
3.6+.
I'll specify defaults for PythonCore on each of them and add them to 3.6
(once the PEP is approved and we agree on the values). It certainly
doesn't harm the usefulness, but we do want to make sure that tools are
handling old PythonCore entries in a consistent way.
If a string value named ``DisplayName`` exists, it should be used to
identify the environment to users. Otherwise, the name of the key should be
used.
To an extent there's the same comment here as for DisplayName for
Company - it needs to be defined with consideration for how it will be
used. This is, of course, more of a "quality of implementation" matter
than a standards one. But the PEP might benefit from an example of
use, maybe showing the output from a hypothetical command line tool
that lists all installations on the machine.
It's defined as being used to "identify ... to users". Equally, the
SupportUrl "may be displayed or otherwise used to direct users to
[support]". I feel that these are strong enough definitions, and that
showing an hypothetical command line tool output might be seen as too
prescriptive (or alternatively, swinging the pendulum too far away from
GUI tools).
If a string value named ``Version`` exists, it should be used to identify the
version of the environment. This is independent from the version of Python
implemented by the environment.
If a string value named ``SysVersion`` exists, it must be in ``x.y`` or
``x.y.z`` format matching the version returned by ``sys.version_info`` in the
interpreter. Otherwise, if the Tag matches this format it is used. If not,
the Python version is unknown.
I'm not too happy with this. [...]
Note that each of these values is recommended, but optional.
SysVersion and SysArchitecture (or a Tag that works as a fallback)
should be mandatory. Otherwise I'm OK with this statement.
Snipped most of the details because I agree it's unsatisfying right now,
but I disagree with enough of the counterproposal that it was getting to
be messy commenting on each bit.
Basically, I added SysArchitecture (to match platform.architecture()[0],
typically '32bit' or '64bit' but extensible without having to define all
potential values in the PEP) with a note that for PythonCore it should
be inferred from the registry path.
SysVersion no longer allows inferring it from the Tag, except for
PythonCore and only when SysVersion is missing.
I'm very keen to not force any of this information to be required as it
is very difficult to know how to deal with interpreters that don't
include it. Does virtualenv refuse to list/use it, even if the install
path is valid, just because SysArchitecture was omitted? What if the
registry becomes corrupt - should Visual Studio refuse to show what
information it can obtain?
I already stated that all information is recommended. The change I've
made now is that tools shouldn't work too hard to guess - if SysVersion
is missing, they can simply say they don't know what version the
interpreter is. If knowing the version is critically important, they can
refuse to use it, but for many applications this is not going to be the
case.
And Python 3.6 will specify all of the keys. Adding it to Python 3.5 is
only likely to cause issues now with people who test against 3.5.3 and
not 3.5.2, which didn't have the keys.
Beneath the environment key, an ``InstallPath`` key must be created. This key
is always named ``InstallPath``, and the default value must match
``sys.prefix``::
HKEY_CURRENT_USER\Software\Python\ExampleCorp\3.6\InstallPath
(Default) = "C:\ExampleCorpPy36"
If a string value named ``ExecutablePath`` exists, it must be a path to the
``python.exe`` (or equivalent) executable. Otherwise, the interpreter
executable is assumed to be called ``python.exe`` and exist in the directory
referenced by the default value.
If a string value named ``WindowedExecutablePath`` exists, it must be a path
to the ``pythonw.exe`` (or equivalent) executable. Otherwise, the windowed
interpreter executable is assumed to be called ``pythonw.exe`` and exist in
the directory referenced by the default value.
These two items assume implicitly that a Python installation must
provide python.exe and pythonw.exe. I'm inclined to make this
explicit. Specifically, I think it's crucial that tools can read the
(console or windowed) executable path as described here, and run that
executable with standard Python command line arguments, and expect it
to work. Otherwise there's little point in the installation
registering its existence.
Again, there's a backwards compatibility argument here in that Python
3.4 and earlier did not create full paths to the executables. But that
can be called out separately.
When you say "assume implicitly ... python.exe and pythonw.exe", do you
mean executables by those names (counterexample - IronPython includes
ipy.exe and ipyw.exe, which you either know specially or would discover
from these keys)? Or BOTH executables (e.g. python.exe AND pythonw.exe)?
Or executables with equivalent behaviour?
I'd argue that the whole PEP only applies to Python interpreters, and if
you don't support standard command line arguments you aren't really a
Python interpreter and shouldn't be registering as one. But I hesitate
to try and define a hard rule that captures all the possible nuances
here - I'd rather deal with it by having users file bugs against
offending interpreters for not working correctly.
I can see an argument for a distribution providing just python.exe and
omitting pythonw.exe (or even the other way around). But I can't see
how I could write generic code to work with such a distribution. So
let's disallow that possibility until someone comes up with a concrete
use case
I think in this case, you'd either specify both keys with the same path
(so tools that want the windowed executable are going to get a console
window) or omit the key and make sure you don't have a "pythonw.exe" in
your install directory (which seems unlikely :) ). These keys are mainly
about the possibility of renaming the executables, as shown in the example.
But I've added a note reminding tools developers that the executable may
not exist, and attempting to launch it may fail (i.e. business as usual).
Other Keys
----------
Some other registry keys are used for defining or inferring search paths
under certain conditions. A third-party installation is permitted to define
these keys under their Company-Tag key, however, the interpreter must be
modified and rebuilt in order to read these values. Alternatively, the
interpreter may be modified to not use any registry keys for determining
search paths. Making such changes is a decision for the third party; this PEP
makes no recommendation either way.
I think we need to be clearer here. ...
Great suggestion. I've revised this section.
(I have vague plans to make the PythonPath subkey redundant in more
cases for 3.6, and I *think* we can probably drop the Modules key
completely, but I'm not entirely sure it's a good idea. Still thinking
about it :) )
Cheers,
Steve
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
