[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Steve Dower]

Steve Dower  added the comment:

Thanks for the PR! This is a good contribution, that wasn't trivial to do. It 
should be in Python 3.11 alpha 5, so please test it out and make sure it's 
behaving as you expect.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Steve Dower]

Steve Dower  added the comment:


New changeset c47c9e6589eb7a272cfe4d352eb87389eb20ec2f by bneuburg in branch 
'main':
bpo-44934: Add optional feature AppendPath to Windows MSI installer (GH-27889)
https://github.com/python/cpython/commit/c47c9e6589eb7a272cfe4d352eb87389eb20ec2f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Steve Dower]

Steve Dower  added the comment:

Removing environment variables is "known" to be unreliable - another reason I 
don't like modifying them from an installer. It's meant to remove the item from 
the list based on value, so if anything has changed then it's not a huge 
surprise it will fail.

I'm more inclined to think that the "[~]:" prefix might be up to something. I 
forget exactly what it means, but if it's not all resolving to an empty string, 
it might concatenate fine but then mess up the removal?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Bastian Neuburger]

Bastian Neuburger  added the comment:

Update: .NET 3.5 installation worked out fine via VS 2019 installer, I also 
added the VS2017 build tools as a precaution since the VM builds take some time.

Afterwards build.bat still failed during wix steps, complaining about missing 
python.exe in PCBuild/win32, not sure if the 32bit interpreter should be 
fetched/built automagically. Then I tried build.bat -x64, which ran succesfully 
and provided attached installer.

To test it (in administrative shell):
1. git clone https://github.com/bneuburg/python_msi_appendpath
2. cd python_appendpath_installer
3. .\python-3.11.0.7539-amd64.exe /quiet InstallAllUsers=1 AppendPath=1 
Include_test=0

Running this in an administrative shell appended InstallDir and 
Installdir/Scripts to the sytem PATH and created the PATHEXT entries for PY and 
PYC as well. So far so good!

I only noticed the following problem:
The first uninstall also succesfully removed all environment variables, however 
after reinstalling and re-uninstalling, only the PATHEXT were removed, the 
Python entries in PATH were not removed. This was also independent of the order 
of software removal (Python Launcher first or second). 

Thus I had a look at it with orca.exe [1]:
1. Install with appendpath=1
2. Get list of installed msis with: wmic product get /format:csv > software.csv
3. Search for appendpath in software.csv which will return the location of the 
msi somewhere in c:\windows\installer
4. Right click that file and select 'Edit with Orca'

In the environment table it has entries in the following form for a Local 
Machine install (c.f. for attached screenshot):

Environment  | Name| Value  | Component 
PATH_LM  | =-*PATH | [~]:[InstallDirectory] | AppendPath_LM

which according to Microsoft documentation [2] seems to be as intended (add on 
install, remove on uninstall). But it just doesn't, also see Sascha's comment 
above. When looking at Orca I also noticed that I forgot to change the position 
of InstallDirectory in PATH for the CU install from first to last, I updated 
the PR accordingly.

Since you have a Microsoft e-mail address: Any chances you could ask the MSI 
pros there what is going on?
I'll be busy with offline stuff for a few weeks, but if you have any ideas how 
to troubleshoot this, let me know. I found a discussion where the order of the 
prefixes for names was slightly changed [3], i.e. *=- instead of =-* but I 
don't know if this matters at all or how to test it.

Sorry for the text book, things like that really keep me digging.

[1] https://docs.microsoft.com/de-de/windows/win32/msi/orca-exe
[2] https://docs.microsoft.com/en-us/windows/win32/msi/environment-table
[3] https://www.itninja.com/question/add-to-path-envronment-variable

--
Added file: https://bugs.python.org/file50233/orca.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Steve Dower]

Steve Dower  added the comment:

I don't think we upload the artifact from the MSI builds, sorry.

If Visual Studio has an option for the older version of .NET, then yeah, 
that'll do it. But because it was an OS component, you may have to do it 
through the OS dialog in the link I posted.

GitHub will get the CLA update soon enough. It's always a little slower - bpo 
is the canonical source.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Bastian Neuburger]

Bastian Neuburger  added the comment:

Ah ok. Do you know if VS 2019 will do the job at all? I encountered a few 
(unrelated) projects on github that state they will only build with VS 2017.

I will install .NET 3.5 via the VS 2019 installer (should do the trick, right?) 
and see how it turns out. 

Could you attach the msi installer build artifact from the build you kicked off 
so I can test if the appendpath stuff works as expected? Or did you already 
have a look at it?

P.S.: bpo now recognizes that I signed the CLA, github not yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Steve Dower]

Steve Dower  added the comment:

Oh I remember this one. The Wix toolset still requires .NET 3.5, which you have 
to install manually these days. Because it's being launched as a console app, 
the UI never gets triggered.

I thought we had this in the devguide already... but maybe not (or not 
prominently enough). Might be worth trying to add some detection for it - 
hopefully a new version of Wix will move to newer .NET too.

See 
https://docs.microsoft.com/en-us/dotnet/framework/install/dotnet-35-windows-10 
for install instructions. It should work after that. The CI build succeeded, 
but it'll need manual testing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Bastian Neuburger]

Bastian Neuburger  added the comment:

Attached is the full output of:

- msbuild.exe -version
- get_externals.bat
- build.bat

For the first few build tasks there are git related errors, which make sense 
since I ran the build from a downloaded zip release of the source instead of a 
git clone. I assume the first "real" error occurs when building the wix 
targets, just grep for MSB4062, this is when things go downhill.

--
Added file: https://bugs.python.org/file50231/buildout.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Steve Dower]

Steve Dower  added the comment:

I kicked off the build on the PR so we can see what errors it finds. Nothing 
jumps out as obviously wrong in the changes, but it only takes one missing 
reference to a file to break it all.

If you want to share the errors from your test machine, copy-pasting all the 
output into a text file and attaching it here is best.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Bastian Neuburger]

Bastian Neuburger  added the comment:

Steve, I signed the CLA and created the PR, I don't know how long it takes 
until the info that the CLA is signed is propagated to bpo or github.

Regarding failed builds: I will try to build again tomorrow when back at the 
office and can copy the errors, how would you like to receive them?

For building I spun up a MS Windows 10 Edge Test VM, installed Visual Studio 
2019 and then tried to run get_externals.bat; that already failed since the 
nuget config was empty and thus had no upstreams configured. Maybe this problem 
stems from the free test VM. After editing the nuget config, that script worked 
out fine and then I run build.bat, which seemed to be quite busy for at least 
10 minutes, then I left my office. Upon return I saw plenty of errors.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Bastian Neuburger]

Change by Bastian Neuburger :


--
keywords: +patch
pull_requests: +26343
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/27889

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Steve Dower]

Steve Dower  added the comment:

That branch looks like you're on the right track. The build scripts in 
Tools/msi should Just Work, so I'd be interested to see the error you get.

You can also submit it as a PR and the MSI will be built automatically.

One thing I noticed is updating the name of the "Add Python to PATH" 
option, which I'd rather keep as "Add" since we won't have both options 
in the UI. The word "Prepend" is quite obscure jargon, and for the range 
of users we have it isn't fair to expect them to know it (or to break 
all of the guidance out there explaining that checkbox already).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Bastian Neuburger]

Bastian Neuburger  added the comment:

Previous URL got "fixed" by roundup, hopefully this one works:

https://github.com/python/cpython/compare/16b9be4861e007ad483611ba0479feb2b90ea783...bneuburg:issue44934

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Bastian Neuburger]

Bastian Neuburger  added the comment:

Maybe this view better shows what I've actually done:

https://github.com/python/cpython/compare/16b9be4861e007ad483611ba0479feb2b90ea783...bneuburg:issue44934?expand=1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Bastian Neuburger]

Bastian Neuburger  added the comment:

Thanks for the rundown, after digging a little bit into the abyss that is the 
WIX toolset if you've never dealt with it before I already started going down 
that route. 

I pushed it to my fork of cpython:
https://github.com/bneuburg/cpython/commits/issue44934

Since I'm pretty sure that my commits are not correct yet (following the 
instructions in Tools/msi/README.txt regarding fetching dependencies building 
the MSI resulted in a failed build before I even touched a single line of code) 
I abstained from handing in a pull request.

Feedback would be very much appreciated. Of course if you'd want to use that 
stuff and refine it I would be totally fine with it ;)

In case I messed up any formal requirements (i.e. wording of commit messages 
etc.): I will get myself up to speed and fix that, but since I'll be offline 
for a week starting Monday I wanted to get my initial draft out there, so in 
case you have any feedback until Monday I can incorporate it before my brain 
pushes that stuff out...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Steve Dower]

Steve Dower  added the comment:

The logic is probably simplest if you clone the Tools/msi/path project entirely 
as pathappend. That will technically allow people to both prepend and append if 
they set both flags, but I think it's okay to say "don't do that", especially 
if AppendPath is only a command line option.

Per-component or optional features are complicated and best avoided if we can. 
For a simple on-off like this, a whole MSI is actually the simplest way to 
manage it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Bastian Neuburger]

Bastian Neuburger  added the comment:

Hi Steve,

Of course there are various reasons for having multiple Python versions on a 
developer workstation but I would assume that the majority of Windows installer 
users simple want to run Python projects, at least this holds true for the 
majority of Python on Windows users in my org.

> if you'd really like this to be in the installer, feel free to go for it.

could you point me to the relevant files and lines for the PrependPath CLI 
option and implementation logic on Github? Not sure if I got it right, but I 
assume the following file is in charge of modifying the env:path:

https://github.com/python/cpython/blob/main/Tools/msi/path/path.wxs

Since I have never worked within the MSI framework, I could not figure out how 
the "call flow" from enabling the definition of the PrependPath in
https://github.com/python/cpython/blob/main/Tools/msi/bundle/bundle.wxs 
to actually modifying the path

Regarging path.wxs: Would you prefer that I add a pathappend/pathappend.wxs or 
should I simply add a new Component ID like so:


ALLUSERS=1
...

Or would I introduce a new condition ALLUSERS_APPEND or something?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Steve Dower]

Steve Dower  added the comment:

> why is this confusion not written here

>From https://docs.python.org/3/using/windows.html#configuring-python

> While the installer provides an option to configure the PATH and PATHEXT 
> variables for you, this is only reliable for a single, system-wide 
> installation. If you regularly use multiple versions of Python, consider 
> using the Python Launcher for Windows.

Now, that isn't *exactly* the issue in the original post, but the warning is 
present.

> If you do software deployment in a company you will only have one version so 
> having more than one version is not the point.

Unfortunately not true. It's been a while since I had actual data on this, but 
last time I did the *majority* had 2 versions installed, and 3-5 versions were 
all more likely than only a single one. So handling multiple versions matters.

> What i have seen now is if you uninstall Python it is not cleaning up the 
> path variable.

We do the best we can by using the native Windows support for installing 
environment variables. It sometimes doesn't work if you (or another installer) 
has modified the variable since you installed Python, or if you've updated it 
yourself. That's out of our control - we can't do a better job than the OS.

> if you make append path optional available like prepend everybody can choose

Contributions are welcome. We aren't paid to implement features - we volunteer 
to manage contributions, so if you'd really like this to be in the installer, 
feel free to go for it. (Last time the installer was modified we had most 
trouble making the UI not break, but it's probably not too hard if this was a 
command-line only.)

--
stage:  -> needs patch
type: behavior -> enhancement
versions: +Python 3.11 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Sascha]

Sascha  added the comment:

hi,

if you know that it will lead to confusion why is this confusion not written 
here:

https://docs.python.org/3/using/windows.html

If you do software deployment in a company you will only have one version so 
having more than one version is not the point. And if you make append path 
optional available like prepend everybody can choose. Anyway.

What i have seen now is if you uninstall Python it is not cleaning up the path 
variable. Is there any parameter i missed?

--
nosy: +Saxomania

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Steve Dower]

Steve Dower  added the comment:

Prepending is used because it makes the most recently installed version of 
Python the "active" one. It also ensures that a deliberate install is going to 
override any applications that may have put their own copy on PATH 
(deliberately or otherwise). There's no real mechanism for managing multiple 
Python entries on PATH, so it just works out better to prepend and let the 
latest install win.

However, we also disable this option by default because it can lead to 
confusion like what you've experienced. The best option here is to leave it 
disabled and update PATH yourself (or use "py.exe" which is always on PATH), 
since you've got the best awareness of your environment.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44934] Windows installer: Append Python to PATH instead of prepending it

[Bastian Neuburger]

New submission from Bastian Neuburger :

Hi there,

in our organization Python 3.9 is installed on Windows with the PrependPath 
option; as expected the Install and Scripts directories are prepended to path.

However if there are Python scripts with the same name as a system command 
(e.g. a script named ping.py vs. the included ping.exe), the Python script gets 
preferred if I run ping in cmd.exe or Powershell, which makes sense, since the 
Python script path is considered before e.g. C:\Windows\System32\. 

Is it possible to either change the option to append the Install and scripts 
directories to the systems PATH instead of prepending it or add an AppendPath 
option?

I searched for a discussion why prepending was chosen instead of appending but 
I didn't find anything.

--
components: Windows
messages: 399737
nosy: bn_append, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows installer: Append Python to PATH instead of prepending it
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com