[issue29399] python3.dll export forwarders not resolved in all situations due to missing python3?.dll dependency and DLL search path behavior

2021-02-25 Thread Paul Moore


Paul Moore  added the comment:

No, because I want to work with whatever version of Python the user puts there. 
Yes, I could search for "python3*.dll" and load the one I find, but I'm writing 
this in C, and I get a migraine whenever I have to write more than about 15 
lines of C code these days :-)

It's not a big deal either way, though. That project turned out to be too much 
effort to be worth it, so it's now mostly just a proof-of-concept experiment.

> most of us consider a critical security vulnerability rather than a feature

:-) Given that my execution model is "run a user-supplied Python script with a 
user-supplied interpreter" I think any attacker has far easier ways of 
compromising things than hijacking python3.dll...

--

___
Python tracker 
<https://bugs.python.org/issue29399>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41490] Update bundled pip to 20.2.1 and setuptools to 49.2.1

2021-02-02 Thread Paul Moore


Paul Moore  added the comment:

That's specifically the ensurepip PR, which is now outdated since pip 21.0.1 is 
in master. But for this bug report, Steve said "I still consider the changes a 
release blocking regression, but if it's not blocking ensurepip then I guess 
the updated packages can go in now" so I don't know if he considers there to be 
something else that this issue should still track.

--

___
Python tracker 
<https://bugs.python.org/issue41490>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41490] Update bundled pip to 20.2.1 and setuptools to 49.2.1

2021-02-02 Thread Paul Moore


Paul Moore  added the comment:

It can be closed.

--

___
Python tracker 
<https://bugs.python.org/issue41490>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41490] Update bundled pip to 20.2.1 and setuptools to 49.2.1

2021-02-01 Thread Paul Moore


Paul Moore  added the comment:

I'm not sure why, but the PR to update the bundled pip to 21.0.1 (and 
setuptools to 52.0.0) merged cleanly, so this may be obsolete now.

--

___
Python tracker 
<https://bugs.python.org/issue41490>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43077] Update bundled pip to 21.0.1 and setuptools to 52.0.0

2021-01-30 Thread Paul Moore


Change by Paul Moore :


--
pull_requests: +23203
pull_request: https://github.com/python/cpython/pull/24388

___
Python tracker 
<https://bugs.python.org/issue43077>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43077] Update bundled pip to 21.0.1 and setuptools to 52.0.0

2021-01-30 Thread Paul Moore


Change by Paul Moore :


--
keywords: +patch
pull_requests: +23202
pull_request: https://github.com/python/cpython/pull/24387

___
Python tracker 
<https://bugs.python.org/issue43077>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43077] Update bundled pip to 21.0.1 and setuptools to 52.0.0

2021-01-30 Thread Paul Moore


Paul Moore  added the comment:


New changeset 4d11ecbb5ed78e6259ee27289c7638aad795f473 by Paul Moore in branch 
'master':
bpo-43077: Update bundled pip to 21.0.1 and setuptools to 52.0.0 (GH-24386)
https://github.com/python/cpython/commit/4d11ecbb5ed78e6259ee27289c7638aad795f473


--

___
Python tracker 
<https://bugs.python.org/issue43077>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43077] Update bundled pip to 21.0.1 and setuptools to 52.0.0

2021-01-30 Thread Paul Moore


Change by Paul Moore :


--
assignee:  -> paul.moore
nosy: +Marcus.Smith, dstufft, ncoghlan, pradyunsg
stage:  -> patch review

___
Python tracker 
<https://bugs.python.org/issue43077>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43077] Update bundled pip to 21.0.1 and setuptools to 52.0.0

2021-01-30 Thread Paul Moore


New submission from Paul Moore :

I've just released pip 21.0.1, so this updates the bundled copy (as well as 
updating setuptools).

I assume this should be backported to 3.8 and 3.9.

--
components: Library (Lib)
messages: 385986
nosy: paul.moore
priority: normal
pull_requests: 23201
severity: normal
status: open
title: Update bundled pip to 21.0.1 and setuptools to 52.0.0
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue43077>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

> PathAllocCombine() is Windows 8+, so sample code that uses it would only be 
> for Python 3.9+.

Yeah, I'm probably going to remove that. I mainly used it because I'm *so* 
spoiled by Python, writing code in C where I have to actually implement stuff 
for myself rather than just using a stdlib function, just feels so tiresome 
these days :-)

Thanks for the other suggestions.

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

> I think here you're in a very small minority who could get away with this, 
> and so I'd hesitate to make it sound like the recommended approach.

Well, the evidence here is that maybe even I shouldn't be doing this :-)

> What I'd actually recommend (on Windows) is to bundle a copy of Python with 
> your application.

That's actually precisely what I'm doing. But I don't exactly have an 
"application", in the sense that I suspect you mean, which is the difficulty.

I'm trying to address the problem that it's a real pain to ship Python scripts 
as "utilities" on a Windows system. You want such scripts to still be plain 
text, because you typically hack on them a lot. You can't rely on shebangs and 
the launcher, because I continually find that PATHEXT doesn't include ".py" so 
they don't run properly, and "py name_of_script.py" doesn't do path searches. 
And anyway, I don't want to run the scripts with my system Python, because I 
don't want to dump all my dependencies in there.

So what I have is a small stub, that looks for a .py file of the same name, 
alongside the executable (so myutil.exe looks for myutil.py). Rather than using 
the system Python, it looks for a copy of Python in a subdirectory next to the 
script, and uses that. I can then install dependencies in the dedicated 
interpreter.

(And yes, PEP 582 would help make things easier in this area, too, but it never 
gained enough traction.)

I use the embedded distribution as that interpreter. I may be able to use a 
virtualenv instead, but I've not tried that yet.

Multiple copies of the launcher, one per script, and you're done :-)

To be honest, it really sucks that this is the most reliable way of managing 
small utility scripts in Python on Windows. But every other solution I've tried 
has had its own issues. I'd *much* prefer something standard, but I don't know 
how to bundle this in a way that makes it straightforward for "ordinary" users, 
so I've decided just to solve my own problem and leave it at that :-)

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

Confirmed. The following code works as I want:

Py_Main_t get_pymain(wchar_t *base_dir) {
wchar_t *dll_path;
HRESULT hr = PathAllocCombine(
base_dir, L"python\\python3.dll",
PATHCCH_ALLOW_LONG_PATHS, _path
);
if (hr != S_OK) {
error(L"Could not construct Python DLL path");
}

HMODULE py_dll = LoadLibraryExW(dll_path, 0, 
LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR | LOAD_LIBRARY_SEARCH_DEFAULT_DIRS);
if (!py_dll) {
error(L"Could not load Python DLL %ls", dll_path);
}
LocalFree(dll_path);

Py_Main_t py_main = (Py_Main_t)GetProcAddress(py_dll, "Py_Main");
if (!py_main) {
error(L"Could not locate Py_Main function");
}

return py_main;
}


If people think it's worthwhile, I can put together a change to 
https://docs.python.org/3/extending/embedding.html#embedding-python-in-another-application
 (maybe a new section, "Embedding on Windows by dynamically loading the stable 
ABI"?) that uses a code snippet like this.

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

Thanks, I'll give that a try. It sounds like I'm just using the APIs 
incorrectly, which would be good (in the sense that I can do what I wanted, I 
just didn't know how ;-))

I wonder whether it would be worth having a section in the docs somewhere 
explaining how to do this, or more generally what the "best practices" are for 
embedding? If I were to try to put something together, would that be worthwhile?

> I noted that the path to "python3.dll" must be fully qualified.

Just to note, my actual code does use an absolute path, I was over-simplifying 
here. But it's worth being clear on it, particularly if I do write something 
for the docs.

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

Thinking about what you said, "the loader waits to load it until first accessed 
via GetProcAddress()" did you mean by that, that the following code should work:

h = LoadLibraryW(L"some/path/to/python3.dll")
py_main = GetProcAddress(h, "Py_Main")

(with some/path/to/python39.dll being loaded on the second line)? Because if 
so, then that's what doesn't work for me.

Or are you suggesting that I do

AddDllDirectoryW(L"some/path/to");
h = LoadLibraryW(L"python3.dll");
py_main = GetProcAddress(h, "Py_Main");


Because I didn't think to try that - and I'm not 100% sure how I'd have to do 
stuff like LOAD_LIBRARY_SEARCH_USER_DIRS and/or SetDefaultDllDirectories to 
make that work. I find the Microsoft docs on all of this really complex and 
hard to follow if you're a non-expert :-(

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

So I need to dynamically load *both* python3.dll and python39.dll, but if I do 
that I can get the functions from python3.dll? What's the point in doing that? 
Surely I might as well just load python39.dll and get the functions from there, 
in that case.

I hoped that by using python3.dll, I'd be able to avoid needing to deal with 
the version-specific DLL at all, so making my code portable to any version of 
Python 3. I'm pretty sure that's how it works if I statically link to 
python3.dll...

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


New submission from Paul Moore :

I am writing a small application using the embedded distribution to run a 
script supplied by the user. The requirements are very simple, so all I need to 
do is set up argv and call Py_Main.

I'm trying to load the Py_Main function dynamically (for flexibility - see 
below) but GetProcAddress returns 0 when loading the function from python3.dll 
(the stable ABI). This seems to be because the symbols in python3.dll are 
special "fowarding" symbols, that GetProcAddress can't handle.

Is there a way to dynamically load the Python API from the stable ABI? If there 
isn't currently, could one be added?

To explain my requirements in a bit more detail, I don't want to statically 
link, because I want to put the Python distribution in a subdirectory, so that 
it isn't visible on PATH along with my executable, but I'd rather avoid the 
complexities involved in adding a dedicated SxS manifest to point the loader to 
the correct subdirectory for the python3.dll.

Furthermore, I want to provide a graceful fallback if the Python distribution 
is missing (initially, just a friendly error message, but in future maybe 
locating an alternative Python installation to use) and static linking won't 
allow that as I can't recover control if the expected Python DLL isn't present.

The reason I want to use the stable ABI is so that I can upgrade the embedded 
distribution without needing to rebuild the C code. I could search for 
python3X.dll, and just ignore the stable ABI. But that's more filesystem code 
than I really want to write in C... And honestly, I feel that dynamically 
linking is a perfect use case for the stable ABI, so it really should be 
supported.

--
components: Windows
messages: 385621
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Unable to dynamically load functions from python3.dll
type: behavior
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42902] a python embedded program may load "C:\Lib\os.py" on windows system

2021-01-12 Thread Paul Moore


Paul Moore  added the comment:

I thought that *all* versions of Python located the standard library by 
searching for os.py...

--

___
Python tracker 
<https://bugs.python.org/issue42902>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42863] Python venv inconsistent folder structure on windows

2021-01-08 Thread Paul Moore


Paul Moore  added the comment:

Many tools hard code the "if windows then 'Scripts' else 'bin'" logic. They 
will simply fail to work with environments that don't have that layout. Yes, 
they probably should use sysconfig, but many don't (and sometimes for arguably 
good reasons, for example tools run using one Python interpreter that 
manipulate the environment of a different one).

It's something that *could* be changed, but the cost would be significant, the 
work needed to manage the transition is non-trivial, and the benefit isn't 
particularly obvious - "consistency" is never very compelling in isolation, and 
"works better with meson" is likely to be met with the argument that Python's 
layout has been around longer than meson, so it's on meson to accommodate 
Python, not the other way around.

I'm not actively against the idea, but neither do I particularly support it 
(I'm pretty sure it would require work for pip, for no benefit to pip's 
existing users, so it's a net loss for me personally).

If you're interested in pushing for this change go for it - I suspect that it 
would probably need a PEP, and at the very least it would need a community 
discussion - but be prepared for it to meet with a certain amount of resistance.

--

___
Python tracker 
<https://bugs.python.org/issue42863>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42855] pathlib.exists on Windows raises an exception on URL like/bad input

2021-01-07 Thread Paul Moore


Paul Moore  added the comment:

So I guess the key question then is whether Path.exists() should trap 
exceptions and interpret them as "does not exist" (on all platforms, although 
it looks like the null character case in Unix has now been fixed). Which 
doesn't seem unreasonable, I guess.

--

___
Python tracker 
<https://bugs.python.org/issue42855>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42855] pathlib.exists on Windows raises an exception on URL like/bad input

2021-01-07 Thread Paul Moore


Paul Moore  added the comment:

"http:" isn't a valid drive letter, I'd imagine.

--

___
Python tracker 
<https://bugs.python.org/issue42855>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42855] pathlib.exists on Windows raises an exception on URL like/bad input

2021-01-07 Thread Paul Moore

Paul Moore  added the comment:

It's an invalid filename so it raises an exception.

You can get the same on Unix by using an invalid filename (embedded null):

>>> from pathlib import Path
>>> Path("/usr/\0").exists()
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.7/pathlib.py", line 1356, in exists
    self.stat()
  File "/usr/lib64/python3.7/pathlib.py", line 1178, in stat
    return self._accessor.stat(self)
ValueError: embedded null byte

You need to be prepared for exceptions if you aren't sure you have a valid 
path. One thing that might be useful, I guess, is a `Path.is_valid()` function. 
But I don't know if all platforms have a way of asking the OS "is this a valid 
pathname?" So catching the exception is probably best.

--

___
Python tracker 
<https://bugs.python.org/issue42855>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42683] asyncio should handle keyboard interrupt while the event loop is running

2020-12-19 Thread Paul Moore


Change by Paul Moore :


--
nosy: +njs

___
Python tracker 
<https://bugs.python.org/issue42683>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42683] asyncio should handle keyboard interrupt while the event loop is running

2020-12-19 Thread Paul Moore


New submission from Paul Moore :

See the comment on Discourse here: 
https://discuss.python.org/t/feeding-data-generated-via-asyncio-into-a-synchronous-main-loop/5436/28
 (and the thread leading up to this comment).

In the thread, @njs states that if the user hits Ctrl-C while the asyncio event 
loop is running, it's possible for internal asyncio data structures to end up 
in an inconsistent state. If that's the case, then this would make 
asyncio-based code unreliable in real-world use.

I don't have a way to reproduce this - from the Discourse thread, I had assumed 
that ctrl-C was safe to use on an asyncio-based program, but was told 
otherwise, and I can't find anything definitive either way.

At a minimum, the asyncio documentation should confirm that it is 
exception-safe (specifically against Ctrl-C, but in general I'd assume that 
asyncio is safe in the face of uncaught exceptions in user-written async code).

--
components: asyncio
messages: 383370
nosy: asvetlov, paul.moore, yselivanov
priority: normal
severity: normal
status: open
title: asyncio should handle keyboard interrupt while the event loop is running
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42683>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42518] Error Message

2020-12-01 Thread Paul Moore


Paul Moore  added the comment:

You need to learn about chained comparisons in Python. This is not a bug. 
Please stop reopening it.

--
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42518>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42293] Installation of pyinstaller in Windows fails with utf8 error

2020-11-14 Thread Paul Moore


Paul Moore  added the comment:

I've raised https://github.com/pypa/pip/issues/9135 for this.

--

___
Python tracker 
<https://bugs.python.org/issue42293>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42293] Installation of pyinstaller in Windows fails with utf8 error

2020-11-14 Thread Paul Moore


Paul Moore  added the comment:

Erik is correct, this is a pip bug. Please raise an issue on the pip tracker 
linking to this issue and we can fix it.

--

___
Python tracker 
<https://bugs.python.org/issue42293>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-20 Thread Paul Moore


Paul Moore  added the comment:


New changeset 5ab27cc518f614a0b954ff3eb125290f264242d5 by Paul Moore in branch 
'master':
bpo-42041: Clarify how subprocess searches for the executable (GH-22715)
https://github.com/python/cpython/commit/5ab27cc518f614a0b954ff3eb125290f264242d5


--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-19 Thread Paul Moore


Paul Moore  added the comment:

I like that, I've updated the PR accordingly.

--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-19 Thread Paul Moore


Paul Moore  added the comment:

My biggest concern with the suggested wording (which in broad terms looks OK to 
me) is that it reintroduces the issue with the redirector.

By specifically saying that the behaviour in `CreateProcess` applies, we lead 
the user to the statement that the search path includes "the directory 
containing the running application", which the user will interpret as the 
directory of `sys.executable`. So we need to qualify this somewhere by 
clarifying that "the running application" may not actually be `sys.executable` 
(without saying what it actually *is*, as that would mean documentin what the 
redirector does, which Steve wants to avoid).

I don't have a good answer here - we have two conflicting preferences (document 
the search behaviour but don't document the way the redirector works) and 
something needs to give. (To be fair, there's a third conflicting priority 
here, which is me wanting everything to be explicit and clear. I'm willing to 
give that up if you and Steve can agree on something, though).

--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-18 Thread Paul Moore


Paul Moore  added the comment:

Exactly. Why isn't the current directory searched? Why isn't foo/bar searched 
for on PATH? Why is it possible for the user to accidentally remove system 
commands from PATH and lose access to them? Any system is confusing and 
surprising to users only familiar with another system.

And I've had very bad experiences in the past with languages/applications that 
presume to define a "cross-platform" abstraction that ends up just "not 
behaving how I expect a native app to". You can build a cross-platform wrapper 
on top of native behaviour (witness `shutil.which`) but you can't do the 
opposite.

Maybe there's scope for a section in the documentation that discusses how to 
use `subprocess` in a platform-agnostic manner. I'd be OK with that, although 
I'd want it to read along the lines of "these are places where Windows and 
POSIX behave differently" and not "here's some weird stuff Windows does that 
POSIX users need to be aware of" which was how your comment read to me.

Never mind, I think we can just agree to differ on this. It's not likely to 
impact this issue or the PR for it.

--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-18 Thread Paul Moore


Paul Moore  added the comment:

Well, I'm not convinced that using `shutil.which` is the right answer for 
anything other than "python". And even there, I'd recommend using 
`sys.executable` over `shutil.which` in pretty much every case.

My view is that if we ignore the "weirdness" introduced by the redirector[1], 
for everything else `subprocess` is fine - it searches for the executable using 
the normal platform semantics. That's nearly always the same across platforms, 
and when it isn't, users may well prefer consistency with the platform over 
identical cross-platform behaviour. And if they *do* want cross-platform 
consistency, at the cost of occasional differences between their program's 
behaviour and (for example) the shell, then they have `shutil.which` available.

Saying that `subprocess` follows platform semantics is easy, and clear. People 
who need to know platform details can find them out. People who want 
Python-defined cross-platform semantics have `shutil.which`.

That's the basis on which I feel that documenting the Windows rules is 
unnecessary. Also, why would we document the Windows rules, but not the POSIX 
rules? They are arguably just as strange to someone who doesn't know them.

[1] I would characterise that weirdness as being that the redirector impacts 
the behaviour of how the platform locates the "python" command under the 
platform search rules of the platform the redirector is used on - even though 
we don't document the behaviour of the redirector so the user cannot infer the 
actual rules without knowing implementation details.

--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-18 Thread Paul Moore


Paul Moore  added the comment:

I don't think we should document this level of detail, both because it's 
basically Windows standard behaviour and therefore not up to us to document, 
and because it's *way* too overwhelming for the average user.

--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-18 Thread Paul Moore


Paul Moore  added the comment:

OK, PR updated as per discussion.

--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-16 Thread Paul Moore


Paul Moore  added the comment:

> We'd need Mark to weigh in (or possibly track down and ask a colleague), but 
> I expect the assumption is that subprocess.Popen("python") searches PATH 
> first.

I'm sure it was, TBH. The executable directory side of things is really just on 
digging a bit deeper - "well, Windows actually uses the executable directory, 
but that's not relevant because I ran the venv Python". That's when the 
mistaken assumption happens.

--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42055] Python Nuget Packages for Windows Installation: Request for more versions

2020-10-16 Thread Paul Moore


Paul Moore  added the comment:

Python 3.6.9 and later were source-only security fix releases. We don't provide 
*any* binary releases for those versions, the nuget distribution is no 
different in that respect.

The release schedule for Python 3.6 is at 
https://www.python.org/dev/peps/pep-0494/. There are similar PEPs for the 
release schedules of other versions.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42055>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-16 Thread Paul Moore


Paul Moore  added the comment:

> This is still true, though, as much as it's ever been.

But it's *not*, if I understand the problem here. Windows resolves a bare 
"python" command by searching the directory containing the running executable 
first. I would reasonably assume that if I type

path/to/venv/scripts/python.exe foo.py

then the "running executable" is "path/to/venv/scripts/python.exe", and so the 
path containing the running executable is "path/to/venv/scripts", so a search 
for "python" will locate "path/to/venv/scripts/python.exe", because that's how 
Windows path search rules work. The problem is that that exe is a redirector, 
and the script is *actually* being run by the system Python.

Of course, the assumption I made is flawed, because there's never been anything 
saying that a program called "python.exe" can't do anything it likes in the 
process of running a script, including redirecting. But it's been the case for 
a long time, and IMO the introduction of the redirector counts as a 
user-visible behaviour change, and we should therefore explicitly point it out.

I really don't see why you are so reluctant to include this. I'm not asking 
that we guarantee any behaviour, or that we commit ourselves to anything. Just 
that we note that people ought not to be making a specific assumption (which it 
appears from this issue, people *have* been making).

> we could phrase it more positively

Cool, I'll update the PR to add this statement or something similar to the 
subprocess docs in the next couple of days.

I'm not going to fight to keep the comments in the venv documentation. I'm 
disappointed, and I feel like we'll end up with people relying on *more* of the 
implementation details because we're keeping things vague, but I have more 
important things to debate, so I'll drop this :-)

--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-16 Thread Paul Moore


Paul Moore  added the comment:

Fair enough. I think we have to be careful here - people do rely on how the 
"internal" aspects of virtual environments work, like it or not, and we've 
definitely broken people's code in the past as a result of taking a hardline 
"it's not documented, so it's not guaranteed" approach. (Disclaimer - we broke 
some of my code, so I'm not unbiased when I say the above ;-))

I take your point that this is a cross-platform issue, and as such would be 
worth putting in the `subprocess` documentation. When I looked, though, I 
couldn't really find anywhere that felt appropriate. I'll take a second look, 
but if you have any suggestions I'd appreciate it.

The bit that I *do* think is a venv gotcha is that it's entirely reasonable for 
a user to expect that if they run path\to\venv\Scripts\python.exe, then their 
Python script will be run by that executable. The redirector breaks that 
assumption, and while I'm fine with the idea that we don't want to document all 
of the details of the redirector, I *do* think we should document that you must 
not make that assumption.

How about this:

   The mechanism by which virtual enviroments are implemented means that
   you should not assume that the executable in the virtual environment
   is the one that will ultimately run your script. As a result, you
   should always use `sys.executable` to identify "this script's Python
   interpreter", as that is guaranteed to be set as you would expect.

I'm not completely happy with the above, but that's mostly because I'd rather 
just document how the process works (even if it's with a disclaimer "this is 
implementation detail and may change without notice"). But that's not my call 
to make, so I'll defer to you on that.

--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-15 Thread Paul Moore


Paul Moore  added the comment:

Eryk - I suggest that we modify (or remove) the note in venv that my PR adds as 
part of such a change, rather than try to qualify the text now (which would 
probably only make it harder for people to understand the issue).

--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-15 Thread Paul Moore


Paul Moore  added the comment:

I've created https://github.com/python/cpython/pull/22715 as a suggested 
documentation fix. Mark, does this cover what you would have needed to know? 
Steve, is what I've written technically accurate?

--
stage: patch review -> 

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-15 Thread Paul Moore


Change by Paul Moore :


--
keywords: +patch
pull_requests: +21683
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22715

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42041] venv subprocess call to python resolves to wrong interpreter

2020-10-15 Thread Paul Moore


Paul Moore  added the comment:

Agreed, this is just "normal Windows behaviour". It's definitely surprising and 
non-obvious (I saw this issue and it never even occurred to me that this was 
what was happening) so I'd support documenting it somewhere. My instinct is 
that the subprocess documentation is the right place, but I suspect that in 
practice, people hitting this issue with a virtual environment won't spot 
anything we write there, so it'll end up just being a "well, we told you so" 
measure rather than helping users in a practical sense.

So I'd agree with Steve's comment - Mark, if you can suggest anywhere we could 
document this which you *would* have spotted when you encountered this issue, 
that would be great.

Hmm, maybe we could document in venv that the Python executable in a venv 
redirects to the base Python, so that the "application directory" for a script 
run from a venv is the base executable's directory, not the venv - with a link 
to the MS documentation that explains how this affects path searches? (IIRC, 
we've been hesitant to document the redirection as it's an "implementation 
detail", but this is clearly a case where it affects user-visible behaviour, so 
we can't really use that argument any more...)

--

___
Python tracker 
<https://bugs.python.org/issue42041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42013] venv on Windows with symlinks is broken if invoked with -I

2020-10-12 Thread Paul Moore


Paul Moore  added the comment:

I'm inclined to think that creating a venv from within another venv should be 
allowed. Tools like pipx can result in *other* tools being run from within a 
virtual environment, and I don't think it's good to disallow that usage - as an 
example, I have virtualenv, tox and nox installed via pipx, and all of them 
create virtual environments.

So I'm +1 on fixing this by calling realpath.

I don't think we need to do anything special for --system-site-packages. The 
docs say "Give the virtual environment access to the system site-packages dir". 
If people are trying to chain venvs using it, they are misreading that comment 
- and the best they could hope for is to request that it's made clearer.

--

___
Python tracker 
<https://bugs.python.org/issue42013>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41774] While removing element from list using for and remove(), which has same items output is not right

2020-09-12 Thread Paul Moore


Paul Moore  added the comment:

Yeah, apologies - I saw the email notification, but Eric removed me from the 
nosy list (quite reasonably) and I didn't notice there had been extra 
discussion :-(

If someone wants to re-open the issue, feel free.

--

___
Python tracker 
<https://bugs.python.org/issue41774>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41774] While removing element from list using for and remove(), which has same items output is not right

2020-09-12 Thread Paul Moore


Paul Moore  added the comment:

There is a bug in your program. You should not mutate the list while looping 
over it.

--
nosy: +paul.moore

___
Python tracker 
<https://bugs.python.org/issue41774>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41774] While removing element from list using for and remove(), which has same items output is not right

2020-09-12 Thread Paul Moore


Change by Paul Moore :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41774>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41490] Update bundled pip to 20.2.1 and setuptools to 49.2.1

2020-09-11 Thread Paul Moore


Paul Moore  added the comment:

I think this reproduces the underlying issue:

>>> import zipfile
>>> from pathlib import Path
>>> p = zipfile.Path("tst.zip")
>>> Path("tst.zip").unlink()
Traceback (most recent call last):
  File "", line 1, in 
  File 
"C:\Users\UK03306\AppData\Local\Programs\Python\Python38\lib\pathlib.py", line 
1321, in unlink
self._accessor.unlink(self)
PermissionError: [WinError 32] The process cannot access the file because it is 
being used by another process: 'tst.zip'
>>>

Basically, zipfile.Path objects don't have any way to close the underlying 
zipfile, so you have to delete the path object to free up the file for deletion.

In the context of importlib, maybe permanently holding a reference to the 
zipfile.Path object can't work, and it needs to be re-opened each time it's 
needed?

--

___
Python tracker 
<https://bugs.python.org/issue41490>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41619] Subprocesses created with DETACHED_PROCESS can pop up a console window

2020-08-23 Thread Paul Moore


Paul Moore  added the comment:

Confirmed, CREATE_NO_WINDOW works - in my test I used it *with* 
DETACHED_PROCESS.

Is it not a bug that the launcher doesn't respect the value of DETACHED_PROCESS 
when calling the Python interpreter, though? If nothing else, it means that 
there are differences between running Python via the launcher and not doing so 
(which are also evident as differences between a virtual environment and a 
non-virtual interpreter, as the venv machinery uses the launcher internally).

--

___
Python tracker 
<https://bugs.python.org/issue41619>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41619] Subprocesses created with DETACHED_PROCESS can pop up a console window

2020-08-23 Thread Paul Moore


Paul Moore  added the comment:

Originally discovered in https://github.com/pypa/virtualenv/issues/1928 - see 
that issue for more context.

--

___
Python tracker 
<https://bugs.python.org/issue41619>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41619] Subprocesses created with DETACHED_PROCESS can pop up a console window

2020-08-23 Thread Paul Moore


New submission from Paul Moore :

The following code pops up an extra console window when run on Windows 10, 
Python 3.8:

from subprocess import DETACHED_PROCESS, Popen
p = Popen(["py", "-c", "from time import sleep; sleep(5)"], 
creationflags=DETACHED_PROCESS)

Adding CREATE_NO_WINDOW doesn't help, nor does adding a STARTUPINFO with 
SW_HIDE.

The problem occurs whether the Python interpreter is started via the py.exe 
wrapper or the actual python.exe. However, changing the process being run from 
"py" to sys.executable makes the problem go away. I've seen similar issues when 
using sys.executable, but the script is being called from a virtualenv.

I can't find any set of options for Popen that reliably avoids showing a 
console window when invoked like this.

--
components: Windows
messages: 375811
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Subprocesses created with DETACHED_PROCESS can pop up a console window
type: behavior
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue41619>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41565] from os.path import join join('3', '{:3') return '{:3' in windows

2020-08-18 Thread Paul Moore


Paul Moore  added the comment:

I think the key point here is that os.path.join is *meant* to have 
platform-native semantics, so it treats '{:' as a drive letter on Windows, but 
Linux doesn't have drive letters, so os.path.join treats '{:' as a normal part 
of a filename on Linux.

--

___
Python tracker 
<https://bugs.python.org/issue41565>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40740] Missing api-ms-win-core-path-l1-1.0.dll for python-3.9.0b1-amd64.exe Under Win7

2020-08-09 Thread Paul Moore


Paul Moore  added the comment:

https://docs.python.org/3.9/using/windows.html should probably be updated:

"""
As specified in PEP 11, a Python release only supports a Windows platform while 
Microsoft considers the platform under extended support. This means that Python 
3.9 supports Windows Vista and newer. If you require Windows XP support then 
please install Python 3.4.
"""

Technically, the link to PEP 11 is sufficient, but "Python 3.9 supports Windows 
Vista and newer" is inaccurate.

If someone wanted to submit a PR, I'm sure it would be welcome.

--

___
Python tracker 
<https://bugs.python.org/issue40740>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41327] Windows Store "stub" Python executables give confusing behaviour

2020-07-21 Thread Paul Moore


Paul Moore  added the comment:

Understood that these would be duplicates, and I certainly don't want to put 
more pressure on you, but I think there would still be value in having a known 
reporting channel for people to say "I am also affected by this":

* MS Developers get a more objective feel for the scale of the issue.
* Users who report the issue can track progress (they will presumably get told 
*what* their report is a duplicate of, and can follow the underlying issue).
* Python package developers have something definite that they can offer to 
users.

My suspicion here is that MS simply doesn't have a user-visible support channel 
for the Store Python stubs, which is why we've ended up discussing this here.

--

___
Python tracker 
<https://bugs.python.org/issue41327>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41327] Windows Store "stub" Python executables give confusing behaviour

2020-07-17 Thread Paul Moore


Paul Moore  added the comment:

I thought that might be the answer. But does anyone know where I can repost 
this as an issue against the Store distribution? I'm happy to report there if I 
can find out how...

Hopefully Steve can point me in the right direction.

--

___
Python tracker 
<https://bugs.python.org/issue41327>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41327] Windows Store "stub" Python executables give confusing behaviour

2020-07-17 Thread Paul Moore


Paul Moore  added the comment:

See, for example, https://github.com/pypa/packaging-problems/issues/379 as an 
illustration of the user (and project maintainer!) confusion this causes.

--

___
Python tracker 
<https://bugs.python.org/issue41327>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41327] Windows Store "stub" Python executables give confusing behaviour

2020-07-17 Thread Paul Moore


New submission from Paul Moore :

First of all, I do know that this is an issue with the Windows Store 
distribution, rather than the python.org one. But (a) I don't know where to 
report a bug against the Store implementation except here, and (b) it's 
arguably a case of the Windows implementation "stealing" the Python command, so 
worth flagging against core Python.

The problem is that Windows 10 installs `python` and `python3` executables by 
default, which open the Windows Store offering to install Python. That's not a 
bad thing - it's nice to have Python available with the OS! Although (see 
below) hijacking the `python` and `python3` commands for this purpose has some 
problems.

But those stubs silently do nothing when run with command line arguments, and 
because the python.org distribution doesn't put Python on PATH by default, 
users end up with the stubs available even if they install python.org Python.

We're getting a lot of bug reports from users as a result of this, when they 
follow standard instructions like "execute `python -m pip`". With the stub on 
the path, this silently does nothing, even though the user has installed 
Python. This is a very bad user experience, and not one that projects like pip 
can do anything to alleviate, other than having extremely convoluted "how to 
run pip" commands:

"""
To run pip, type `python -m pip` if you're on Unix, and `py -m pip` on Windows. 
Unless you're using the Windows Store version when you should do `python -m 
pip` on Windows too. Or on Unix when you might need to do `python3 -m pip`. 
Or...
"""

I don't have a good answer to this issue. Maybe the Windows Store stubs could 
detect core Python and do something better if it's installed? Maybe the stubs 
should print an explanatory message if run with command line arguments? Maybe 
Store Python should be available on Windows via some less error-prone mechanism?

I'm pretty sure if a Linux distribution shipped a `python` command that didn't 
run Python the way users expected, there would be complaints. After all, we 
have PEP 394 that explicitly states how we expect Unix systems to work. Maybe 
we need something similar for Windows?

--
assignee: steve.dower
components: Windows
messages: 373845
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows Store "stub" Python executables give confusing behaviour
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue41327>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40863] bytes.decode changes/destroys line endings on windows

2020-06-04 Thread Paul Moore


Paul Moore  added the comment:

Because if you open a file in text mode (without "b" in the mode), Python 
writes \n (newline) characters as \r\n (carriage return, line feed) which are 
the Windows textfile representation of "Newline".

>From the documentation of the built in open() function,

"When writing output to the stream, if newline is None, any '\n' characters 
written are translated to the system default line separator, os.linesep."

--
resolution:  -> not a bug

___
Python tracker 
<https://bugs.python.org/issue40863>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40448] ensurepip uses cache directory

2020-05-20 Thread Paul Moore


Paul Moore  added the comment:

PR looks good to me. There's a test failure that needs fixing and the PR needs 
a news entry, but otherwise looks fine.

--

___
Python tracker 
<https://bugs.python.org/issue40448>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36264] os.path.expanduser should not use HOME on windows

2020-05-16 Thread Paul Moore


Paul Moore  added the comment:

> Perhaps the best approach for the sake of POSIX compatibility is to set HOME 
> on startup to the correct value? 

If Python starts setting `HOME`, that has the potential to affect programs 
called in a subprocess, possibly breaking them (making them not find the user's 
config, for example). Even if we only set `HOME` when it's not already set, we 
can get this issue. (Vim, for example, will break if you set HOME to somewhere 
other than where the user has their config stored).

Certainly, it's arguable that such programs are not respecting Windows 
conventions correctly, but in practical terms that's not really that relevant. 
The user will just see it as "Python can't call my program correctly".

This is a very real issue, that I used to hit a lot in the past, when native 
Windows support was a lot less common in open source code than it is now. And 
it was nearly always made worse by the programs that tried to be "too clever" 
about hiding the differences between Windows and Unix.

So I'm a strong -1 on Python doing anything with `HOME` here.

--

___
Python tracker 
<https://bugs.python.org/issue36264>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40134] Inconsistent ANSI escape code handling on Windows 10

2020-04-01 Thread Paul Moore


Paul Moore  added the comment:

This works fine for me in Windows terminal, but I see the behaviour described 
when using the conventional "Command prompt" window.

Enabling ANSI codes is handled via SetConsoleMode (see here: 
https://docs.microsoft.com/en-us/windows/console/setconsolemode). The following 
proof of concept script correctly displays coloured text:

from ctypes import *
kernel32 = windll.kernel32
kernel32.SetConsoleMode(kernel32.GetStdHandle(-11), 7)
ansi_red = "\x1b[31m"
ansi_normal = "\x1b[0m"
print(ansi_red + "This is red!" + ansi_normal)

Agreed this would be worthwhile setting on stdout by default. The code at 
https://docs.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences#example-of-enabling-virtual-terminal-processing
 seems to be an example of how to do this while still supporting older systems

--

___
Python tracker 
<https://bugs.python.org/issue40134>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39090] Document various options for getting the absolute path from pathlib.Path objects

2020-02-12 Thread Paul Moore


Paul Moore  added the comment:

You've provided links to your branches, but not to the specific text you're 
proposing to add. Can you link to a diff or something that shows what you've 
added more precisely?

--

___
Python tracker 
<https://bugs.python.org/issue39090>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39090] Document various options for getting the absolute path from pathlib.Path objects

2020-02-10 Thread Paul Moore


Paul Moore  added the comment:

> In short -- I understand that this is a complex issue, but making an absolute 
> path is a pretty common use case, and we've had os.path.abspath() for 
> decades, so there should be one obvious way to do it, and it should be easily 
> discoverable.

+1 on this.

Given that (as far as I can tell from the various discussions) `resolve` works 
fine as long as the file exists, maybe the key distinction to make is whether 
you have an existing file or not.

(More subtle questions like UNC path vs drive letter, mentioned on the 
Discourse thread, are probably things that we can defer to a "more advanced 
cases" discussion in the docs).

--
nosy: +paul.moore

___
Python tracker 
<https://bugs.python.org/issue39090>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39255] Windows and Unix run-time differences

2020-01-08 Thread Paul Moore


Paul Moore  added the comment:

For me, I headed straight for "Sharing state between processes" and the "Shared 
memory" object. That's probably because I was reviewing someone else's code, 
rather than writing my own, but nevertheless when coding I do tend to dive 
straight for the section that describes what I want to do, and miss "overview" 
type discussions.

The way the shared memory object is described, it reads that it is just that - 
shared. And so I'd assume that if a shared memory object is in multiple 
processes in a pool, it would be the *same* shared memory region, and the value 
would be accessible from all the processes.

>From there, for me at least, it's easy to proceed to the mistake of thinking 
>that the global initialisation of the x and y variables creates the *same* 
>shared memory objects in each process in the pool. Clearly it doesn't, hence 
>this is "not a bug" but for me it's an easy mistake to make.

Maybe it would be enough just to add a comment to the shared memory object 
documentation that said "every shared memory object is independent - there is 
no way to create a reference to the same shared memory object in multiple 
processes, instead you need to create the object in one process and pass it to 
all of the others". That would probably have made me stop and think long enough 
to not make the mistake I did.

--

___
Python tracker 
<https://bugs.python.org/issue39255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39255] Windows and Unix run-time differences

2020-01-08 Thread Paul Moore


Paul Moore  added the comment:

Agreed it's not a bug, but I will say it took me a while to work out *why* it's 
not a bug (namely, that even though the OP is using shared memory values, the 
code relies on fork semantics to share the two Value objects that *reference* 
the shared memory).

It would be worth adding a note to the documentation on shared memory values at 
https://docs.python.org/3.8/library/multiprocessing.html#sharing-state-between-processes
 to make it clearer that it's the user's responsibility to make sure the Value 
object is passed to every subprocess that wants to interact with it.

--

___
Python tracker 
<https://bugs.python.org/issue39255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39051] Python not working on Windows 10

2019-12-15 Thread Paul Moore


Paul Moore  added the comment:

Can you also confirm that the installation of Python was done with the standard 
Python installer from the python.org website, and is not another distribution 
(such as Anaconda)?

--

___
Python tracker 
<https://bugs.python.org/issue39051>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39051] Python not working on Windows 10

2019-12-15 Thread Paul Moore


Paul Moore  added the comment:

> sys.path = [
>'C:\\Developing\\Python\\python38.zip',
>'C:\\Developing\\Python\\python-3.8.0.amd64\\Lib\\',
>'C:\\Developing\\Python\\python-3.8.0.amd64\\Lib\\lib-tk',
>'C:\\Developing\\Python\\python-3.8.0.amd64\\DLLs\\',
>'C:\\Developing\\Python',

The subdirectory python-3.8.0.amd64 in the various paths above should not be 
present, and would not be in a normal clean Python installation. It appears 
that somehow your installation has been corrupted.

I would recommend a clean install. if that doesn't work, it's likely that some 
environment variables or registry paths in your system are set incorrectly (or 
at least, in ways the standard installation is not expecting).

--

___
Python tracker 
<https://bugs.python.org/issue39051>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39025] Windows Python Launcher does not update PATH to Scripts directory

2019-12-11 Thread Paul Moore


Paul Moore  added the comment:

Most items in the Scripts directory can be run using the `-m` flag to Python, 
so you can use something like `py -m pip` to run pip without needing the 
Scripts directory in PATH.

If an individual project like PyQt doesn't support -m, it's relatively easy for 
them to add, so it might be worth suggesting to them.

--

___
Python tracker 
<https://bugs.python.org/issue39025>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38999] Python launcher on Windows does not detect active venv

2019-12-09 Thread Paul Moore


Paul Moore  added the comment:

See PEP 486 (https://www.python.org/dev/peps/pep-0486/). This is intended 
behaviour, as it is assumed that explicitly specifying a Python version means 
that the user had a specific Python interpreter in mind.

There are also technical issues - if the shebang says #!/usr/bin/python2, and a 
virtual environment is active, how do we know if it's a Python 2 or 3 
environment (without launching the interpreter, which is a significant extra 
cost for the launcher)? It would definitely be wrong to launch a Python 3 
interpreter in that case.

--

___
Python tracker 
<https://bugs.python.org/issue38999>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38999] Python launcher on Windows does not detect active venv

2019-12-09 Thread Paul Moore


Change by Paul Moore :


--
Removed message: https://bugs.python.org/msg358060

___
Python tracker 
<https://bugs.python.org/issue38999>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38999] Python launcher on Windows does not detect active venv

2019-12-09 Thread Paul Moore


Paul Moore  added the comment:

See PEP 486 
(https://www.youtube.com/watch?v=Lmq13nrEFaM=PLuE3ABw6vYeR0gcPT6uj25ZZhmSP9vRB2=28=0s).
 This is intended behaviour, as it is assumed that explicitly specifying a 
Python version means that the user had a specific Python interpreter in mind.

There are also technical issues - if the shebang says #!/usr/bin/python2, and a 
virtual environment is active, how do we know if it's a Python 2 or 3 
environment (without launching the interpreter, which is a significant extra 
cost for the launcher)? It would definitely be wrong to launch a Python 3 
interpreter in that case.

--

___
Python tracker 
<https://bugs.python.org/issue38999>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38974] using filedialog.askopenfilename() freezes python 3.8

2019-12-04 Thread Paul Moore


Change by Paul Moore :


--
nosy: +gpolo, serhiy.storchaka

___
Python tracker 
<https://bugs.python.org/issue38974>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38974] using filedialog.askopenfilename() freezes python 3.8

2019-12-04 Thread Paul Moore


Paul Moore  added the comment:

Can you provide a minimal, self-contained, example of a program that 
demonstrates this behaviour in 3.8, but works in 3.7. It's not really possible 
to determine what the issue might be without a means of reproducing the problem.

--
components: +Tkinter -Windows

___
Python tracker 
<https://bugs.python.org/issue38974>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38021] pep425 tag for AIX is inadequate

2019-11-26 Thread Paul Moore


Paul Moore  added the comment:

> replacement platform_tag, not compatibility tag.

Ah, I see, sorry. In that case, this should be fine, it's purely a CPython 
question. There's obviously a follow-on discussion about how that platform tag 
is *incorporated* into the compatibility tags, but as you say, that's a 
separate point.

--

___
Python tracker 
<https://bugs.python.org/issue38021>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38905] venv python reports wrong sys.executable in a subprocess on Windows

2019-11-25 Thread Paul Moore


Paul Moore  added the comment:

I presume there's also the option of setting up the environment (or however 
it's done now - I know the details changed as the feature was developed) so 
that the "base" python.exe pretends to be the venv one, exactly as the wrapper 
does.

However, that may well have other difficult-to-fix implications, not least that 
calling the base Python using an explicit full path should act like the base 
Python, and *not* like the venv one.

IMO, the key thing here is that either the various limitations/quirks of 
redirecting to the base Python should either be treated as bugs, or they should 
be documented (even if only in the form of explicitly saying not to rely on any 
specific behaviour - e.g. "running an unqualified python and expecting a PATH 
search to pick up the same executable as the parent shell would is not 
supported and may produce unexpected results").

Virtual environments are a key part of most Python development workflows, and 
virtualenv is in the process of switching to use the core venv module 
internally. When that happens, there will be a lot more visibility for 
unexpected behaviours like this one.

--

___
Python tracker 
<https://bugs.python.org/issue38905>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38905] venv python reports wrong sys.executable in a subprocess on Windows

2019-11-25 Thread Paul Moore


Paul Moore  added the comment:

Yes, it does.

I think we'd need input from Steve Dower here, as these changes were made (I 
believe) in support of the Windows Store build of Python, so any changes would 
need to be considered in the light of how they would affect that. I do, 
however, consider this to be a regression that should be fixed.

BTW, just for completeness,

>>> subprocess.check_output([sys.executable, '-c', 'import sys; 
>>> print(sys.executable)'])

works as I'd expect, and that's the idiom that is often used. So relying on a 
path search to find the correct Python can be considered an unusual case (but 
nevertheless one I'd expect to be fixed).

I assume that the issue here is that the code is being run by the python.dll in 
the base environment, and whens searching for executables, Windows gives "exes 
that are in the same directory as the currently executing code" priority over 
PATH. See 
https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessw,
 specifically

"""
If the file name does not contain a directory path, the system searches for the 
executable file in the following sequence:

1. The directory from which the application loaded.
2. The current directory for the parent process.
3. The 32-bit Windows system directory. Use the GetSystemDirectory function to 
get the path of this directory.
4. The 16-bit Windows system directory. There is no function that obtains the 
path of this directory, but it is searched. The name of this directory is 
System.
5. The Windows directory. Use the GetWindowsDirectory function to get the path 
of this directory.
6. The directories that are listed in the PATH environment variable. Note that 
this function does not search the per-application path specified by the App 
Paths registry key. To include this per-application path in the search 
sequence, use the ShellExecute function.
"""

--

___
Python tracker 
<https://bugs.python.org/issue38905>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38905] venv python reports wrong sys.executable in a subprocess on Windows

2019-11-25 Thread Paul Moore


Paul Moore  added the comment:

The behaviour in this area is different between 3.7.0, 3.7.2, and 3.7.3 (at 
least). I have reproduced the issue with 3.7.3. Steve Dower made changes to the 
way the python executable works in venvs in the point releases of 3.7 - see 
https://github.com/pypa/virtualenv/issues/1380 and 
https://github.com/pypa/virtualenv/issues/1339 for some discussion of how this 
affected virtualenv.

I suspect this issue is related - from 3.7.2 onwards, the python.exe in a venv 
is a redirector which runs the "base" python.exe, but with sys.executable 
showing the redirector rather than the actual running exe.

--

___
Python tracker 
<https://bugs.python.org/issue38905>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38021] pep425 tag for AIX is inadequate

2019-11-23 Thread Paul Moore


Paul Moore  added the comment:

Thanks for the clarification, Nick. Yes, I agree that the basic "this is the 
tag that matches this precise CPython installation" is not the difficult part 
of the problem here, and if that is all that this issue is targeting, there 
should be no major problem.

My understanding was that Michael wanted to address the other part of the 
issue, that installers would somehow need to encapsulate the question of what 
binaries were compatible with what installations, and that, as you say, is a 
*much* murkier question.

If you're comfortable that the two aspects can be cleanly separated (and 
ideally that the second can be avoided altogether) then I'm much happier.

--

___
Python tracker 
<https://bugs.python.org/issue38021>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38021] pep425 tag for AIX is inadequate

2019-11-22 Thread Paul Moore


Paul Moore  added the comment:

What complicates the issue for AIX (and reminds me very strongly of the MacOS 
and manylinux situations, both of which were defined after the original tag 
PEP, and so contain additional insights) is the business of certain tags being 
compatible across multiple AIX versions.

That's something that needs to be fairly clearly specified, so that 
implementors and maintainers understand the design. And even more so for a 
niche platform like AIX, as we can't rely on a platform expert being available. 
(Consider for example a pip issue "this wheel says it's compatible with AIX 
x.y.z, my machine is AIX x.y.w which is documented as compatible, why isn't it 
working?")

It's possible that all of this may not have any relevance to the specific 
change to core Python, but it's hard to be sure of that when there's nothing 
other than your explanation to go on. A tagging spec would act as a clear 
reference to work from (even if it's basically just you writing up your 
knowledge, doing so would at least allow people to say "hey - I don't follow 
that bit, can you clarify").

To put it another way, you need somebody to sign off on this change as correct. 
You'll have an easier time of getting that if there's a written up spec that 
Python developers can refer to, without having to go back to (and understand) 
all of the AIX source material that it's based on.

--

___
Python tracker 
<https://bugs.python.org/issue38021>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38021] pep425 tag for AIX is inadequate

2019-11-22 Thread Paul Moore


Paul Moore  added the comment:

PyPA member here - if this PR is defining new compatibility tags, I would have 
expected it to need discussion as a revision to PEP 425, much like the 
manylinux efforts did. (It may actually be a closer parallel with MacOS, which 
didn't have a tagging PEP - I'm not sure how that would relate here, but as 
MacOS is a much more mainstream platform I'd be inclined to err on the side of 
caution and look for AIX to be explicitly covered in the tag specs).

I thought that was something that had been discussed on the Pip tracker, but 
maybe the implications weren't clear (I don't really understand the AIX scheme, 
so I'm relying on someone else, probably Michael, to propose something that 
could be added to PEP 425 and to lead/guide the discussion).

--
nosy: +paul.moore

___
Python tracker 
<https://bugs.python.org/issue38021>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38533] v3.7.5 py script run ok with python.exe but not pythonw.exe (python silent console not working)

2019-10-20 Thread Paul Moore


Paul Moore  added the comment:

Can you give an example of a script that fails? If you try to print (or 
otherwise use the standard IO streams) pythonw will fail, because there are no 
stdio streams for a GUI executable - and the traceback, which goes to stderr by 
default, will be lost.

This is my best guess as to your issue here, but without a reproducible 
example, it's hard to say anything useful.

--

___
Python tracker 
<https://bugs.python.org/issue38533>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38369] Python 3.7.4 venv command does not install pip

2019-10-04 Thread Paul Moore


Paul Moore  added the comment:

Sigh. Never mind, this appears to be because virtualenv uses its own hacked 
copy of site.py which is missing a lot of the venv support code.

I'll report this over on virtualenv.

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38369>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38369] Python 3.7.4 venv command does not install pip

2019-10-04 Thread Paul Moore


Paul Moore  added the comment:

Further update - this appears to also happen on Ubuntu (at least the version 
available through the Windows Linux subsystem), so it's not a Windows-specific 
issue.

--
assignee: steve.dower -> 

___
Python tracker 
<https://bugs.python.org/issue38369>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38369] Python 3.7.4 venv command does not install pip

2019-10-04 Thread Paul Moore


Paul Moore  added the comment:

This appears to be somehow caused by running Python 3.7.4 from an active 
virtualenv (which I'd forgotten I had active when I did the test).

Regardless, I'd have expected that the command would either correctly create a 
venv containing pip, or fail to create the venv at all. Creating an incomplete 
venv is extremely confusing for the user, if nothing else.

It looks like this is *not* a regression - it's been like this since 3.6.8 at 
least.

Actually, the created venv looks pretty broken to me - it has the *parent* 
environment in `sys.path`, but does not have its *own* environment there:

>.\ve-368a\Scripts\python.exe
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', 'C:\\Work\\Scratch\\vv\\ve-368a\\Scripts\\python36.zip', 
'C:\\Users\\UK03306\\.virtualenvs\\f1980c81dac32d4\\DLLs', 
'C:\\Users\\UK03306\\.virtualenvs\\f1980c81dac32d4\\lib', 
'C:\\Users\\UK03306\\.virtualenvs\\f1980c81dac32d4\\Scripts', 
'C:\\Utils\\PythonVersions\\Python-3.6.8\\Lib', 
'C:\\Utils\\PythonVersions\\Python-3.6.8\\DLLs', 
'C:\\Users\\UK03306\\.virtualenvs\\f1980c81dac32d4', 
'C:\\Users\\UK03306\\.virtualenvs\\f1980c81dac32d4\\lib\\site-packages']

Note the lack of ve-368a directories, and the presence of a lot of directories 
from .virtualenvs (the parent environment).

--

___
Python tracker 
<https://bugs.python.org/issue38369>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38369] Python 3.7.4 venv command does not install pip

2019-10-04 Thread Paul Moore


New submission from Paul Moore :

The venv module with Python 3.4 on Windows doesn't install pip - even though 
the --without-pip flag is not specified. This appears to be a regression since 
pip is installed when using 3.7:

>C:\Utils\PythonVersions\Python-3.7.3\python.exe -m venv ve-373
>dir .\ve-373\Scripts\pip.exe


Directory: C:\Work\Scratch\vv\ve-373\Scripts


ModeLastWriteTime Length Name
- -- 
-a   04/10/2019 10:08 102765 pip.exe


>py -m venv ve-374
>dir .\ve-374\Scripts\pip.exe
dir : Cannot find path 'C:\Work\Scratch\vv\ve-374\Scripts\pip.exe' because it 
does not exist.
At line:1 char:1
+ dir .\ve-374\Scripts\pip.exe
+ 
+ CategoryInfo  : ObjectNotFound: 
(C:\Work\Scratch...Scripts\pip.exe:String) [Get-ChildItem], 
ItemNotFoundException
+ FullyQualifiedErrorId : 
PathNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand
>py -V
Python 3.7.4

--
assignee: steve.dower
components: Library (Lib)
messages: 353911
nosy: paul.moore, steve.dower
priority: normal
severity: normal
status: open
title: Python 3.7.4 venv command does not install pip
type: behavior
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue38369>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37850] Console: holding right arrow key reproduces entire previous input

2019-08-14 Thread Paul Moore


Paul Moore  added the comment:

This is normal behaviour of the Windows console command line editing. Python 
simply inherits the standard console behaviour.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue37850>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25172] Unix-only crypt should not be present on Windows.

2019-08-13 Thread Paul Moore


Paul Moore  added the comment:

To clarify, I think adding the label needs core dev (or maybe triager) rights 
on github, so I don't think it's something you can do yourself.

--

___
Python tracker 
<https://bugs.python.org/issue25172>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25172] Unix-only crypt should not be present on Windows.

2019-08-13 Thread Paul Moore


Paul Moore  added the comment:

I added the label for you.

--

___
Python tracker 
<https://bugs.python.org/issue25172>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36714] Tweak doctest 'example' regex to allow a leading ellipsis in 'want' line

2019-08-09 Thread Paul Moore


Paul Moore  added the comment:

It shouldn't be hard to update the regex to accept either "... " followed by 
other text or "..." on a line on its own, surely?

--
nosy: +paul.moore

___
Python tracker 
<https://bugs.python.org/issue36714>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25172] Unix-only crypt should not be present on Windows.

2019-08-08 Thread Paul Moore


Paul Moore  added the comment:

That's where I decided it was too complex for now :-)

The whole test_crypt.py file is skipped if you can't import crypt, via the use 
of the test.support.import_module() function. To be able to write a test for a 
platform where the whole point is that you can't import crypt, you'd have to 
rewrite that. So you'd get something like

try:
import crypt
except ImportError as e:
if sys.platform == "win32":
assert "not supported" in e.msg
raise unittest.SkipTest("crypt is not present on this platform")

But you can't use "assert" like that in top-level code (or at least I don't 
think you can - I'm not that familiar with unittest, but I think assertions 
have to live in classes in unittest).

At this point I gave up. It starts to be quite a big rewrite for a relatively 
minor benefit.

The alternative would be, if there was a "Windows-specific tests" test module, 
we could have put a test for this situation in there. But I don't think there 
is such a module.

--

___
Python tracker 
<https://bugs.python.org/issue25172>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25172] Unix-only crypt should not be present on Windows.

2019-08-08 Thread Paul Moore


Paul Moore  added the comment:

On reflection, adding a test for the Windows-specific behaviour looks like it 
could be more complex than I'd anticipated, so I'm happy for this to go in 
without an extra test. I'll look at adding a test in a follow-up PR (or if 
someone else wants to, that's fine as well).

--

___
Python tracker 
<https://bugs.python.org/issue25172>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25172] Unix-only crypt should not be present on Windows.

2019-08-08 Thread Paul Moore


Paul Moore  added the comment:


New changeset f4e725f224b864bf9bf405ff7f863cda46fca1cd by Paul Moore 
(shireenrao) in branch 'master':
bpo-25172: Raise appropriate ImportError msg when crypt module used on Windows 
(GH-15149)
https://github.com/python/cpython/commit/f4e725f224b864bf9bf405ff7f863cda46fca1cd


--

___
Python tracker 
<https://bugs.python.org/issue25172>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25172] Unix-only crypt should not be present on Windows.

2019-08-06 Thread Paul Moore


Paul Moore  added the comment:

It might be worth adding a test (running only on Windows) to confirm that if 
you try to import crypt you get an import error with a message that includes 
the phrase "not supported". That will ensure that we don't get regressions here 
in future, while still not tying us too strictly to the exact wording.

--

___
Python tracker 
<https://bugs.python.org/issue25172>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25172] Unix-only crypt should not be present on Windows.

2019-08-06 Thread Paul Moore


Paul Moore  added the comment:

The proposed wording seems good to me.

--

___
Python tracker 
<https://bugs.python.org/issue25172>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37369] Issue with pip in venv on Powershell in Windows

2019-07-05 Thread Paul Moore


Paul Moore  added the comment:

On Fri, 5 Jul 2019 at 21:33, Steve Dower  wrote:
> Yeah. We probably need to be clear about which approaches are actually 
> supported and which ones are not.
>
> Ultimately though, I think this is a situation where the breakage is forced 
> upon us (by the operating system), and so fixing the behaviour of our 
> built-in functionality pretty clearly outweight preserving the assumptions 
> made by third-parties (especially since those assumptions are going to break).

Well, given that this is an issue specific to the store distribution
of Python, I'd say that it's the release of the store build that
forces it on us, not the OS, and releasing the store build was our
choice, so if we did so with the store build not meeting existing
implicit (i.e. undocumented, I agree) contracts, then we made that
choice. I think it was the right choice, because the store
distribution is a huge plus, but I do think we should be cautious
about impacting the existing user base simply because of a very new,
and explicitly documented as "still experimental" build. (Yes, I'm
still a bit sore about the hassle involved in getting virtualenv to
work - although the final resolution to that issue was IMO clean and a
huge improvement on the previous status quo, so I'm not *really* sore
:-))

> > IMO, we should bite the bullet here and actually document the
> > interfaces involved in virtual environments, so that tools have a
> > stable contract to work with. That way we'd limit the need for people
> > to rely on undocumented behaviour in the first place.
>
> I feel like I've made this argument before, but I suspect if we'd documented 
> them prior to this change, we'd now have no choice but to remain broken.

Hardly. We would simply have had to document the changed behaviour and
follow our existing transition/compatibility processes. Documenting
things doesn't lock them in stone, it just requires us to give due
consideration to our users when we change things. Conversely, *not*
documenting things makes it easier for us to change things, at the
cost of not giving users supported ways to do certain things that they
may still need to do.

> The interfaces that should be documented are "-m venv", venv.VenvBuilder, and 
> the activation scripts, none of which had to change. But if we start making 
> promises about how "python.exe" actually behaves in a venv then we're going 
> to paint ourselves into a corner very quickly - the venv approach just 
> doesn't scale well to the range of systems people use. (Obligatory mention 
> that PEP 582 would have avoided this whole mess by not requiring us to 
> duplicate python.exe anywhere. But it just wasn't meant to be.)

I assume you're missing my key point here (not unlikely, as I didn't
state it explicitly - I guess I'm relying on undocumented information
myself ;-)). The point here is that venv doesn't exist in a vacuum,
before it existed, virtualenv was the standard way to produce virtual
environments. And because neither virtualenv nor venv document any of
their internal details, they are plagued with incompatibilities, and
work badly together. As a result, tools like pipenv, or pew, or pipx,
can't support both but end up picking just one - and because of
backward compatibility requirements, they pick virtualenv, which has
significantly hindered adoption of venv, which is (for hopefully
obvious reasons) a substantially more robust solution.

It may be that it's just too late to worry about this, and we should
simply accept that tools will be dropping Python 2 support very soon
now, at which point there's no real reason to continue using
virtualenv (apart from no-one having the time to make the change, but
if virtualenv adopts venv as its internal mechanism we can get around
that). But personally, I'd rather not rely on an assumption like that,
just yet (maybe in 3.9?).

(Side issue, I don't really see the relevance of PEP 582 here, as I
don't think it's something that tools like tox and pipenv could
replace virtual environments with - but maybe I'm wrong, if so the
arguments for PEP 582 didn't really get that point across).

--

___
Python tracker 
<https://bugs.python.org/issue37369>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37369] Issue with pip in venv on Powershell in Windows

2019-07-05 Thread Paul Moore


Paul Moore  added the comment:

On Thu, 27 Jun 2019 at 18:08, Steve Dower  wrote:
> However, it does mean that we need to start reporting a sys.executable that 
> is not inside sys.prefix, and I'm not sure whether that is a good idea.
>
> Looking at venv (+Vinay for confirmation here), it seems to treat "home" as 
> "the directory containing the original Python binary" rather than "the 
> directory containing the Python install". So in theory (and in my simple 
> testing), using the sys.executable-not-in-prefix here is fine. But is that 
> intentional?

I'm pretty sure there is code in the wild that expects  /
python.exe to be the Python executable (when you're not in a virtual
environment). I seem to recall that virtualenv might do that sort of
thing. Sorry I can't be precise here, I've just come back from holiday
and won't be back into the swing of open source for a while yet, but
my recollection is that it gets done in situations where
sys.executable isn't (easily) accessible - for example, when getting
sys.prefix from the registry (um, I don't think that's right, as the
executable path is in the registry too - but you get the idea).

What I'd say is that we should be cautious here, as we risk trading
one breakage for another, rather than actually fixing the issue.
Certainly the assumption I quote above is unwarranted, and code making
it can't complain too much if it breaks, but the problem is that you
*have* to make that assumption as the necessary information isn't
available any other way. So we should do some careful testing of 3rd
party tools like virtualenv, pipenv, pipx, pew, tox, etc, to make sure
any "solution" here doesn't break them, and provide or document
transition options for them if necessary (because such tools can't
realistically avoid continuing to support older versions of Python).

IMO, we should bite the bullet here and actually document the
interfaces involved in virtual environments, so that tools have a
stable contract to work with. That way we'd limit the need for people
to rely on undocumented behaviour in the first place.

Paul

--

___
Python tracker 
<https://bugs.python.org/issue37369>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37354] Write PowerShell Activate.ps1 to be static so it can be signed

2019-06-20 Thread Paul Moore


Paul Moore  added the comment:

How will this interact with EnvBuilder.install_scripts() (which explicitly 
states that it performs textual substitution)?

Note that I'm not aware of anyone who actually uses the ability to subclass 
EnvBuilder, but I wouldn't be surprised to find that people do...

--

___
Python tracker 
<https://bugs.python.org/issue37354>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37282] os problems on absolute paths containing unicode characters on windows

2019-06-14 Thread Paul Moore


Paul Moore  added the comment:

This works for me on Python 3.7. Can you confirm if you see the problem on 3.7, 
and if so, provide a more detailed example of how to reproduce?

--

___
Python tracker 
<https://bugs.python.org/issue37282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37164] dict creation with converted zip objects produces inconsistent behavior

2019-06-05 Thread Paul Moore


Paul Moore  added the comment:

Works fine for me:

Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> a = ['1','2','3']
>>> b = [1,2,3]
>>> c = zip(a,b)
>>> print(dict(list(c)))
{'1': 1, '2': 2, '3': 3}
>>> print(dict(list(zip(a,b
{'1': 1, '2': 2, '3': 3}
>>> d = zip(b,a)
>>> print(dict(list(d)))
{1: '1', 2: '2', 3: '3'}

Are you sure you didn't try to use c twice? If you do, it will start up from 
where it left off (at the end) and so generate no further values:

>>> print(dict(list(c)))
{}

You need to remember that c is an iterator, and this is how iterators work. (If 
you're coming from Python 2, this is new behaviour in Python 3).

--
resolution:  -> not a bug

___
Python tracker 
<https://bugs.python.org/issue37164>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37111] Logging - Inconsistent behaviour when handling unicode

2019-06-01 Thread Paul Moore


Paul Moore  added the comment:

> User would like Python logging of Unicode characters to be consistent

It is consistent. The encoding of

logging.basicConfig(filename='c:\\my_log.log')

is consistent with the encoding of

open('c:\\my_log.log')

Both use the system default encoding, which is not UTF-8. There is some 
discussion going on right now, as to whether it *should* be, but it isn't, and 
I wouldn't consider changing the behaviour of loging *without* changing the 
behaviour of open() to be consistent.

Logging to the console is consistent with the standard IO streams, and it was 
PEP 528 (https://www.python.org/dev/peps/pep-0528/) that made that change - 
before that, the standard IO streams, and logging to the console, used the 
console codepage.

So, while I agree that the behaviour takes a bit of work to understand, it's 
not (IMO) inconsistent, nor is it (IMO) a bug.

--

___
Python tracker 
<https://bugs.python.org/issue37111>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36774] f-strings: Add a !d conversion for ease of debugging

2019-05-02 Thread Paul Moore


Paul Moore  added the comment:

> So the question is: how do you get repr by default, but allow the format spec?
>
> The only thing I've come up with is:
> - f"{expr!d}" expands to f"expr={repr(expr)}", but
> - f"{expr!d:spec} expands to f"expr={format(expr, spec)}"
>
> I think this is the most useful version. But is it too complex to explain?

Agreed, this is the most useful version. Not only do I not think it's
too complicated to explain, I actually think it's the obvious
behaviour, and what people would expect even without an explanation.

If asked, I'd explain it as:

f"{expr!d:spec}" expands to "expr=". If ":spec" is omitted, repr() is used.

That seems simple enough to me - the key is that we're just saying "if
:spec is omitted, we use repr".

--

___
Python tracker 
<https://bugs.python.org/issue36774>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36774] f-strings: Add a !d conversion for ease of debugging

2019-05-02 Thread Paul Moore


Paul Moore  added the comment:

+1 from me. It's something I'd find useful, and it's a natural extension of the 
f-string syntax.

> I can't decide if I'm going to allow a format specifier.

The only useful interpretation IMO would be for {expr!d:fmt} to expand to 
expr={expr:fmt}. If you're not willing to include that in the initial 
implementation, I'd rather see :fmt reserved for now, with the intention that 
it's implemented like this at a later date. Having :fmt apply to the whole 
string including the "expr=" bit would be basically useless to me. For a 
motivating example, consider f"{datetime.now()!d:%Y-%m-%d}", which is something 
I could easily imagine using.

Steven D'Aprano:
> I think there are enough use-cases for having access to
> expressions, complete with source code, as first-class
> values to make this a general feature of the language
> and not baked into f-strings. I have a proto-PEP
> discussing this.

I have no problem with something like this, but I don't think it precludes the 
proposed f-string extension. The use cases are sufficiently different that I'd 
expect the two features to live happily together - there's no need to block the 
f-string extension for a proposal like this.

--
nosy: +paul.moore

___
Python tracker 
<https://bugs.python.org/issue36774>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   3   4   5   6   7   >