Re: Compiling python on windows with vs

[Eryk Sun via Python-list]

On 6/15/23, Thomas Schweikle via Python-list  wrote:
>
> No. This flag is not inherited. Someone has to set it for created
> directories. It is easy to confirm: take a directory not under MSYS or
> cygwin control (because it is mounted by MSYS or cygwin), set the flag,
> then create directories. They all will have caseSensitivInfo disabled.

That was how the attribute was implemented initially in Windows 10,
but subsequently it was made inheritable. For example:

C:\Temp\test>mkdir spam
C:\Temp\test>fsutil file setCaseSensitiveInfo spam enable
Error:  Access is denied.

Setting the case-sensitive attribute requires the right to add files
and directories (i.e. "WD" = "write data" / "add file"; "AD" = "append
data" / "add subdirectory") and the right to remove files and
directories (i.e. "DC" = "delete child"). The owner of a directory
doesn't necessarily inherit these rights from the parent directory
(which is my case here), but the owner of any object usually has the
implicit right to modify discretionary security. Let's simply grant
the owner (i.e. "OW" = "owner rights") full control of the directory
(i.e. "F"), inheritable to child directories (i.e. "CI" = "container
inherit").

C:\Temp\test>icacls spam /grant *OW:(CI)(F)
processed file: spam
Successfully processed 1 files; Failed processing 0 files

C:\Temp\test>fsutil file setCaseSensitiveInfo spam enable
Case sensitive attribute on directory C:\Temp\test\spam is enabled.

Now, create a child directory and confirm that it inherits the
case-sensitive flag.

C:\Temp\test>mkdir spam\eggs
C:\Temp\test>fsutil file queryCaseSensitiveInfo spam\eggs
Case sensitive attribute on directory C:\Temp\test\spam\eggs is enabled.

> Python itself isn't the problem here. It is MSBuild.exe. For some reason
> this tool lowercases sometimes whole paths to files included. This does
> not matter if case sensitivity is disabled. It matters if case
> sensitivity is enabled! There is no reason MSBUild.exe does it. But it
> is done for some paths (as someone else pointed out).

For the specific problem you had when building 3.10 and 3.11, it's
actually a bug in Python's source code, which is no longer present in
3.12+. It can be fixed in 3.11, but 3.10 no longer gets bug fixes.
Here's the link to the issue on GitHub:

https://github.com/python/cpython/issues/105737

I encountered a different bug when building the main branch. Building
the _decimal extension module includes an .asm file. The relative path
in the project file has the correct case, but the build system
resolved the fully-qualified path as all lower case. This problem only
occurred for this single file, out of hundreds of relative paths in
the project files, so it's not like the build system is completely
broken when working in case-sensitive directories.

There are probably a few such bugs that need to be fixed in msbuild,
the compiler, and linker. After all, these tools have been developed
and tested for decades on only case-insensitive filesystems. But you
don't have to be on the bleeding edge. There's no reason to make
directories case-sensitive for repositories that are intended for use
on Windows, such as CPython.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compiling python on windows with vs

[Eryk Sun via Python-list]

On 6/15/23, Thomas Schweikle via Python-list  wrote:
>
> In this case: not sure what is going on.

Possibly you have a setting configured that affects the behavior of
Git via the MinGW-w64 runtime, such that calling mkdir() ends up
calling NtSetInformationFile() to set the FileCaseSensitiveInformation
for the directory.

Does the mkdir command in Git bash create a case-sensitive directory?
It doesn't for me. I have to manually enable case sensitivity via
`chattr +C`.

What do you get for `which git` and `git --version`?

$ which git
/mingw64/bin/git

$ git --version
git version 2.41.0.windows.1

> $ fsutil file queryCaseSensitiveInfo .

The MSYS2 environment includes lsattr and chattr commands, with the
case-sensitive flag mapped to "C". It's probably more convenient than
typing `fsutil file queryCaseSensitiveInfo` or `fsutil file
setCaseSensitiveInfo`.

$ lsattr -d test
 test
$ chattr +C test
$ lsattr -d test
---C test

> core.ignorecase is not regarded in any way. It does not mater if it is
> set or not.

Git tests the case-sensitivity of the target directory to configure
core.ignorecase when cloning a repo. If it's case insensitive, then
core.ignorecase is enabled. This overrides the global value. AFAIK,
the ignorecase setting is unrelated to actually setting the case
sensitivity of created directories; it just affects how Git behaves on
a case-insensitive filesystem.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compiling python on windows with vs

[Eryk Sun via Python-list]

On 6/14/23, Inada Naoki via Python-list  wrote:
>> Since Git enables Windows NTFS case sensitivity while checking out sources
>
> I didn't know that. Would you give us a link to this feature?
> As far as I know, `git config core.ignorecase` doesn't mean NTFS case
> sensitive.

If a repo is cloned into a case-insensitive directory, then
core.ignorecase should be enabled automatically. If a repo is cloned
into a case-sensitive directory, then core.ignorecase should not be
enabled automatically.

I searched through relevant issues on the Git for Windows repo on
GitHub, and I found nothing to indicate that a capability to
automatically enable NTFS case sensitivity has been added. I searched
through the source of Git and Git for Windows, and I didn't find any
references to WinAPI SetFileInformationByHandle: FileCaseSensitiveInfo
or NTAPI NtSetInformationFile: FileCaseSensitiveInformation, nor the
use of fsutil file setCaseSensitiveInfo.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compiling python on windows with vs

[Eryk Sun via Python-list]

On 6/13/23, Thomas Schweikle via Python-list  wrote:
>
> Since Git enables Windows NTFS case sensitivity while checking out
> sources ... is it a bug or a "feature"? And: is there a simple

AFAIK the Windows version of Git (you're not using the Linux version
of Git via WSL, right?) does not automatically enable NTFS case
sensitivity. But a newly created directory does inherit the case
sensitivity of its parent directory. Make sure to clone the CPython
repo in a directory that has case sensitivity disabled.

>_testconsole.c
> C:\Users\sct-muc\Documents\Projekte\cpython\PC\_testconsole.c(13,10):
> fatal  error C1083: Datei (Include) kann nicht geöffnet werde
> n: "..\modules\_io\_iomodule.h": No such file or directory
> [C:\Users\sct-muc\Documents\Projekte\cpython\PCbuild\_testconsole.vcxpro
> j]

I just built the main branch in a case sensitive tree. I had no
problem building "_testconsole.c". However, building the _decimal
extension module raised a couple of serious warnings. In
"PCbuild/_decimal.vcxproj", there's an include for
"..\Modules\_decimal\libmpdec\vcdiv64.asm". However, MSBuild resolved
this relative path with all lower-case names, i.e. "modules" instead
of the correct name "Modules", and it incorrectly tried to output
"vcdiv64.obj" in a subdirectory of "pcbuild" instead of the correct
name "PCbuild". This appears to be a bug in MSBuild. A lot of Windows
programs don't handle case-sensitive directories well, including
Python's standard library. It's understandable when comparing paths,
but the behavior in this case is inexcusably bad.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Assistance Request - Issue with Installing 'pip' despite Python 3.10 Installation

[Eryk Sun via Python-list]

On 6/10/23, Thomas Passin via Python-list  wrote:
>
> Yes; I didn't want to get too esoteric with commands that are hard to
> figure out and remember, because then why not use Powershell, whose
> commands are hard to figure out and remember?

Using `dir /s [/ad] [/b] "[path\]pattern"` with a wildcard pattern is
a simple way to recursively search for a filename or directory,
without needing to pipe the output to a findstr/grep/awk command. It's
also fast. Of course, CMD's wildcards aren't nearly as powerful as
regular expressions.

The examples I included with `for` loops in CMD were for completeness
to show how to get the results in a loop variable for further
processing in a batch script. Personally, I use `for` loops a lot even
when working at the command prompt, but I'm a dinosaur in that regard.
Using PowerShell really should be preferred nowadays.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Assistance Request - Issue with Installing 'pip' despite Python 3.10 Installation

[Eryk Sun via Python-list]

On 6/10/23, Thomas Passin via Python-list  wrote:
>
> We can find pip.exe using good old-fashioned dir (we don't need any
> new-fangled Powershell):
>
> C:\Users\tom>dir AppData\Local\Programs\Python /Aa /S /W /B |find
> "pip"|find "Scripts"

CMD's `dir` and `for` commands support simple wildcard matching. For
example, the following recursively searches for a file named
"pip*.exe" under "%ProgramFiles%\Python311":

C:\>dir /b /s "%ProgramFiles%\Python311\pip*.exe"
C:\Program Files\Python311\Scripts\pip.exe
C:\Program Files\Python311\Scripts\pip3.11.exe
C:\Program Files\Python311\Scripts\pip3.exe

C:\>for /r "%ProgramFiles%\Python311" %f in (pip*.exe) do @(echo %f)
C:\Program Files\Python311\Scripts\pip.exe
C:\Program Files\Python311\Scripts\pip3.11.exe
C:\Program Files\Python311\Scripts\pip3.exe

The following recursively searches for a directory named "pip" under
"%ProgramFiles%\Python311:

C:\>dir /ad /b /s "%ProgramFiles%\Python311\pip"
C:\Program Files\Python311\Lib\site-packages\pip

Or search for a directory name that starts with "pip":

C:\>dir /ad /b /s "%ProgramFiles%\Python311\pip*"
C:\Program Files\Python311\Lib\site-packages\pip
C:\Program Files\Python311\Lib\site-packages\pip-22.3.1.dist-info
C:\Program Files\Python311\Lib\site-packages\win32\Demos\pipes

With a recursive `for /r path [/d]` loop, the strings in the set have
to include wildcard characters to actually check for an existing file
or directory, else each string in the set simply gets appended to the
directory names in the recursive walk. For example, the following
recursively searches for a directory (i.e. /d) named "pip*" under
"%ProgramFiles%\Python311":

C:\>for /r "%ProgramFiles%\Python311" /d %d in (pip*) do @(echo %d)
C:\Program Files\Python311\Lib\site-packages\pip
C:\Program Files\Python311\Lib\site-packages\pip-22.3.1.dist-info
C:\Program Files\Python311\Lib\site-packages\win32\Demos\pipes

To match a specific name, you can filter the matches using an `if`
statement to compare the base filename of the loop variable (i.e.
[n]ame + e[x]tension) with the required name. For example:

C:\>for /r "%ProgramFiles%\Python311" /d %d in (pip*) do @(
More? if "%~nxd"=="pip" echo %d)
C:\Program Files\Python311\Lib\site-packages\pip
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Assistance Request - Issue with Installing 'pip' despite Python 3.10 Installation

[Eryk Sun via Python-list]

On 6/8/23, Thomas Passin via Python-list  wrote:
>
> It always gets installed, though.

By default, the option to install pip is enabled. It's implemented by
executing ensurepip after the interpreter is installed. However,
ensurepip may silently fail during installation. As a CPython triager
I've come across this problem a couple of times, but it should be
rare. It can possibly be resolved by manually executing ensurepip via
the following command:

py [-3[.X]] -m ensurepip --default-pip --upgrade --verbose

If Python is installed for all users, the latter should be executed
from a shell that has administrator access. Even if this command also
fails, the verbose output in the console may be helpful to further
diagnose the problem.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Assistance Request - Issue with Installing 'pip' despite Python 3.10 Installation

[Eryk Sun via Python-list]

On 6/7/23, Thomas Passin via Python-list  wrote:
> On 6/7/2023 6:28 PM, Eryk Sun wrote:
>
>> That won't be of any help if pip isn't installed. By default, Python's
>> installer attempts to install pip by running the ensurepip package,
>> but sometimes it fails. It can help to try to manually run ensurepip
>> in the shell. For example:
>>
>>  py -m ensurepip --default-pip --upgrade --verbose
>
> Yes, but why should anyone besides the OP think pip isn't installed? Let
> him try py -m pip.  If pip isn't installed he will see something like

I didn't mean to imply that the OP shouldn't first try to run `py -m
pip` or `py -3.10 -m pip`.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Assistance Request - Issue with Installing 'pip' despite Python 3.10 Installation

[Eryk Sun via Python-list]

On 6/7/23, Thomas Passin via Python-list  wrote:
>
> You have by now seen several responses, and the one most likely to be
> helpful is to run pip with
>
> py -m pip

That won't be of any help if pip isn't installed. By default, Python's
installer attempts to install pip by running the ensurepip package,
but sometimes it fails. It can help to try to manually run ensurepip
in the shell. For example:

py -m ensurepip --default-pip --upgrade --verbose
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does os.path relpath produce an incorrect relative path?

[Eryk Sun]

On 5/26/23, cactus  wrote:
>
> Surprisingly (for me at least) the alternative provided by the pathlib
> module 'relative_to' method doesn't provide for full relative path
> computation.  I was expecting this would offer everything that os.path
> offers but it doesn't in this case.

Starting with Python 3.12, the relative_to() method has a walk_up
parameter. It defaults to False for the sake of backward
compatibility.

https://docs.python.org/3.12/library/pathlib.html#pathlib.PurePath.relative_to
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does os.path relpath produce an incorrect relative path?

[Eryk Sun]

On 5/25/23, BlindAnagram  wrote:
>
> vcx_path = 'C:\\build.vs22\\lib\\lib.vcxproj'
> src_path = 'C:\\lib\\src\\'
> rel_path = '..\\..\\..\\lib\\src'
>
> [snip]
>
> The first of these three results produces an incorrect relative path
> because relpath does not strip off any non-directory tails before
> comparing paths.

The start path is assumed to be a directory, which defaults to the
current working directory, and the input paths are first resolved as
absolute paths. In order to reach src_path from vcx_path, one has to
traverse 3 levels up to the root directory.

https://docs.python.org/3/library/os.path.html#os.path.relpath
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PythonPath / sys.path

[Eryk Sun]

On 5/13/23, Grizzy Adams via Python-list  wrote:
>
> I have tried adding my dir in registry to the existing PythonPath
>
> [HKEY_CURRENT_USER\Software\Python\PythonCore\3.4\PythonPath]
> @="D:\\Shades\\Tools\\Python\\Lib;D:\\Shades\\Tools\\Python\\DLLs"
>
> that did not help,

The default value of the above registry key is only used if Python
can't calculate the default sys.path based on searching for "os.py",
the landmark module in the standard library. This can happen sometimes
when Python is embedded or run from a symlink.

The default value of each subkey of
"[HKCU|HKLM]\Software\Python\PythonCore\\PythonPath", on the
other hand, usually does extend sys.path. Except that running Python
with the -E or -I command-line options (both supported by Python 3.4)
makes the interpreter ignore PYTHON* environment variables, including
PYTHONPATH, as well as registry settings such as subkeys of
"PythonPath".

Even with the -E or -I command-line options, .pth files in the system
site-packages are still processed. A .pth file can be used to extend
sys.path as long as the site module is imported, i.e. as long as the
-S command-line option isn't used. One difference, however, is that
directories in .pth files are appended to sys.path and thus do not
override the standard library.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help on ctypes.POINTER for Python array

[Eryk Sun]

On 5/11/23, Jason Qian via Python-list  wrote:
>
> in the Python, I have a array of string
> var_array=["Opt1=DG","Opt1=DG2"]
> I need to call c library and  pass var_array as parameter
> In the   argtypes,  how do I set up ctypes.POINTER(???)  for var_array?
>
> func.argtypes=[ctypes.c_void_p,ctypes.c_int, ctypes.POINTER()]
>
> In the c code:
> int  func (void* obj, int index,  char** opt)

The argument type is ctypes.POINTER(ctypes.c_char_p), but that's not
sufficient. It doesn't implement converting a list of str objects into
an array of c_char_p pointers that reference byte strings. You could
write a wrapper function that implements the conversion before calling
func(), or you could set the argument type to a custom subclass of
ctypes.POINTER(ctypes.c_char_p) that implements the conversion via the
from_param() class method.

https://docs.python.org/3/library/ctypes.html#ctypes._CData.from_param

Here's an example of the latter.

C library:

#include 

int
func(void *obj, int index, char **opt)
{
int length;
for (length=0; opt[length]; length++);
if (index < 0 || index >= length) {
return -1;
}
return printf("%s\n", opt[index]);
}


Python:

import os
import ctypes

lib = ctypes.CDLL('./lib.so')
BaseOptions = ctypes.POINTER(ctypes.c_char_p)

class Options(BaseOptions):
@classmethod
def from_param(cls, param):
if isinstance(param, list):
new_param = (ctypes.c_char_p * (len(param) + 1))()
for i, p in enumerate(param):
new_param[i] = os.fsencode(p)
param = new_param
return BaseOptions.from_param(param)

lib.func.argtypes = (ctypes.c_void_p, ctypes.c_int, Options)


demo:

>>> opts = ['Opt1=DG', 'Opt1=DG2']
>>> lib.func(None, 0, opts)
Opt1=DG
8
>>> lib.func(None, 1, opts)
Opt1=DG2
9
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Do subprocess.PIPE and subprocess.STDOUT sametime

[Eryk Sun]

On 5/9/23, Thomas Passin  wrote:
>
> I'm not sure if this exactly fits your situation, but if you use
> subprocess with pipes, you can often get a deadlock because the stdout
> (or stderr, I suppose) pipe has a small capacity and fills up quickly
> (at least on Windows),

The pipe size is relatively small on Windows only because
subprocess.Popen uses the default pipe size when it calls WinAPI
CreatePipe(). The default size is 4 KiB, which actually should be big
enough for most cases. If some other pipe size is passed, the value is
"advisory", meaning that it has to be within the allowed range (but
there's no practical limit on the size) and that it gets rounded up to
an allocation boundary (e.g. a multiple of the system's virtual-memory
page size). For example, here's a 256 MiB pipe:

>>> hr, hw = _winapi.CreatePipe(None, 256*1024*1024)
>>> _winapi.WriteFile(hw, b'a' * (256*1024*1024))
(268435456, 0)
>>> data = _winapi.ReadFile(hr, 256*1024*1024)[0]
>>> len(data) == 256*1024*1024
True

> then it blocks until it is emptied by a read.
> But if you aren't polling, you don't know there is something to read so
> the pipe never gets emptied.  And if you don't read it before the pipe
> has filled up, you may lose data.

If there's just one pipe, then there's no potential for deadlock, and
no potential to lose data. If there's a timeout, however, then
communicate() still has to use I/O polling or a thread to avoid
blocking indefinitely in order to honor the timeout.

Note that there's a bug in subprocess on Windows. Popen._communicate()
should create a new thread for each pipe. However, it actually calls
stdin.write() on the current thread, which could block and ignore the
specified timeout. For example, in the following case the timeout of 5
seconds is ignored:

>>> cmd = 'python -c "import time; time.sleep(20)"'
>>> t0 = time.time(); p = subprocess.Popen(cmd, stdin=subprocess.PIPE)
>>> r = p.communicate(b'a'*4097, timeout=5); t1 = time.time() - t0
>>> t1
20.2162926197052

There's a potential for deadlock when two or more pipes are accessed
synchronously by two threads (e.g. one thread in each process). For
example, reading from one of the pipes blocks one of the threads
because the pipe is empty, while at the same time writing to the other
pipe blocks the other thread because the pipe is full. However, there
will be no deadlock if at least one of the threads always polls the
pipes to ensure that they're ready (i.e. data is available to be read,
or at least PIPE_BUF bytes can be written without blocking), which is
how communicate() is implemented on POSIX. Alternatively, one of the
processes can use a separate thread for each pipe, which is how
communicate() is implemented on Windows.

Note that there are problems with the naive implementation of the
reader threads on Windows, in particular if a pipe handle leaks to
descendants of the child process, which prevents the pipe from
closing. A better implementation on Windows would use named pipes
opened in asynchronous mode on the parent side and synchronous mode on
the child side. Just implement a loop that handles I/O completion
using events, APCs, or an I/O completion port.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Christoph Gohlke and compiled packages

[Eryk Sun]

On 4/12/23, Mike Dewhirst  wrote:
>
> Collecting psycopg2==2.9.3

x86 and x64 wheels are available for Python 3.11 if you can use
Psycopg 2 version 2.9.5 or 2.9.6 instead of 2.9.3:

https://pypi.org/project/psycopg2/2.9.5/#files
https://pypi.org/project/psycopg2/2.9.6/#files
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Windows installer from python source code without access to source code

[Eryk Sun]

On 4/6/23, Jim Schwartz  wrote:
> Never mind.  I found it on the web.  I needed to point my PYTHONPATH to
> sitepackages:

In most cases an application should be isolated from PYTHON*
environment variables. If you're creating a Python application or
embedding Python in an application, use the embeddable distribution,
and add any additional required sys.path directories to the included
"._pth" file (e.g. "python311._pth").

https://docs.python.org/3/library/sys_path_init.html#pth-files
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Python-Dev] Small lament...

[Eryk Sun]

On 4/1/23, Skip Montanaro  wrote:
> Just wanted to throw this out there... I lament the loss of waking up on
> April 1st to see a creative April Fool's Day joke on one or both of these
> lists, often from our FLUFL... Maybe such frivolity still happens, just not
> in the Python ecosystem?

I thought this one was funny:

https://github.com/python/cpython/issues/103172
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Windows Gui Frontend

[Eryk Sun]

On 4/1/23, Jim Schwartz  wrote:
> Are there any ide’s that will let me design the screen and convert it to
> python?  I doubt it because it was mentioned that this is time consuming.
>
> Thanks for the responses everyone. I appreciate it.

For Qt, the WYSIWYG UI editor is Qt Designer. The basics are covered
in the following PySide tutorial:

https://www.pythonguis.com/pyside2-tutorial
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Windows Gui Frontend

[Eryk Sun]

On 4/1/23, Jim Schwartz  wrote:
> I have another question.  I have an app written in python, but I want to
> add a windows GUI front end to it.  Can this be done in python?  What
> packages would allow me to do that?

Here are a few of the GUI toolkit libraries in common use:

* tkinter (Tk)
* PyQt (Qt)
* PySide (Qt)
* wxPython (wxWidgets)
* PyGObject (GTK)

tkinter is included in Python's standard library.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python not showing correct version

[Eryk Sun]

On 4/1/23, Barry Scott  wrote:
>
> I find user environment on windows to be less flexible to work with then
> adding a py.ini. On my Windows 11 I added
> %userprofile%\AppData\Local\py.ini.
> To make python 3.8 the default that py.exe uses put this in py.ini:
>
> [defaults]
> python=3.8-64
> python3=3.8-64

Using "py.ini" has the advantage that launcher always reads the file.
The value of the environment variables, on the other hand, may be
stale. If you keep a lot of shells running, it would be tedious to
have to manually update the PY_PYTHON* variables in each shell. That
said, it should be rare that one needs to change the persisted default
versions. For temporary changes, the PY_PYTHON* environment variables
are more flexible and take precedence over "py.ini".

If one doesn't use "py.ini" to set the defaults, it's easy to modify
the persisted user environment using "setx.exe"[^1]. For example:

setx.exe PY_PYTHON 3.8
setx.exe PY_PYTHON3 3.8

setx.exe broadcasts a WM_SETTINGCHANGE "Environment" window message,
which causes Explorer to update its environment. Thus any program run
from Explorer will see the new values. A program launched in a new tab
in Windows Terminal also gets a fresh environment. However, existing
CLI shells (CMD, PowerShell, bash), and programs started by them, will
still have the old environment values. The latter is where using
"py.ini" to set the defaults has the advantage.

---

[^1]: Note that "setx.exe" should never be used to set the persisted
user or machine "Path" value to the current %PATH%. When loading the
environment, the user "Path" gets appended to the machine "Path".
Setting the entire expanded and concatenated value to one or the other
leads to a bloated, redundant PATH value, and it also loses the
flexible configuration based on REG_EXPAND_SZ values.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python not showing correct version

[Eryk Sun]

On 3/31/23, Thomas Passin  wrote:
>
> The store app doesn't install py.exe, does it?

That's a significant downside of the store app. You can install Python
3.7-3.11 from the store, and run them explicitly as "python3.7.exe",
"pip3.7.exe", "python3.11.exe", "pip3.11.exe", etc. But without the
launcher there's no shebang support for scripts, so you can't easily
associate a script with a particular Python version or a particular
virtual environment. Shell shortcuts/links (.LNK files) can work
around the lack of shebang support (add .LNK to the PATHEXT
environment variable). But having to create a separate shell link is
inconvenient, and it's not cross-platform.

The OP installed the standard Python 3.8 distribution, which does
install the launcher by default. The launcher can run all installed
versions, including store app installations. By default it runs the
highest available version, which will probably be the 3.10 store app
in the OP's case. To make 3.8 the default without having to remove
3.10, set the environment variables "PY_PYTHON=3.8" and
"PY_PYTHON3=3.8" in the user environment.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python not showing correct version

[Eryk Sun]

On 3/31/23, Sumeet Firodia  wrote:
>
> One more thing is that pip --version also refers to python 3.10
>
> C:\Users\admin>pip --version
> pip 23.0.1 from
> C:\Users\admin\AppData\Local\Packages
> \PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0
> \LocalCache\local-packages\Python310\site-packages\pip
> (python 3.10)

You're running pip from the store app distribution of Python 3.10.
Your settings have the store app set as the default for the "python"
and "pip" commands. To change this, open the system settings for "App
execution aliases" and disable the "python.exe", "pythonw.exe", and
"pip.exe" aliases. You might have to also disable the alias for
"python3.exe" if it's causing problems (e.g. if you don't want a
shebang like "#!/usr/bin/env python3" to run the store app). It
shouldn't cause problems to leave the "python3.10.exe" and
"pip3.10.exe" aliases enabled.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Windows installer from python source code without access to source code

[Eryk Sun]

On 3/31/23, Jim Schwartz  wrote:
> I want a windows installer to install my application that's written in
> python, but I don't want the end user to have access to my source code.

Cython can compile a script to C source code for a module or
executable (--embed). The source can be compiled and linked normally.
For example, the following builds a "hello.exe" executable based on a
"hello.py" script.

> cython -3 --embed hello.py
> set "PYI=C:\Program Files\Python311\include"
> set "PYL=C:\Program Files\Python311\libs"
> cl /I"%PYI%" hello.c /link /libpath:"%PYL%"
> copy hello.exe embed
> embed\hello.exe
Hello, World!

I extracted the complete embeddable distribution of Python 3.11 into
the "embed" directory. You can reduce the size of the installation, if
needed, by minimizing the zipped standard library and removing pyd
extensions and DLLs that your application doesn't use.

The generated "hello.c" is large and not particularly easy to read,
but here are some snippets [...]:

[...]
/* Implementation of 'hello' */
static PyObject *__pyx_builtin_print;
static const char __pyx_k_main[] = "__main__";
static const char __pyx_k_name[] = "__name__";
static const char __pyx_k_test[] = "__test__";
static const char __pyx_k_print[] = "print";
static const char __pyx_k_Hello_World[] = "Hello, World!";
[...]
  /* "hello.py":1
 * print("Hello, World!") # <<
 */
  __pyx_tuple_ = PyTuple_Pack(1, __pyx_kp_u_Hello_World);
if (unlikely(!__pyx_tuple_)) __PYX_ERR(0, 1, __pyx_L1_error)
[...]
  /* "hello.py":1
 * print("Hello, World!") # <<
 */
  __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_print, __pyx_tuple_,
  NULL);
if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 1, __pyx_L1_error)
[...]
int wmain(int argc, wchar_t **argv) {
[...]
if (argc && argv)
Py_SetProgramName(argv[0]);
Py_Initialize();
if (argc && argv)
PySys_SetArgv(argc, argv);
[...]
  m = PyInit_hello();
[...]
if (Py_FinalizeEx() < 0)
return 2;
[...]
return 0;
[...]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ole version set as default

[Eryk Sun]

On 3/29/23, Pranav Bhardwaj  wrote:
>I am Pranav Bhardwaj and I stuck in a problem. My problem is
> that in my system I have python 3.11.2 but when I type python in my command
> prompt, my command prompt show that python version 2.7.13 as a default.

Run the following command in the shell

where.exe python

Note the first directory where "python" is found. Edit both the system
and the user environment variables to remove that directory from both
the system and user "Path" environment variable. Open a new shell from
Explorer, and confirm that "python" no longer runs Python 2. If
"python" still runs Python 2, repeat this procedure.

Next run the Python 3.11 installer again to modify the installation.
On the second page, enable the option to add Python to the environment
variables, and click "install". After it's done, open a new shell.
Running "python" should start Python 3.11. Also "pip" should work. You
should also be able to run Python using the "py" launcher and pip via
"py -m pip".

To ensure that that .py file association is correct, open the system
settings app and navigate to "Apps" -> "Default Apps".  Enter ".py"
and ensure that it's associated with the Python app that has a rocket
on its icon (i.e. Python's "py" launcher). Repeat this for ".pyw",
".pyz", and ".pyc".
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Baffled by readline module

[Eryk Sun]

On 3/9/23, Greg Ewing via Python-list  wrote:
> On 10/03/23 4:00 pm, 2qdxy4rzwzuui...@potatochowder.com wrote:
>> My ~/.pythonrc contains the following:
>>
>>  import readline
>>  import rlcompleter
>>  readline.parse_and_bind( 'tab: complete' )
>
> I don't have a ~/.pythonrc, so that's not what's doing it
> for me.

If it's available, the readline and rlcompleter modules are imported
if stdin is a tty, [I]solated mode isn't enabled, and either [i]nspect
is enabled or the REPL is being run. For example:

$ python -c "import sys;print('readline' in sys.modules)"
False

$ python -ic "import sys;print('readline' in sys.modules)"
True
>>>

$ python -ic "import sys;print('readline' in sys.modules)" 0>>

$ python -Iic "import sys;print('readline' in sys.modules)"
False
>>>

This is determined by the following function in "Modules/main.c:"

pymain_import_readline(const PyConfig *config)
{
if (config->isolated) {
return;
}
if (!config->inspect && config_run_code(config)) {
return;
}
if (!isatty(fileno(stdin))) {
return;
}

PyObject *mod = PyImport_ImportModule("readline");
if (mod == NULL) {
PyErr_Clear();
}
else {
Py_DECREF(mod);
}
mod = PyImport_ImportModule("rlcompleter");
if (mod == NULL) {
PyErr_Clear();
}
else {
Py_DECREF(mod);
}
}

The hook for input() is handled by PyOS_Readline() in
"Parser/myreadline.c". It depends on the exported function pointer
PyOS_ReadlineFunctionPointer. When imported, the readline extension
module in "Modules/readline.c" sets PyOS_ReadlineFunctionPointer to
its call_readline() function.

---

On Windows, Python is not distributed with a readline module. Instead,
Python relies on the console host's builtin support for command-line
editing, history, and aliases.

The Windows console API function ReadConsoleW() has a parameter that
supports implementing custom command-line completion. This is how the
CMD shell implements tab completion for file paths. However, no one
has developed a version of rlcompleter for the Windows console.
Instead, there's a third-party "pyreadline3" package (note the "3";
don't install "pyreadline" by mistake) that implements readline for
the Windows console, which supports rlcompleter.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Bug 3.11.x behavioral, open file buffers not flushed til file closed.

[Eryk Sun]

On 3/5/23, aapost  wrote:
>
> If a file is still open, even if all the operations on the file have
> ceased for a time, the tail of the written operation data does not get
> flushed to the file until close is issued and the file closes cleanly.

This is normal behavior for buffered file I/O. There's no timer set to
flush the buffer after operations have "ceased for a time". It
automatically flushes only when the buffer is full or, for line
buffering, when a newline is written.

The default buffer size is based on the raw file object's _blksize
attribute. If st_blksize can't be determined via fstat(), the default
_blksize is 8 KiB.

Here's an example on Linux. In this example, the buffer size is 4 KiB.

>>> f = open('abc', 'w')
>>> os.fstat(f.fileno()).st_blksize
4096
>>> f.buffer.raw._blksize
4096
>>> f.writelines(f'{i}\n' for i in range(5))
>>> with open('abc') as g: g.readlines()[-1]
...
'49626\n'

>>> pre_flush_size = os.path.getsize('abc')
>>> f.flush()
>>> post_flush_size = os.path.getsize('abc')
>>> post_flush_size - pre_flush_size
2238

Verify that this makes sense, based on what was left in the buffer
prior to flushing:

>>> remaining_lines = 5 - 49626 - 1
>>> bytes_per_line = 6
>>> remaining_lines * bytes_per_line
2238
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python not found

[Eryk Sun]

On 1/27/23, Bela Gesztesi  wrote:
>
> I'm not that familiar with the steps to be taken.
>
> How do I find the app version of Python for my desktop?
> or
> I don't know how to disable the "python.exe" and "python3.exe" app aliases

To install the app version, run "python3" from the command line. This
will open the Microsoft Store to install the Python app. (If it
doesn't, simply open the store from the start menu.) It may not open
to the latest version of Python, which currently is 3.11.1. Search for
"python" to find the latest version. Make sure the package is from
"Python Software Foundation". Click the "Get" button.

OR

To disable the aliases for the default installer app, begin by
right-clicking the start button and select "settings". Click on "Apps"
in the sidebar. Then click on "Apps & Features" in the window. Click
the drop-down arrow on "More settings". Click on "App execution
aliases". Disable the aliases for "App Installer / python.exe" and
"App Installer / python3.exe".

Some of the above names will be localized to your preferred user
interface language. Hopefully the translations are obvious.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Module use of python3_d.dll conflicts with this version of Python

[Eryk Sun]

On 1/26/23, Olivier B.  wrote:
>
> Does someone know why it would have been chosen to be different for
> debug builds?

It's assumed that a debug build would normally link with
"pythonXY_d.dll". Maybe it should be more defensive. Refer to the
following setup in PC/pyconfig.h:

/* For an MSVC DLL, we can nominate the .lib files used by extensions */
#ifdef MS_COREDLL
#   if !defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_BUILTIN)
/* not building the core - must be an ext */
#   if defined(_MSC_VER)
/* So MSVC users need not specify the .lib
file in their Makefile (other compilers are
generally taken care of by distutils.) */
#   if defined(_DEBUG)
#   pragma comment(lib,"python312_d.lib")
#   elif defined(Py_LIMITED_API)
#   pragma comment(lib,"python3.lib")
#   else
#   pragma comment(lib,"python312.lib")
#   endif /* _DEBUG */
#   endif /* _MSC_VER */
#   endif /* Py_BUILD_CORE */
#endif /* MS_COREDLL */
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python not found

[Eryk Sun]

On 1/26/23, Bela Gesztesi  wrote:
>
> C:\DJI>py comm_og_service_tool.py WM231 --port COM3 GimbalCalib JointCoarse
>
> Python was not found; run without arguments to install from the Microsoft
> Store, or disable this shortcut from Settings > Manage App Execution
> Aliases.

Do what it's telling you to do. Either install the app version of
Python, or disable the "python.exe" and "python3.exe"  app aliases.

The "python.exe" and "python3.exe" app aliases that are distributed
with Windows run a "PythonRedirector" app that just opens the
Microsoft Store to download the latest version of the app version of
Python. When run with command-line arguments, they print the message
that's quoted above.

What's happening here is that the py launcher reads a shebang line
from "comm_og_service_tool.py" of the form "#!/usr/bin/env python3".
It thus searches PATH for "python3.exe" and finds the app alias. We
should probably update the launcher to ignore an alias to the
"PythonRedirector" app.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: IDLE "Codepage" Switching?

[Eryk Sun]

On 1/17/23, Stephen Tucker  wrote:
>
> 1. Can anybody explain the behaviour in IDLE (Python version 2.7.10)
> reported below? (It seems that the way it renders a given sequence of bytes
> depends on the sequence.)

In 2.x, IDLE tries to decode a byte string via unicode() before
writing to the Tk text widget. However, if the locale encoding (e.g.
the process ANSI code page) fails to decode one or more characters,
IDLE lets Tk figure out how to decode the byte string.

Python 2.7 has an older version of Tk that has peculiar behavior on
Windows when bytes in the range 0x80-0xBF are written to a text box.
Bytes in this range get translated to native wide characters (16-bit
characters) in the halfwidth/fullwidth Unicode block, i.e. translated
to Unicode U+FF80 - U+FFBF.

If IDLE decodes using code page 1252, then the ordinals 0x81, 0x8d,
0x8f, 0x90 and 0x9d can't be decoded. IDLE thus passes the undecoded
byte string to Tk. The example you provided that demonstrates the
behavior contains ordinal 0x9d (157).

I get similar behavior for the other undefined ordinal values in code
page 1252. For example, using IDLE 2.7.18 on Windows:

>>> print '\x81\xa1'
ﾁﾡ
>>> print 'a\xa1'
a¡

In the first case, ordinal 0x81 causes decoding to fail in IDLE, so
the byte string is passed as is to Tk, which maps it to
'\uff81\uffa1'. In the second case, OTOH, "\xa1" is decoded by IDLE as
"¡".

> 2. Does the IDLE in Python 3.x behave the same way?

No, in 3.x only Unicode str() objects are written to the Tk text
widget. Moreover, the text widget doesn't have the same behavior in
newer versions. It ignores bytes in the control-block range 0x80-0x9F,
and it decodes bytes in the range 0xA0-0xBF normally.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: No module named 'playsound'‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏

[Eryk Sun]

On 1/9/23, MRAB  wrote:
>
> On Windows it's best to use pip via the Python Launcher:
>
> py -m pip show playsound

Python's app distribution on the Microsoft Store doesn't include the
py launcher, and we don't (but should) have a standalone app or
desktop version of the launcher. Unlike the desktop distribution,
however, the app distribution installs a versioned name as a link,
such as "python3.11". (Note that a venv virtual environment only has a
"python" command on Windows.)

If "pip" is in PATH, a versioned name such as "pip3.11" should also be
available. If multiple versions of the app distribution are installed,
and for some reason the "python" and "pip" links are mapped to
different versions, one can match up the versioned names "pip3.11" and
"python3.11" to ensure consistency. Or simply run "python -m pip".
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: subprocess equivalent for "os.execvp()"?

[Eryk Sun]

On 1/9/23, c.bu...@posteo.jp  wrote:
>
> On Python for Windows what is the appropriate way how a process can call
> itself again?
>
> Let me give you an example [1]:
> There is a project "bitcli" having two entry points
>
> [project.scripts]
> bitcli = "bitcli.__main__:main"
> bitcli-root = "bitcli.__main__:run_main_as_root_via_policykit"
>
> The first is usual.
>
> But the second does call "bitcli" via "pkexec" to give it some root
> rights.
>
> This application is intended to be run as user or root by the user
> himself.
>
>  def run_main_as_root_via_policykit():
>  cmd = ['pkexec', '--disable-internal-agent', 'bitcli']
>
>  # See https://github.com/python/cpython/issues/39569
>  os.execvp(cmd[0], cmd)

The nearest equivalent on Windows, if the current process doesn't have
administrator access and high integrity level, would be to spawn a
process with administrator access and elevated integrity level by
calling ShellExecuteExW() with the "runas" operation. The original
process should wait on the spawned process and proxy its exit status.
It should also add the spawned process to a kill-on-close,
silent-breakaway job, such that terminating the original process also
terminates the spawned process.

The standard library doesn't support calling ShellExecuteExW() or
working with job objects, except via ctypes. Or use the PyWin32
package.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: subprocess equivalent for "os.execvp()"?

[Eryk Sun]

On 1/8/23, c.bu...@posteo.jp  wrote:
>
> is there an equivalent in the subprocess module for "os.execvp()" to
> replace the current process with the new called one?

A note for Windows users

Avoid using any of the `os.exec*` functions on Windows. There's no
support for replacing a Windows process image, so the `exec*()`
functions simply spawn a child process and terminate the current one.
This is a mess in general because a waiting parent isn't aware of the
spawned descendant. It's particularly bad for a console process that
inherits the console session, since two processes will try to use
console I/O simultaneously.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: No solution for "--verbose" (on stdout) output in Pythonds standard library?

[Eryk Sun]

On 1/4/23, c.bu...@posteo.jp  wrote:
>
> often seen "--verbose" switch in command line applications should
> affect only the messages given to the users. This means messages on
> "stdout". That is what this question is about.

Is this additional context information such as help and definitions?
If it's anything to do with diagnostics and telemetry, then logging to
stderr is normal. In my experience the verbosity level typically
applies to the latter. I might step it up as I delve deeper into a
problem and need to examine specific details.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Eryk Sun]

On 1/3/23, Chris Angelico  wrote:
>
> FDs can also be buffered. If it's buffering you want to avoid, don't
> mess around with exactly which one you're writing to, just flush.

I meant to flush a C FILE stream or Python file before writing
directly to the file descriptor, in order to avoid out-of-sequence and
interlaced writes that make no sense. Any OS buffering on the file
doesn't matter in this regard.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Eryk Sun]

On 1/3/23, Grant Edwards  wrote:
>
> That's definitely a better option, but I'm pretty sure I've seen
> multiple examples over the decades where fd 0 was used instead. Was
> /dev/tty a "recent" enough invention that I would have seen
> application code that was written before it existed? Or maybe I was
> just looking at code written by people who weren't aware of /dev/tty?

POSIX has required "/dev/tty" for 20 years, or longer. It's in the
2004 edition of the spec, which was a minor update to the 2001
edition.

https://pubs.opengroup.org/onlinepubs/95399/basedefs/xbd_chap10.html

> No, the whole point is _not_ to write to stdout (which would corrupt
> the program's output). The way I remember it was that you wrote the
> prompt to fd 0, and then read the password from fd 0. That way it
> worked even when fd 1 (stdout) was redirected. It still failed if
> stdin was redirected...

FYI, that wouldn't work on Windows. The "\\.\CONIN$" and "\\.\CONOUT$"
files can be opened with read-write access, but it's for a different
purpose.

A console input buffer can be read from via read() or WinAPI
ReadFile(). It can also be read from using IOCTLs that work with
16-bit wide-character strings, which is the basis of WinAPI
ReadConsoleW() and ReadConsoleInputW(). A console input buffer can be
*written to* via the IOCTL-based function WriteConsoleInputW().

A console screen buffer can be written to via write() or WinAPI
WriteFile(). It can also be written to using IOCTLs that work with
16-bit wide-character strings, which is the basis of WriteConsoleW()
and WriteConsoleOutputW(). A console screen buffer can be *read from*
via the IOCTL-based function ReadConsoleOutputW().
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Eryk Sun]

On 1/3/23, Chris Angelico  wrote:
>
> writing the FD is the same as using stdout

Except stdout may be buffered. One should probably flush the buffer
before each raw write to the file descriptor.

> use /dev/tty to reach for the console directly.

On Windows, open "CON" (read or write), "CONIN$" (read-write), or
"CONOUT$" (read-write). Or use the more explicit device paths
"\\.\CON", "\\.\CONIN$", and "\\.\CONOUT$". If the process has no
console, one can call WinAPI AllocConsole() or
AttachConsole(process_id) to obtain one.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Eryk Sun]

On 1/3/23, Weatherby,Gerard  wrote:
> If sys.stdout is a tty, it typically flushes on newline. e. g.

Sorry if I wasn't clear. Yes, a tty file will be buffered, but it's
line buffering, which isn't an issue as long as lines are written to
stdout. I was referring to full buffering of pipe and disk files. This
can be a problem when printing diagnostics. We went the information
written to the file immediately, not whenever the buffer happens to
fill up. Thus sys.stderr is unbuffered.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Eryk Sun]

On 1/3/23, c.bu...@posteo.jp  wrote:
>
> If the user request the usage info via "-h" it goes to stdout.

The standard file for application output is sys.stdout. Note that
sys.stdout may be buffered, particularly if it isn't a tty. When a
file is buffered, writes are aggregated and only written to the OS
file when the buffer fills up or is manually flushed.

> Why does logging behave different? DEBUG and INFO imho should go to
> stdout not stderr.

The standard file for error messages and other diagnostic information
is sys.stderr. This file should never be buffered.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: Installation hell

[Eryk Sun]

On 12/19/22, Chris Angelico  wrote:
> On Tue, 20 Dec 2022 at 09:12, Thomas Passin  wrote:
>
>> @echo off
>> setlocal
>> : Find effective drive for this file.
>> set ed=%~d0
>> path %ed%\python37\Scripts;%ed%\python37;%PATH%

For reference, in case not everyone on the list knows what "%~d0"
means, the CMD shell supports extracting the drive (d), path (p), name
(n), and extension (x) components of a path name that's stored in a
parameter such as "%0". The full path (f) is resolved beforehand. For
example:

C:\Temp>set var=spam\eggs.py

C:\Temp>for %c in (%var%) do @echo drive: "%~dc"
drive: "C:"

C:\Temp>for %c in (%var%) do @echo path: "%~pc"
path: "\Temp\spam\"

C:\Temp>for %c in (%var%) do @echo name: "%~nc"
name: "eggs"

C:\Temp>for %c in (%var%) do @echo extension: "%~xc"
extension: ".py"

C:\Temp>for %c in (%var%) do @echo full path: "%~dpnxc"
full path: "C:\Temp\spam\eggs.py"

C:\Temp>for %c in (%var%) do @echo full path: "%~fc"
full path: "C:\Temp\spam\eggs.py"

 > So much easier to do on a Unix-like system, where you don't need to
> concern yourself with "effective drive" and can simply use relative
> paths.

A relative path in the PATH environment variable would depend on the
current working directory. Surely the added paths need to be absolute.
However, Thomas didn't have to reference the drive explicitly. The
expression "%~dp0" is the fully-qualified directory of the executing
batch script, and an absolute path can reference its ancestor
directories using ".." components.

> I know we're not here to bash Windows, but... drive letters
> really need to just die already.

I don't foresee drive-letter names getting phased out of Windows. And
Windows itself is unlikely to get phased out as long as Microsoft
continues to profit from it, as it has for the past 37 years.

The drive concept is deeply ingrained in the design of NT, the Windows
API, shells, and applications. While assigning drive names "A:", "B:",
and "D:" to "Z:" can be avoided, the system volume, i.e. drive "C:",
still has to be accessed in the normal way, or using another one of
its persistent names, such as r"\\?\BootPartition".

The latter still uses the filesystem mount point on the root path of
the device (e.g. "?\\BootPartition\\"), which you probably take
issue with. That's a deeply ingrained aspect of Windows. Even mount
points set on filesystem directories are actually bind mount points
that ultimately resolve to the root path on the volume device (e.g.
"\\Device\\HarddiskVolume4\\").  This differs from how regular mount
points work on Unix, for which a path like "/dev/sda1/etc" is
gibberish.

Below I've outlined the underlying details of how logical drives (e.g.
"C:"), UNC shares (e.g. r"\\server\share"), other device names, and
filesystem mount points are implemented on NT.

---

NT Device Names

In contrast to Unix, NT is organized around an object namespace, not a
root filesystem. Instances of many object types can be named. Some
named object types also support a parse routine for paths in the
namespace of an object (e.g. the configuration manager's registry
"Key" type and the I/O manager's "Device" type).

The object manager uses two object types to define the object
namespace: Directory and Symbolic Link. Directory objects form the
hierarchical tree. At the base of the tree is the anonymous root
directory object (i.e. "\\"). A directory is implemented as a hashmap
of named objects. A directory can be set as the shadow of another
directory, creating a union directory for name lookups.

Unless otherwise stated, the following discussion uses "directory" and
"symlink" to refer to a directory object and a symbolic-link object,
respectively -- not to a filesystem directory or filesystem symlink.

A canonical NT device name (e.g. "C:", "PIPE", "UNC") is implemented
in the object namespace as a symlink that targets the path of a real
device object. The real device is typically in the r"\Device"
directory. A canonical device name might be a persistent name for an
enumerated device (e.g. "C:" -> r"\Device\HarddiskVolume2"). In some
cases the real device name is persistent, but it's different from the
canonical name (e.g. "PIPE" -> r"\Device\NamedPipe", or "UNC" ->
r"\Device\Mup").

The symlink that implements a canonical device name is created either
in the r"\Global??" directory or in a directory that's used for local
device names in a given logon session (e.g.
r"\Sessions\0\DosDevices\"). The global device
directory generally contains system devices. Mapped drives and
substitute drives typically use a local device directory, so users
don't have to worry about conflicting drive assignments in these
cases.

The global device directory is the shadow of each local device
directory, forming a union for name lookups. If the same device name
is defined in both the local and global directories, the local device
name takes precedence. However, each local device directory also
contains 

Re: How to enter escape character in a positional string argument from the command line?

[Eryk Sun]

On 12/19/22, Jach Feng  wrote:
>
> That's really good for Linux user! How about Windows?

In CMD, typing the "^" escape character at the end of a line ignores
the newline and prompts for "more" input. If you press enter again,
you'll get another "more" prompt in which you can write the rest of
the command line. Command-line arguments are separated by spaces, so
you have to start the next line with a space if you want it to be a
new argument. Also, "^" is a literal character when it's in a
double-quoted string, which requires careful use of quotes. For
example:

C:\>py -c "import sys; print(sys.orig_argv[3:])" spam^
More?
More?  eggs^
More?
More? " and spam"
['spam\n', 'eggs\n and spam']

The above is easier in PowerShell, which supports entering multiline
strings without having to escape the newline. The second-level prompt
is ">> ". For example:

> py -c "import sys; print(sys.orig_argv[3:])" spam"
>> " eggs"
>>  and spam"
['spam\n', 'eggs\n and spam']
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Installation hell

[Eryk Sun]

On 12/18/22, Jim Lewis  wrote:
>
> Sometimes it's a path issue.

For whatever reason, Python installations on Windows lack versioned
executable names (except for the Microsoft Store distribution). Thus
adding multiple installations to PATH doesn't help unless "python.exe"
is manually linked or copied to versioned names, e.g. "python3.11.exe"
-> "python.exe" and "python3.exe" -> "python.exe". In this case, the
first installation in PATH is the default for running "python" and
"python3".

Using the "py.exe" launcher is the more general and preferred
solution. It allows running any registered Python installation. It's
also the installed handler for the ".py" file type, and it supports
Unix-style shebangs.

> Or exe naming confusion: python, python3, phthon311, etc.

I don't understand what's supposedly confusing here. Here are some
commands to help discover what "py" or "python" commands and
installations are available.

* CMD: where py*.exe
* PowerShell: get-command py*.exe
* Python launcher: py --list-paths

> Or library compatibility issues - took an hour to find out that
> pygame does not work with the current version of python.

It should be a trivial task to discover that wheel packages for pygame
aren't currently available for Python 3.11. It is not unreasonable to
expect Python developers to familiarize themselves with pip and PyPI.

If you search a bit deeper, you'll find a site with unofficial Windows
builds of many packages, including pygame for Python 3.11:

https://www.lfd.uci.edu/~gohlke/pythonlibs/#pygame

Otherwise, while building some packages from source can be quite
involved and difficult, I'd expect anyone with a degree in computer
science to be able to build pygame if necessary. They even provide a
simple guide:

https://www.pygame.org/wiki/CompileWindows

> Then the kludgy PIP app and using a DOS box under Windows with
> command prompts which is ridiculous.

How is pip "kludgy" (i.e. sloppy, hasty, shoddy, or inelegant)? How is
using a command-line interface "ridiculous"? Many programmers and
system administrators actually prefer using command-line interfaces
and text user interfaces (TUI) for many if not most development and
administration tasks. It's a matter of opinion.

---

Nerdy operating system discussion...

> using a DOS box under Windows

The term "DOS box" refers to a virtual machine running 16-bit MS-DOS
in virtual 8086 (v86) mode under Windows 3.1 or Windows 9x, with the
MS-DOS keyboard and display drivers hooked to redirect I/O to a
desktop window. There is no "DOS box" in Windows NT systems. Windows
client systems switched to NT starting with Windows XP. Thus the term
"DOS box" is about two decades out of date.

Also, "DOS" is not synonymous with a command-line interface shell
(e.g. the 16-bit MS-DOS shell, "COMMAND.COM"). A "Disk Operating
System" is one that supports disk drives and filesystems, which
includes most operating systems from the late 1960s onward (e.g.
DOS/360 from 1966). There were several DOS systems for personal
computers in the 1980s, such as Apple ProDOS, Atari DOS, Tandy TRSDOS,
Commodore DOS, and Microsoft MS-DOS. A DOS system can use a graphical
shell, such as running Windows 1.0 (1985) on MS-DOS.

The "python.exe" executable is a Windows application that's flagged to
require a console session. If it doesn't inherit a console session,
then the system allocates one automatically. However, any Windows
application can allocate, attach to, and detach from a console session
via AllocConsole(), AttachConsole(), and FreeConsole().

Prior to Windows 7, each console session in the current NT session is
hosted on threads in the subsystem process for Windows, "csrss.exe".
Starting with Windows 7, console sessions are hosted by instances of
"conhost.exe". The console host also implements a builtin terminal
window for command-line interface (CLI) and text user interface (TUI)
applications. (A console session can be allocated without a terminal
window if a console application is spawned with the creation flag
CREATE_NO_WINDOW.)

Prior to Windows 10, the connection between a console session and its
terminal is hardwired in the console host. This makes it difficult to
implement an alternate terminal, though some alternate terminals have
managed to do so in creative ways (e.g. ConEmu). Starting with Windows
10, alternate terminals can use an open source implementation of the
console host, named "openconsole.exe". The most important example is
Microsoft's "Windows Terminal". Starting with Windows 11, an alternate
terminal can also register as the default terminal for console
applications. For example, if Windows Terminal is set as the default
terminal, then running "python.exe" from Explorer opens a new tab in
Terminal.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: unable to resolve readline issues

[Eryk Sun]

On 12/2/22, biglee12...@gmail.com  wrote:
>
> From this point on Python became unusable as I uninstalled rebooted then
> reinstalled to find I have the same issues as stated.  Finally uninstalled
> Python as it doesn't perform as usual especially trying to understand the
> use of pyreadline, gnureadline and or just readline.

When Python runs interactively, it implicitly tries to import the
readline module. On POSIX, Python has a builtin readline module that
usually uses the GNU Readline library.

On Windows, Python does not include a readline module. Instead, if
standard I/O is a console, the high-level WinAPI ReadConsoleW()
function is used, which implements its own line editor and
command-line history. It's not as general, flexible, or capable as the
readline interface, so a third-party pyreadline package was
implemented for Windows. However, as far as I know, pyreadline is no
longer actively developed. Thus it has out-of-date code, which may be
broken in newer releases of Python, such as isinstance(x,
collections.Callable).

Your choice is to either patch pyreadline to fix the bug or uninstall it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.7+ cannot print unicode characters when output is redirected to file - is this a bug?

[Eryk Sun]

On 11/13/22, Jessica Smith <12jessicasmit...@gmail.com> wrote:
> Consider the following code ran in Powershell or cmd.exe:
>
> $ python -c "print('└')"
> └
>
> $ python -c "print('└')" > test_file.txt
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "C:\Program Files\Python38\lib\encodings\cp1252.py", line 19, in
> encode
> return codecs.charmap_encode(input,self.errors,encoding_table)[0]
> UnicodeEncodeError: 'charmap' codec can't encode character '\u2514' in
> position 0: character maps to 

If your applications and existing data files are compatible with using
UTF-8, then in Windows 10+ you can modify the administrative regional
settings in the control panel to force using UTF-8. In this case,
GetACP() and GetOEMCP() will return CP_UTF8 (65001), and the reserved
code page constants CP_ACP (0),  CP_OEMCP (1), CP_MACCP (2), and
CP_THREAD_ACP (3) will use CP_UTF8.

You can override this on a per-application basis via the
ActiveCodePage setting in the manifest:

https://learn.microsoft.com/en-us/windows/win32/sbscs/application-manifests#activecodepage

In Windows 10, this setting only supports "UTF-8". In Windows 11, it
also supports "legacy" to allow old applications to run on a system
that's configured to use UTF-8.  Setting an explicit locale is also
supported in Windows 11, such as "en-US", with fallback to UTF-8 if
the given locale has no legacy code page.

Note that setting the system to use UTF-8 also affects the host
process for console sessions (i.e. conhost.exe or openconsole.exe),
since it defaults to using the OEM code page (UTF-8 in this case).
Unfortunately, a legacy read from the console host does not support
reading non-ASCII text as UTF-8. For example:

>>> os.read(0, 6)
SPĀM
b'SP\x00M\r\n'

This is a trivial bug in the console host, which stems from the fact
that UTF-8 is a multibyte encoding (1-4 bytes per code), but for some
reason the console team at Microsoft still hasn't fixed it. You can
use chcp.com to set the console's input and output code pages to
something other than UTF-8 if you have to read non-ASCII input in a
legacy console app. By default, this problem doesn't affect Python's
sys.stdin, which internally uses wide-character ReadConsoleW() with
the system's native text encoding, UTF-16LE.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Problems with IDLE in Windows 8.1 and installer x86 Version 3.10.8

[Eryk Sun]

On 11/12/22, darkst...@o2online.de  wrote:
>
 import _tkinter
> Traceback (most recent call last):
>   File "", line 1, in 
> ImportError: DLL load failed while importing _tkinter: Das angegebene Modul
> wurd
> e nicht gefunden.

Loading the extension module "_tkinter.pyd" tries to load two TCL/Tk
DLL files that should be in the same directory: "tcl86t.dll" and
"tk86t.dll". Previously I asked you to look for these two files, and
you said they were there, but maybe one is corrupt.

Please try the following in the interactive shell:

import os, sys, ctypes
tcl86 = os.path.join(sys.prefix, 'DLLs', 'tcl86t.dll')
tk86 = os.path.join(sys.prefix, 'DLLs', 'tk86t.dll')

Run the following two statements one after the other in the shell:

ctypes.CDLL(tcl86)
ctypes.CDLL(tk86)

Either or both will fail if the DLL or one of its dependencies can't
be found or loaded.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange UnicodeEncodeError in Windows image on Azure DevOps and Github

[Eryk Sun]

On 11/11/22, 12Jessicasmith34 <12jessicasmit...@gmail.com> wrote:
>
> any idea why this would be happening in this situation? AFAIK, stdout
> *is* a console when these images are running the python process.

If sys.std* are console files, then in Python 3.6+,
sys.std*.buffer.raw will be _io._WindowsConsoleIO. The latter presents
itself to Python code as a UTF-8 file stream, but internally it uses
UTF-16LE with the wide-character API functions ReadConsoleW() and
WriteConsoleW().

> is there a way I can check the locale and code page values that you
> mentioned? I assume I could call GetACP using ctypes, but maybe
> there is a simpler way?

io.TextIOWrapper uses locale.getpreferredencoding(False) as the
default encoding. Actually, in 3.11+ it uses locale.getencoding()
unless UTF-8 mode is enabled, which is effectively the same as
locale.getpreferredencoding(False). On Windows this calls GetACP() and
formats the result as "cp%u" (e.g. "cp1252").
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Create a Python Launcher on Desktop

[Eryk Sun]

On 11/11/22, ohins...@gmail.com  wrote:
>
> I am real a beginner of Python.  Not able to create a Python launcher
> (shortcut) on Desktop after the installation.

Did you already install Python? If not, open the Store, and install
the Python 3.11 app that's published by the Python Software
Foundation. After Python is installed, you'll have shortcuts in the
start menu that run Python in a terminal or that run the IDLE
development environment.

On the context menu of a shortcut, there's an action to pin it to the
top-level start menu, which is more convenient than having to click on
"all apps" to find it. Also, the context menu of a pinned shortcut has
an action to pin it to the taskbar, for even more convenient access.
The running application icon on the taskbar also lets you pin the app.

If you really want a shortcut on your desktop, it depends on which
distribution you installed. If you installed the app version of 3.11,
then you have "python3.11.exe" to run Python in a terminal and
"idle3.11.exe" to IDLE. To create a shortcut, right click the desktop;
select the action to create a new shortcut; and enter one of the
latter executable names. After creating the shortcut, on its context
menu select the "properties" action, and then modify the "start in"
folder to your preferred working directory.

If you installed the standard distribution from python.org, then you
already have normal file shortcuts (as opposed to app shortcuts) in
the start menu, which you can copy to the desktop. The context menu of
the start-menu shortcut has an action to open the file location. This
opens an Explorer window. Right click the shortcut in that window and
drag it to the desktop. Release the mouse button and select the action
"copy here".
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange UnicodeEncodeError in Windows image on Azure DevOps and Github

[Eryk Sun]

On 11/10/22, Jessica Smith <12jessicasmit...@gmail.com> wrote:
>
> Weird issue I've found on Windows images in Azure Devops Pipelines and
> Github actions. Printing Unicode characters fails on these images because,
> for some reason, the encoding is mapped to cp1252. What is particularly
> weird about the code page being set to 1252 is that if you execute "chcp"
> it shows that the code page is 65001.

If stdout isn't a console (e.g. a pipe), it defaults to using the
process code page (i.e. CP_ACP), such as legacy code page 1252
(extended Latin-1). You can override just sys.std* to UTF-8 by setting
the environment variable `PYTHONIOENCODING=UTF-8`. You can override
all I/O to use UTF-8 by setting `PYTHONUTF8=1`, or by passing the
command-line option `-X utf8`.

Background

The locale system in Windows supports a common system locale, plus a
separate locale for each user. By default the process code page is
based on the system locale, and the thread code page (i.e.
CP_THREAD_ACP) is based on the user locale. The default locale of the
Universal C runtime combines the user locale with the process code
page. (This combination may be inconsistent.)

In Windows 10 and later, the default process and thread code pages can
be configured to use CP_UTF8 (65001). Applications can also override
them to UTF-8 in their manifest via the "ActiveCodePage" setting. In
either case, if the process code page is UTF-8, the C runtime will use
UTF-8 for its default locale encoding (e.g. "en_uk.utf8").

Unlike some frameworks, Python has never used the console input code
page or output code page as a locale encoding. Personally, I wouldn't
want Python to default to that old MS-DOS behavior. However, I'd be in
favor of supporting a "console" encoding that's based on the console
input code page that's returned by GetConsoleCP(). If the process
doesn't have a console session, the "console" encoding would fall back
on the process code page from GetACP().
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Problems with IDLE in Windows 8.1 and installer x86 Version 3.10.8

[Eryk Sun]

On 11/11/22, darkst...@o2online.de  wrote:
>
> What can I do for the next step to find, why IDLE isn’t working?

The question is why tkinter isn't working. IDLE not working is just a
symptom of the underlying problem. In the command prompt, run 32-bit
Python 3.10 via `py -3.10-32`. In Python's interactive shell, run
`import _tkinter`. Please paste any resulting traceback and error
message in your reply to this message.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Problems with IDLE in Windows 8.1 and installer x86 Version 3.10.8

[Eryk Sun]

On 11/9/22, darkst...@o2online.de  wrote:
> Is there no one who can help?

If you can't run IDLE via `py -3.10-32 -m idlelib`, then something
isn't installed properly. You reported an error that IDLE fails to
load because importing tkinter fails. Did you try `import tkinter` in
the REPL? tkinter depends on the _tkinter extension module. Try
`import _tkinter`. If the latter fails because of a missing DLL
dependency, check the "DLLs" directory in the installation directory
for the TCL/Tk dependencies. They're "tcl86t.dll" and "tk86t.dll" for
Python 3.10. The installation directory should also have a "tcl"
directory, which should contain "tcl8.6" and "tk8.6" directories among
others, with many .tcl files.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Problems with IDLE in Windows 8.1 and installer x86 Version 3.10.8

[Eryk Sun]

On 11/3/22, darkst...@o2online.de  wrote:
> Is there a reason, why it is not installed? Its the same check mark in the
> installer like IDLE…

Did you try what I suggested? Modify the installation to remove the
tkinter/IDLE component. Then modify it again to select the component
to be reinstalled. Also, try to repair the installation. This may
reset any DLLs or extension modules that were missing or that were the
wrong version.

Ignore the suggestion from Nithish to install tkinter via pip. tkinter
is part of the standard library and cannot be installed via pip. There
is no tkinter package on the Python package index (pypi.org).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Problems with IDLE in Windows 8.1 and installer x86 Version 3.10.8

[Eryk Sun]

On 11/1/22, Nithish Ramasamy  wrote:
>
> pip install tkinter
> Wait some minutes to install tkinter

There is no tkinter package on PyPI. It's part of the standard library
and included with the python.org installer as an optional component.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Problems with IDLE in Windows 8.1 and installer x86 Version 3.10.8

[Eryk Sun]

On 11/1/22, darkst...@o2online.de  wrote:
>
> **IDLE can’t Import TKINTER
>
> Python may not be configured for TK**
>
> Checkmark for TK is set in the Installation Progress. What went wrong and ho
> can I fix it?

Run the following command to check whether the ImportError has any
further information.

py -3.10-32 -c "from tkinter import *"

It could be a missing extension module or DLL, or mismatched version.
You could try modifying the installation to remove tkinter and IDLE,
and then again to restore it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Problems with IDLE in Windows 8.1 and installer x86 Version 3.10.8

[Eryk Sun]

On 10/31/22, darkst...@o2online.de  wrote:
>
> I installed the Standard Distribution from python.org again, and i ensured,
> that the checkmark test Suite is enabled. Idle does’nt start. The installer
> says “Installation successfully” at the end.
>
> What went wrong and how can I further delimit the fault/find the error?

Open a command prompt and run the following command:

py -3.10-32 -m idlelib

If it fails, an exception and traceback should be printed to the console.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Problems with IDLE in Windows 8.1 and installer x86 Version 3.10.8

[Eryk Sun]

On 10/31/22, darkst...@o2online.de  wrote:
>
> i uninstalled this, because my Idle doesn’t start by clicking on the Icon.
> Are there any Solutions for the problem?

If it's the standard distribution from python.org, run the installer
again, and ensure that the test suite is installed.

In 3.10.8, IDLE mistakenly depends on the test suite. This mistake is
fixed in 3.10.9, which is planned to be released in early December.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Find the path of a shell command (shutil.which)

[Eryk Sun]

On 10/12/22, Weatherby,Gerard  wrote:
>
> When no path is specified, the results of
> os.environ() are used,
> returning either the “PATH” value or a fallback of
> os.defpath.

The documentation is out of sync with what the code actually does.
os.defpath is hard coded in posixpath and ntpath, but shutil.which()
actually prefers to query the system's default path to use via
os.confstr(), if it's supported:

if path is None:
path = os.environ.get("PATH", None)
if path is None:
try:
path = os.confstr("CS_PATH")
except (AttributeError, ValueError):
# os.confstr() or CS_PATH is not available
path = os.defpath

As documented by POSIX, the CS_PATH string "can be used as a value of
the PATH environment variable that accesses all of the standard
utilities of POSIX.1-2017, that are provided in a manner accessible
via the exec family of functions".

https://pubs.opengroup.org/onlinepubs/9699919799/functions/confstr.html
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fail 3.10.8 version installation on Windows 11 21H2

[Eryk Sun]

On 10/12/22, Kirill Ratkin via Python-list  wrote:
>
> Do anyone face issue like in log below?
>
> I got last installer (3.10.8) and try to install it 'for all users' with
> downloading precompiled (pdb) files.

There was a permission problem on the Python website that prevented
downloading the 3.8.10 debug binaries and symbol files. It's fixed
now.

https://github.com/python/cpython/issues/98193
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to fix Python error, The term '.../python.exe' is not recognized as the name of a cmdlet, function, script file, or operable program, in VS Code?

[Eryk Sun]

On 10/11/22, LouisAden Capellupo via Python-list  wrote:
> Variables. 
> C:\Users\It'sMeLil'Loui\AppData\Local\Programs\Python\Python310\Scripts\,
> and C:\Users\It'sMeLil'Loui\AppData\Local\Programs\Python\Python310\.

I suggest that you switch to a user account that doesn't have single
quotes in the name. Using a name that contains single quotes is asking
for trouble. In most command-line shells, which includes PowerShell
(but not CMD), a literal (verbatim) string is contained by single
quotes. The shell usually consumes the quote character if it's not
escaped. See the following sections of the PowerShell documentation:

https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_quoting_rules?view=powershell-7.2#single-quoted-strings

https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_quoting_rules?view=powershell-7.2#including-quote-characters-in-a-string
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Debugging automatic quotation in subprocess.Popen()

[Eryk Sun]

On 10/7/22, c.bu...@posteo.jp  wrote:
>
> I need to improve my understanding about how subprocess.Popen() does
> quote arguments. I have special case here.
>
> Simple example:
> Popen(['ls', '-l']) results on a shell in "ls -l" without quotation.

The shell is only used if Popen is instantiated with `shell=True`. The
above example does not use the shell. It runs the "ls" executable
directly. On POSIX systems, fork() and exec() are called to create the
child process. The argument list is passed to exec() and becomes the
argv array of the application's main() entry point function.

On Windows systems, CreateProcessW() is called to created the child
process. It requires a command-line string instead of an argument
array. The argument list gets joined into a command-line string via
subprocess.list2cmdline(), which is based on the rules that are used
by the Windows C runtime library when it parses a command line into
the argv array of an application's [w]main() entry point. That said, a
Windows application is free to parse its command line however it
wants, so listcmdline() is little more than a best guess. On Windows,
Popen() may have to be called directly with a command-line string in
some cases.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to replace an instance method?

[Eryk Sun]

On 9/19/22, 2qdxy4rzwzuui...@potatochowder.com
<2qdxy4rzwzuui...@potatochowder.com> wrote:
> On 2022-09-18 at 09:11:28 +,
> Stefan Ram  wrote:
>
>> r...@zedat.fu-berlin.de (Stefan Ram) writes (abbreviated):
>> >types.MethodType( function, instance )
>> >functools.partial( function, instance )
>> >new_method.__get__( instance )
>>
>>   I wonder which of these three possibilities expresses
>>   the idea of creating a new method from a function and
>>   an instance most clearly.
>
> The first one.  And only the first one.
>
> The second one requires too much inside knowledge of Python to make the
> leap from currying to instance method.
>
> The third one doesn't even mention the function.

The OP's example named the function "new_method". In general the third
case would be func.__get__(instance). It's how the interpreter binds a
new method when a function from the class hierarchy is accessed as an
instance attribute.

When a function from the class hierarchy is accessed as an attribute
of the class, it's equivalent to calling func.__get__(None, cls),
which just returns a reference to the function. To use the descriptor
protocol to bind a function as a method of the class requires wrapping
it with the classmethod descriptor type. For example,
classmethod(func).__get__(None, cls) returns a method object with
__self__ that references the cls type. Of course, calling
types.MethodType(func, cls) is easier to understand and the preferred
way.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to replace an instance method?

[Eryk Sun]

On 9/18/22, Stefan Ram  wrote:
> r...@zedat.fu-berlin.de (Stefan Ram) writes (abbreviated):
>>types.MethodType( function, instance )
>>functools.partial( function, instance )
>>new_method.__get__( instance )
>
>   I wonder which of these three possibilities expresses
>   the idea of creating a new method from a function and
>   an instance most clearly.

Using types.MethodType(function, instance) is the most clear and
correct of the three. Using the function's descriptor __get__() method
is equivalent in terms of the result. That said, the descriptor
protocol is an intermediate-level concept, so manually calling
__get__() isn't friendly to novices or casual users of the language.

Using a functools.partial isn't the expected method type, with
__func__ and __self__ attributes, and, unlike a method, it doesn't
expose the wrapped function's __code__, __name__, __module__, __doc__,
__annotations__, __defaults__, __kwdefaults__, __closure__,
__globals__, or __builtins__. If dynamic inspection matters, using a
functools.partial won't work directly with dis.dis(),
inspect.getfile(), inspect.getsource(), inspect.getdoc(),
inspect.get_annotations(), inspect.getcallargs(), or
inspect.getclosurevars().
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to replace an instance method?

[Eryk Sun]

On 9/17/22, Chris Angelico  wrote:
>
> A method IS a function. A bound method is a function with one argument
> locked in, but still a function.

We were talking past each other. A method object is not a function
object. You're talking about a function defined in a class that's
accessed as a method of an instance of the class. In the class, that's
just a function object; it's exactly a function, not merely
fundamentally the same thing as one. It's only when accessed as an
attribute of an instance of the type that the function's descriptor
__get__() method is called to bind it and the instance to a method
object that sets the instance as its __self__ and the function as its
__func__.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to replace an instance method?

[Eryk Sun]

On 9/17/22, Chris Angelico  wrote:
>
> The two are basically equivalent. Using functools.partial emphasizes
> the fact that all you're doing is "locking in" the first parameter;
> using the __get__ method emphasizes the fact that functions are,
> fundamentally, the same thing as methods. Choose whichever one makes
> sense to you!

Functions are really not "fundamentally, the same thing as methods".
They're only the same in that they're both callable. Also, a method's
__getattribute__() falls back on looking up attributes on the
underlying function (i.e. the method's __func__), such as inspecting
the __name__ and __code__. A fundamental difference is that, unlike a
function, a method is not a descriptor. Thus if a method object is set
as an attribute of a type, the method does not rebind as a new method
when accessed as an attribute of an instance of the type.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to replace an instance method?

[Eryk Sun]

On 9/16/22, Ralf M.  wrote:
> I would like to replace a method of an instance, but don't know how to
> do it properly.

A function is a descriptor that binds to any object as a method. For example:

>>> f = lambda self, x: self + x
>>> o = 42
>>> m = f.__get__(o)
>>> type(m)

>>> m.__self__ is o
True
>>> m(10)
52
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Can you mock a C function using ctypes?

[Eryk Sun]

On 9/15/22, Grant Edwards  wrote:
>
> Does the pointer have to be passed? Or can it be "poked" into a global
> variable?

If the library exports the function pointer as a global variable, it
can be set in ctypes using the in_dll() method. Unfortunately, as far
as I know, a ctypes function prototype can't be used directly for
this, at least not easily, since the function pointer it creates
doesn't have a `value` or `contents` attribute to set the target
address. Instead, the exported global can be accessed as a void
pointer. It's clunky, but it works.

For example, Python's REPL supports a callback for reading a line from
the terminal. Normally it's either hooked by a C extension, such as
the readline module, or set to the default function
PyOS_StdioReadline(). We can use a ctypes callback to hook into this
and chain to the previous function.

import ctypes

PyOS_RFP = ctypes.CFUNCTYPE(
ctypes.c_void_p,
ctypes.c_void_p, # stdin (FILE *)
ctypes.c_void_p, # stdout (FILE *)
ctypes.c_char_p, # prompt
)

@PyOS_RFP
def readline_hook(stdin, stdout, prompt):
print('HOOKED: ', end='', flush=True)
return prev_rfp(stdin, stdout, prompt)

rfp_vp = ctypes.c_void_p.in_dll(ctypes.pythonapi,
'PyOS_ReadlineFunctionPointer')

if rfp_vp:
# there's a previous RFP that can be hooked
prev_rfp = ctypes.cast(rfp_vp, PyOS_RFP)
rfp_vp.value = ctypes.cast(readline_hook, ctypes.c_void_p).value

In this trivial example, "HOOKED: " is printed to the terminal before
each line is read:

HOOKED: >>> def f():
HOOKED: ... pass
HOOKED: ...
HOOKED: >>>

Note that I defined the return type of PyOS_RFP to be c_void_p. Using
c_char_p as the return type would convert the result to a bytes
object, which is wrong in this case. If the setfunc of the callback's
return type has to keep a reference to a converted Python object, such
as a bytes object, then the callback has no choice but to leak the
reference in order to keep the object alive indefinitely, which
usually causes a memory leak.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Can you mock a C function using ctypes?

[Eryk Sun]

On 9/15/22, Grant Edwards  wrote:
>
> Can that be done using ctypes?
>
> For example, I open a library that contains functon foo() where foo()
> calls external function bar() which is not contained in the library.
> Then, I provide a Python bar() function that gets called by foo() when
> foo() is called?

That's straight forward if the library allows the application to pass
a function pointer to bar(). ctypes function prototypes are defined by
ctypes.CFUNCTYPE(restype, *argtypes, use_errno=False,
use_last_error=False) for the cdecl calling convention, or similarly
by ctypes.WINFUNCTYPE() to use the Windows stdcall calling convention.
A prototype can be instantiated as a function pointer that calls a
Python function, e.g. c_bar = prototype(bar).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make a variable's late binding crosses the module boundary?

[Eryk Sun]

On 8/31/22, Jach Feng  wrote:
>
> I found that using "from test import *" in test2.py makes test2's relation
> to "test" almost like the function's relation to the module where it is
> defined. You can refer to all the variables of the module

Note that in general, if __all__ doesn't exist to explicitly define
the contents of a star import, then it imports everything except names
that begin with an underscore.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Local variable definition in Python list comprehension

[Eryk Sun]

On 9/1/22, James Tsai  wrote:
>
> I find it very useful if I am allowed to define new local variables in a
> list comprehension. For example, I wish to have something like
> [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or
> [(x, y) for x in range(10) with y := x ** 2 if x + y < 80].
>
> For now this functionality can be achieved by writing
> [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80].

You can assign a local variable in the `if` expression. For example:

>>> [(x, y) for x in range(10) if x + (y := x**2) < 30]
[(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Download Python 3.10.6 for windows

[Eryk Sun]

On 8/31/22, George Rwaga  wrote:
>
> I am not sure why the default for installing Python is
> C:\Users\x.x\AppData\local\programs\Python\Python310.

The "programs" folder in the above path is the default location to
install programs just for user "x.x." More precisely, the
default location is "%LOCALAPPDATA%\Programs". A user can easily
change this path to a custom location in Explorer, so one shouldn't
rely on the default value in general. Instead, use the shell protocol
path, shell:userprogramfiles. You can use the latter directly in
Explorer, such as the Win+R dialog. From the command line in CMD or
PowerShell, use the `start` command. For example:

>start shell:userprogramfiles

Python's installer defaults to the user program files directory when
installing just for the current user. When the setup is changed to
install for all users, it automatically switches to using
"%ProgramFiles%" or "%ProgramFiles(x86)%" (e.g. "C:\Program Files").
It will try to elevate to get administrator access when installing for
all users.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Download Python 3.10.6 for windows

[Eryk Sun]

On 8/30/22, George Rwaga  wrote:
>
> 1. I installed Python 3.10.6 in the default directory
> C:\Users\x.x\AppData\local\programs\Python\Python310
> After the installation, there was no shortcut on my desktop.

Shortcuts are created in the start menu. The installer doesn't modify
the user's desktop or the desktop of all users, which many users don't
want and would find annoying. Just copy the shortcuts from the start
menu to the desktop if that's what you want. Right-click the icon in
the start menu and choose to open the location. You can copy shortcuts
from the opened Explorer window.

> 2. I then decided to install Python3.10.6 in a customized directory
> C:\program files\Python\Python310
> I am being asked to verify access to this directly. I went to properties,
> made what I thought were the relevant changes. But I keep getting asked to
> verify access.

Installing to "Program Files" requires elevating to get administrator
access. All users have the right to read and execute files in a
directory created in "Program Files", but adding, removing, or
deleting files and directories requires administrator access, as it
should.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: subprocess.popen how wait complete open process

[Eryk Sun]

On 8/21/22, simone zambonardi  wrote:
> Hi, I am running a program with the punishment subrocess.Popen(...) what I
> should do is to stop the script until the launched program is fully open.
> How can I do this? I used a time.sleep() function but I think there are
> other ways. Thanks

In Windows, WaitForInputIdle() waits until a thread in a process
creates one or more windows and its message loop goes idle. Usually
this is the main UI thread. Console processes are not supported.

For example:

import ctypes
import subprocess

user32 = ctypes.WinDLL('user32', use_last_error=True)

INFINITE = 0x_
WAIT_FAILED = 0x_
WAIT_TIMEOUT = 0x_0102

# Waiting on a console process fails with ERROR_NOT_GUI_PROCESS.
# This case be handled in other ways, depending on the need.
ERROR_NOT_GUI_PROCESS = 1471

user32.WaitForInputIdle.restype = ctypes.c_ulong
user32.WaitForInputIdle.argtypes = (ctypes.c_void_p, ctypes.c_ulong)

def wait_for_input_idle(proc, timeout=None):
if isinstance(proc, subprocess.Popen):
handle = int(proc._handle)
args = p.args
else:
handle = int(proc)
args = ''
if timeout is None:
timeout_ms = INFINITE
elif timeout < 0:
raise ValueError('timeout cannot be negative')
else:
timeout_ms = int(timeout * 1000)
if timeout_ms >= INFINITE:
raise OverflowError('timeout is too large')
status = user32.WaitForInputIdle(handle, timeout_ms)
if status == WAIT_FAILED:
raise ctypes.WinError(ctypes.get_last_error())
elif status == WAIT_TIMEOUT:
raise subprocess.TimeoutExpired(args, timeout)
assert status == 0
return


if __name__ == '__main__':
import time
t0 = time.time()
p = subprocess.Popen(['pythonw.exe', '-m', 'idlelib'])

try:
wait_for_input_idle(p, 5)
except:
p.terminate()
raise

wait_time = time.time() - t0
print(f'wait time: {wait_time:.3f} seconds')
try:
p.wait(5)
except subprocess.TimeoutExpired:
p.terminate()
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Persistent Error: Python was not found

[Eryk Sun]

On 8/15/22, Dennis Lee Bieber  wrote:
>
>   Just double-clicking on the file will run it. The problem is that it
> will open a command shell, run, and then close the command shell UNLESS one
> explicitly codes some sort of "hold" at the end of the program

The console window is a terminal, not a shell. If an application is
flagged as a console app, as is "python.exe", and the process doesn't
inherit a console, the initialization code in kernelbase.dll allocates
a new console session. This could be implemented by the classic
conhost.exe host, or, in Windows 11, by an openconsole.exe session
that's associated with a tab in Windows Terminal. If it's the latter,
Terminal can be configured to keep a tab open after the console
session has ended. The tab will display the exit status of the process
that allocated the console session.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Persistent Error: Python was not found

[Eryk Sun]

On 8/15/22, Gisle Vanem via Python-list  wrote:
> Eryk Sun wrote:
>
>> If the redirector app
>> is run without arguments, it will open the Microsoft Store to install
>> the latest version of the Python store app distribution. Currently
>> that means Python 3.10.
>
> That is true with cmd. But with a shell like 4NT, I get:
>c:\> "%LocalAppData%\Microsoft\WindowsApps\python.exe"
>4NT: (Sys) No access to the file.
> "C:\Users\gvane\AppData\Local\Microsoft\WindowsApps\python.exe"
>
> No matter what I do with this "App Alias" setting.
> What a broken and confusing design this AppX design is.

An app execution alias is a reparse point with the tag
IO_REPARSE_TAG_APPEXECLINK (0x801B). Neither the I/O manager no
any driver in the kernel supports this type of reparse point. For
better or worse, this is intentional. As such, if CreateFileW() is
called on an alias without using the flag
FILE_FLAG_OPEN_REPARSE_POINT, the attempt to traverse the link fails
with ERROR_CANT_ACCESS_FILE (1920). Note that Python's os.stat() was
updated to open the reparse point directly in this case instead of
failing. But a lot of applications, in particular non-Microsoft shells
such as MSYS bash (and apparently 4NT) haven't been updated similarly.

Even if the link could be traversed, the target file under
"%ProgramFiles%\WindowsApps" doesn't grant execute access to users
unless they have an access token that includes a WIN://SYSAPPID
attribute that corresponds to the executed app. How this works in
practice when executing an app is that CreateProcessW() handles
ERROR_CANT_ACCESS_FILE by opening the reparse point, reading the
relevant app information, and creating a custom access token that
allows it to execute the app directly via the internal equivalent of
CreateProcessAsUserW().
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Persistent Error: Python was not found

[Eryk Sun]

On 8/15/22, Jonathan Owah  wrote:
> Thank you so much for your assistance .
>
> The fault was actually mine: I was running a command
> with python3, instead of just python.
> python3 works for Mac, but not Windows.

If the Python 3.10 store app is installed with all aliases enabled,
then "python", "python3", and "python3.10" all work. The standard
distribution from python.org, on the other hand, only has a "python"
executable.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Persistent Error: Python was not found

[Eryk Sun]

On 8/13/22, Jonathan Owah  wrote:
>
> I've been trying to configure my laptop to run python scripts.
> This is the error I keep getting:
> Python was not found; run without arguments to install from the Microsoft
> Store, or disable this shortcut from Settings > Manage App Execution
> Aliases.

If you keep seeing this message, then the shell is finding and running
Microsoft's default "python.exe" redirector app execution alias that's
located in "%LocalAppData%\Microsoft\WindowsApps". By default, this
directory is set at the beginning of the user "Path" value in the
registry and thus takes precedence (but not over the system "Path").
Confirm this by running `where.exe python`.

An app execution alias is a special type of filesystem symbolic link
to a store app's executable. These aliases are created in a user's
"%LocalAppData%\Microsoft\WindowsApps" directory. Store apps
themselves are usually installed in "%ProgramFiles%\WindowsApps",
which is a system managed directory that even administrators can't
easily modify (and shouldn't modify). Each user on a system has their
own set of installed store apps, even though the apps are installed
only once at the system level.

By default, Windows creates "python.exe" and "python3.exe" aliases for
the "App Installer" PythonRedirector app. In the alias manager, these
two will be clearly listed as aliases for "App Installer". If you run
this redirector app with one or more command-line arguments, it will
print the above quoted message to the console. If the redirector app
is run without arguments, it will open the Microsoft Store to install
the latest version of the Python store app distribution. Currently
that means Python 3.10.

In my experience, the app execution alias manager component of Windows
is unreliable. A disabled alias might still exist in
"%LocalAppData%\Microsoft\WindowsApps", or an old alias might be left
in place when an app is installed.  Once the real Python store app is
installed, go back into the alias manager and toggle the "python.exe"
and "python3.exe" aliases off and back on. If that doesn't resolve the
problem, manually delete the "python.exe" and "python3.exe" aliases
from "%LocalAppData%\Microsoft\WindowsApps". Then toggle them off and
on again in the alias manager. Hopefully they'll be created to
correctly alias the real Python app instead of the "App Installer"
redirector.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: exec() an locals() puzzle

[Eryk Sun]

On 7/20/22, george trojan  wrote:
>
> 1. This works as I expect it to work:
>
> def f():
> i = 1
> print(locals())
> exec('y = i; print(y); print(locals())')
> print(locals())
> exec('y *= 2')
> print('ok:', eval('y'))
> f()

In CPython, the locals of a function scope (as opposed to a class
scope or module scope) are optimized by storing them in an array in
the current frame. locals(), however, always returns the non-optimized
locals mapping of the current frame, i.e. the value of
sys._getframe(0).f_locals. In the case of optimized locals, the
locals() call first updates this dict with a snapshot of the current
values of local variables. This is useful for introspection, but
modifying the snapshot cannot extend or modify the optimized local
variables of the function scope.

exec() and eval() default to using the global and local variable
mappings that are returned by globals() and locals(). This allows them
to access the snapshot of the containing function's optimized locals,
but they can't extend or modify the function's optimized locals. At
most they can add dynamic locals to the snapshot, as used by
subsequent calls to locals(), exec() and eval().
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pip upgrade causing issues in 3.10

[Eryk Sun]

On 7/20/22, Mike Dewhirst  wrote:
> On 20/07/2022 4:43 am, David Raymond wrote:
>> C:\Program Files\Python310\Scripts>..\python.exe -m pip install --upgrade
>> pip
>> ERROR: Could not install packages due to an OSError: [WinError 32] The
>> process cannot access the file because it is being used by another
>> process: 'c:\\program files\\python310\\scripts\\'
> There's your problem. The 'other' process is your cmd.exe within which
> you are typing etc.
>
> Python scripts dir should be on the path so you don't have to execute
> anything from within it. Windows is obviously tripping over its own toes
> trying to delete and install something in the same dir while you also
> have your foot in it.

This should only occur if uninstalling pip deletes all of the files in
the "Scripts" directory. In this case, pip will 'compress' the
individual delete operations to remove the entire directory. It begins
by trying to 'stash' the "Scripts" directory, i.e. by renaming it to a
temporary adjacent name. It uses shutil.move() for this, which tries
os.rename() and falls back on shutil.copytree() and shutil.rmtree().

Of course this fails with a sharing violation if the directory is open
as the working directory in any process, including the current Python
process, because an open for a working directory doesn't share
delete/rename access. pip fails to handle this error. It crashes and
leaves a mess. The empty "Scripts" directory and the copytree() copy
aren't rolled back properly, and neither are the stashed (renamed)
"pip" and "pip*dist-info" directories in site packages.

This is not user error. It is a bug in pip. It should be able to
recover gracefully from failing to delete the directory. Moreover, it
shouldn't even have to roll back the operations if it deleted the
directory due to compression of explicit removals from the install
"RECORD". In particular there is no reason to delete the "Scripts"
directory, even if it's left empty.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: script folder is empty

[Eryk Sun]

On 7/17/22, Scott Baer  wrote:
>
> I've done some troubleshooting, and nothing is in the C:\Program
> Files\Python310\Scripts folder.

The installer may have silently failed to install pip. Run the
following command to check whether the package was installed.

"C:\Program Files\Python310\python.exe" -m pip -V

The standard library includes pip (a third-party package) via the
"ensurepip" bundle. You can use it to install pip with the following
command:

"C:\Program Files\Python310\python.exe" -m ensurepip --default-pip

Since your Python installation is for all users, this command must be
run from an elevated shell, i.e. "run as administrator". If your user
is in the administrators group and UAC is enabled (the default
configuration), then the system logs you on without administrator
rights and privileges. You have to elevate via the UAC consent prompt
in order to run a program with administrator access.

For installing other packages, in most cases you should run pip in a
virtual environment, or at least just for the current user via --user.
Installing to the system site packages should be limited to common
packages that you need for system services and administration across
multiple accounts, such as the pywin32 package.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Byte arrays and DLLs

[Eryk Sun]

On 6/30/22, Rob Cliffe via Python-list  wrote:
>
> AKAIK it is not possible to give ctypes a bytearray object and persuade
> it to give you a pointer to the actual array data, suitable for passing
> to a DLL.

You're overlooking the from_buffer() method. For example:

>>> ba = bytearray(10)
>>> ca = (ctypes.c_char * len(ba)).from_buffer(ba)
>>> ca.value = b'spam'
>>> ba
bytearray(b'spam\x00')

Note that the bytearray can't be resized while a view of the data is
exported. For example:

>>> ba.append(97)
Traceback (most recent call last):
  File "", line 1, in 
BufferError: Existing exports of data: object cannot be re-sized

>>> del ba[-1]
Traceback (most recent call last):
  File "", line 1, in 
BufferError: Existing exports of data: object cannot be re-sized
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: "CPython"

[Eryk Sun]

On 6/20/22, Paulo da Silva  wrote:
>
> Yes, but that does not necessarily means that the C has to refer to the
> language of implementation. It may well be a "core" reference to
> distinguish that implementation from others with different behaviors.

If the reference implementation and API ever switched to a different
programming language, I'd personally be fine with changing the 'C" in
"CPython" to mean "canonical", but not "core". The term "core" is used
for building the interpreter core with access to internals (i.e.
Py_BUILD_CORE, Py_BUILD_CORE_BUILTIN, Py_BUILD_CORE_MODULE, and
Include/internal/pycore*.h). It does not refer to the overall
implementation and API for embedding and extension modules.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ModuleNotFoundError

[Eryk Sun]

On 6/17/22, Zoltan Szenderak  wrote:
>
> print(sys.version_info) and executable:
> Unable to initialize device PRN

That's the command-line "print" program. You need to first start the
Python shell via python.exe. The prompt should change to ">>> ". Then
run print(sys.version) and print(sys.executable).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PYLAUNCH_DEBUG not printing info

[Eryk Sun]

On 6/9/22, Peter Otten <__pete...@web.de> wrote:
>
> Looks like the variable is now called PYLAUNCHER_DEBUG:
>
> https://docs.python.org/3.11/using/windows.html#diagnostics

I wonder why Steve changed the name of the environment variable
without supporting the old "PYLAUNCH_DEBUG" name, at least for a
couple of releases. Since 3.11 is still in beta, with the first
release candidate expected in August, there's still time to restore
support for the old name.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: .0 in name

[Eryk Sun]

On 5/28/22, Chris Angelico  wrote:
>
> be extremely confusing; so to keep everything safe, the interpreter
> generates a name you couldn't possibly want - same as for the function
> itself, which is named "" or "", angle brackets
> included.

To clarify, "" is the co_name and co_qualname value of the
code object, which was compiled for the list comprehension. These
names are also used as the __name__ and __qualname__ of the temporary
object that's created by MAKE_FUNCTION. They are not identifiers. The
code object is a constant, which is referenced solely by its index in
the co_consts tuple. The temporary function is referenced on the
stack.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip does not find after Python 3.10.4 installed

[Eryk Sun]

On 5/26/22, ANTHONY CHU  wrote:
> The Python 3.10.4 (64-bit) and Python Launcher had been (standard) installed
> successfully. But I could not find pip anywhere.  I uninstalled and
> re-installed a couple of times, it is still the problem. I checked the
> installed directory
> C:\Users\x\AppData\Local\Programs\Python\Python310\Tools\Scripts\

FYI, scripts and binaries for packages such as pip are installed in
"Scripts", not in "Tools\Scripts".

In Windows, it's common to run pip via `py -m pip`. Note that the
latter command will use an active virtual environment, if any, else it
defaults to the highest version of Python that's installed. You can
use a specific installed version via `py -X.Y[-32] -m pip`.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Discerning "Run Environment"

[Eryk Sun]

On 5/18/22, Chris Angelico  wrote:
>
> Real solution? Set the command prompt to codepage 65001. Then it
> should be able to handle all characters. (Windows-65001 is its alias
> for UTF-8.)

I suggest using win_unicode_console for Python versions prior to 3.6:

https://pypi.org/project/win_unicode_console

This package uses the console's native 16-bit character support with
UTF-16 text, as does Python 3.6+. Compared to the console's incomplete
and broken support for UTF-8, the console's support for UTF-16 (or
just UCS-2 prior to Windows 10) is far more functional and reliable
across commonly used versions of Windows 7, 8, and 10.

Reading console input as UTF-8 is still limited to ASCII up to and
including Windows 11, which for me is a showstopper. Non-ASCII
characters are read as null bytes, which is useless. Support for
writing UTF-8 to the console screen buffer is implemented correctly in
recent builds of Windows 10 and 11, and mostly correct in Windows 8.
Prior to Windows 8, writing UTF-8 to the console is badly broken. It
returns the number of UTF-16 codes written instead of the number of
bytes written, which confuses buffered writers into writing a lot of
junk to the screen.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Seeking deeper understanding of python equality (==)

[Eryk Sun]

On 5/14/22, Jonathan Kaczynski  wrote:
>
> So, I'm still wondering how Py_TYPE(v)->tp_richcompare resolves to __eq__
> on a user-defined class. Conversely, my understanding is, for a type
> defined in cpython, like str, there is usually an explicitly
> defined tp_richcompare function.

Sometimes it's simplest to directly examine an object using a native
debugger (e.g. gdb in Linux; cdb/windbg in Windows).

With a debugger attached to the interpreter, create two classes, one
that doesn't override __eq__() and one that does:

>>> class C:
... pass
...
>>> class D:
... __eq__ = lambda s, o: False
...

In CPython, the id() of an object is its address in memory:

>>> hex(id(C))
'0x2806a705790'
>>> hex(id(D))
'0x2806a6bbfe0'

Break into the attached debugger to examine the class objects:

>>> kernel32.DebugBreak()

(1968.1958): Break instruction exception - code 8003 (first chance)
KERNELBASE!wil::details::DebugBreak+0x2:
7ffd`8818fd12 cc  int 3

Class C uses the default object_richcompare():

0:000> ?? *((python310!PyTypeObject *)0x2806a705790)->tp_richcompare
 0x7ffd`55cac288
 _object*  python310!object_richcompare+0(
_object*,
_object*,
int)

Class D uses slot_tp_richcompare():

0:000> ?? *((python310!PyTypeObject *)0x2806a6bbfe0)->tp_richcompare
 0x7ffd`55cdef1c
 _object*  python310!slot_tp_richcompare+0(
_object*,
_object*,
int)

Source code of slot_tp_richcompare():

https://github.com/python/cpython/blob/v3.10.4/Objects/typeobject.c#L7610-L7626
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Solved] Re: Windows registry PermissionError

[Eryk Sun]

On 5/13/22, Mike Dewhirst  wrote:
> On 13/05/2022 4:14 pm, Eryk Sun wrote:
>> Since self.connect() is always called, you should document that the
>> initial hkey parameter has to be one of the following predefined key
>> handles:
>>
>>  HKEY_LOCAL_MACHINE
>>  HKEY_USERS
>
> I'm targeting HKEY_CURRENT_USER so I assume HK_USERS includes that.

Using HKEY_CURRENT_USER with RegConnectRegistryW() to access a remote
registry isn't well defined and not documented as supported. If it
works at all, the API probably just opens a subkey of the remote
HKEY_USERS based on the string SID (security identifier string) of the
current user. That may fail as not found since user SIDs are unique to
machines, unless it's on a domain.

Bear in mind that the remote registry service is running on the remote
machine as SYSTEM (S-1-5-18) in the non-interactive services session.
If it literally accessed its "current user", then it would open
"HKEY_USERS\S-1-5-18".
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Solved] Re: Windows registry PermissionError

[Eryk Sun]

Since self.connect() is always called, you should document that the
initial hkey parameter has to be one of the following predefined key
handles:

HKEY_LOCAL_MACHINE
HKEY_USERS
HKEY_PERFORMANCE_DATA

WinAPI RegConnectRegistryW() only matters when the target computer is
a different machine, in which case an RPC proxy handle is returned.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Windows registry PermissionError

[Eryk Sun]

On 5/12/22, Mike Dewhirst  wrote:
>
>  access=wr.KEY_ALL_ACCESS + wr.KEY_WRITE,

The access parameter is a bit mask of access rights that combine via
bitwise OR (|), not via arithmetic addition.

KEY_ALL_ACCESS (0x000F_003F) is a superset of KEY_WRITE (0x0002_0006):

KEY_WRITE = (
READ_CONTROL   | # 0x0002_
KEY_SET_VALUE  | # 0x_0002
KEY_CREATE_SUB_KEY | # 0x_0004
)# 0x0002_0006

KEY_ALL_ACCESS = (
DELETE | # 0x0001_
READ_CONTROL   | # 0x0002_
WRITE_DAC  | # 0x0004_
WRITE_OWNER| # 0x0008_
KEY_QUERY_VALUE| # 0x_0001
KEY_SET_VALUE  | # 0x_0002
KEY_CREATE_SUB_KEY | # 0x_0004
KEY_ENUMERATE_SUB_KEYS | # 0x_0008
KEY_NOTIFY | # 0x_0010
KEY_CREATE_LINK| # 0x_0020
)# 0x000F_003F

The result of the arithmetic addition `KEY_ALL_ACCESS + KEY_WRITE` is
0x0011_0045, which is wrong and meaningless. Registry key objects do
not support SYNCHRONIZE (0x0010_) access; DELETE (0x0001_)
access isn't needed; 0x_0040 is not a supported key right;
KEY_CREATE_SUB_KEY (0x_0004) access isn't needed; and
KEY_QUERY_VALUE (0x_0001) isn't sufficient.

You should limit the requested access to the specific access rights
that are required for querying and setting values in the key:

access=(wr.KEY_QUERY_VALUE | wr.KEY_SET_VALUE)

> def setvalue(self, vname, value):
>return wr.SetValueEx(self.select(), vname, 0, 1, value)

You shouldn't hard code the value of the data type constant. Use
wr.REG_SZ instead of 1.

The return value of self.select() is a winreg PyHKEY object that wraps
the OS handle for the key object. You're relying on implicit closing
of this handle based on referencing counting. It's cleaner to use it
in a `with` statement, as you would for a file object returned by
open(). For example:

with self.select() as hkey:
wr.SetValueEx(hkey, vname, 0, wr.REG_SZ, value)

>  lmregistry = Registry(
>  hkey=wr.HKEY_LOCAL_MACHINE,
>  sub_key="SOFTWARE\WOW6432Node\XXX Technology\AppName",

You really shouldn't open the "WOW6432Node" key directly. It is an
implementation detail of the WOW64 subsystem that runs 32-bit
applications on a 64-bit system. If you need to operate on the
registry keys of 32-bit applications from a native 64-bit process,
open the normal path using the access right KEY_WOW64_32KEY
(0x_0200). For example:

hkey = wr.HKEY_LOCAL_MACHINE
subkey = r"SOFTWARE\XXX Technology\AppName"
access = (
wr.KEY_QUERY_VALUE |
wr.KEY_SET_VALUE |
wr.KEY_WOW64_32KEY
)

Typically you'd first try opening the path without either
KEY_WOW64_32KEY or KEY_WOW64_64KEY. The default view matches the
current process.

https://docs.microsoft.com/en-us/windows/win32/winprog64/accessing-an-alternate-registry-view

Remember to escape the backslash separators in string literals of key
paths, or use raw string literals as I used in the above example.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Difference in Setup Between Windows 10 Running Python 3.9 and Windows 11 Running Python 3.10

[Eryk Sun]

On 5/1/22, Brent Hunter  wrote:
>
> I was recently running a Windows 10 machine Python 3.9.  I simply created a
> batch file titled "Start-AIG.bat" which simply contained the following:
> "pythonw AIG.py".  It started a python program titled "AIG.py" and the
> Python dialog box was displayed on my screen, running all day and night.  I
> set up Windows to run this batch file upon startup and it worked fine.  I
> remember having to do a bunch of research before I learned that I needed to
> put "pythonw AIG.py" in the batch file as opposed to "python AIG.py".

When running a command at startup, it's best to use the full path of
the application and the full path of any files passed on the command
line. For example, run something like the following:

"path\to\pythonw.exe" "path\to\AIG.py"

This removes any dependency on the search PATH or the current working
directory. Also, the script shouldn't depend on the initial working
directory. Any files that it needs should be in a well-known location,
such as the script directory,
os.path.dirname(os.path.abspath(__file__)).

Also, there's no need to use a batch script just to run a single
command. When you use a batch script, a visible console gets created
for CMD. It doesn't close until CMD exits, which in this case won't be
until pythonw.exe exits, unless you use `START` to run the command
without waiting. It's simpler to just run the command directly using a
shortcut (i.e. LNK file) or the task scheduler.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Verifying I installed Python correctly

[Eryk Sun]

On 4/25/22, Barry  wrote:
>
>> On 25 Apr 2022, at 21:14, Jack Dangler  wrote:
>
>> Have you tried
>>
>> python3 hello.py
>
> Will not work on windows. Python is always installed as python.exe and
> py.exe only.

Yes, except the app versions installed from the Microsoft Store do
create appexec aliases for python.exe, python3.exe, and python3.x.exe
(e.g. python3.10.exe). The aliases for installed versions can be
toggled separately, so "python.exe" could be enabled to run 3.9, while
"python3.exe" is enabled to run 3.10. The same applies to the "pip*"
and "idle*" aliases.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to have python 2 and 3 both on windows?

[Eryk Sun]

On 4/23/22, Sunil KR via Python-list  wrote:
>
> I am happy with how the python starts up. When I use python I get
> python 2. I am ok with using py -3 for my new scripts, even using the
> shebang like #!py -3

`#!py -3` is not a valid shebang for the py launcher. Use `#!python3`
to run a script with the highest version of Python 3 that's registered
on your system. Or for cross-platform support, use
`#!/usr/bin/python3`.

> I don't want to put a unix (or for that matter windows) path in the shebang,
> as it is not platform portable

The launcher supports builtin virtual shebangs, including:

* #!python[X[.Y][-32|-64]]
* #!/usr/bin/python[X[.Y][-32|-64]]
* #!/usr/bin/env python[X[.Y][-32|-64]]
* #!/usr/local/bin/python[X[.Y][-32|-64]]

The implementation of the `/usr/bin/env` virtual shebang searches PATH
for a given command that's exactly "python" (no version spec) or any
command that doesn't start with "python". If a "python" command has a
version spec, then PATH is not searched, and it works the same as a
`#!pythonX[.Y]` shebang. The latter is because a normal Python
installation doesn't include versioned executable names, so the
launcher doesn't even try to search PATH for something like
"python3.11.exe".

You can set the default version to use via the PY_PYTHON environment
variable, e.g. `set PY_PYTHON=3.9`. If Python 3 is requested without a
specific version (e.g. via `py -3` at the command line, or the shebang
`#!python3`), then the default 3.x version to run can be set via the
PY_PYTHON3 environment variable, e.g. `set PY_PYTHON3=3.10`. The
default for Python 2 is set similarly via PY_PYTHON2.

The launcher also supports real file paths in shebangs. Generally
you'd only use a real path in order to run in a virtual environment,
such as #!"C:\Path\to\venv\Scripts\python.exe".
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: upgrade pip

[Eryk Sun]

On 4/22/22, Tola Oj  wrote:
> im trying to upgrade my pip so i can install openpyxl. i though i had
> successfully upgraded pip, and then I was trying to install openpyxl, but I
> was getting this:
>
> C:\Users\ojomo>"C:\Program Files\Python310\python.exe" -m pip install
> --upgrade
>
> [...]
>
> "C:\Users\ojomo\AppData\Roaming\Python\Python310\site-packages\pip\__main__.py",
> line 29, in 

You have pip installed for the current user, but Python is installed
for all users. I'd delete the package directory
"%AppData%\Python\Python310\site-packages\pip". Then run ensurepip and
upgrade from an administrator shell. This will install and update pip
for all users.

https://docs.python.org/3/library/ensurepip.html
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: No shortcut Icon on Desktop

[Eryk Sun]

On 4/15/22, Grant Edwards  wrote:
>
> The problem is that people run the installer, don't see a desktop
> icon, and think nothing has been installed.

Such people need to learn how to use the start menu, where all of
Python's shortcuts are installed in a folder named "Python ". One
can also press the Windows key and type "python" to get a filtered
view of the application shortcuts.

> Or they think the installer "is python", and run it over and over
> again trying to "run Python".

I'm not opposed to renaming the installer to something that makes it
more obvious that it's only an installer. But, to put this in
perspective, I think you're talking about a small number of people out
of the millions of users who I presume install and use Python without
a problem. It could be that thousands of people install Python and
give up without complaining when they can't use it, but I doubt it.

> If the installer, by default, created an IDLE desktop shortcut and a
> cmd.exe shortcut that ran Python, I believe it would eliminate most of
> those problems.

The installed shortcuts are to IDLE (via "pythonw.exe") and
"python.exe". IDLE is a graphical integrated development environment
(shell, editor, debugger) that by default runs an interactive shell.
The "python.exe" executable is a console (terminal) application that
by default runs an interactive shell if its arguments do not specify a
script file, -m module, or -c command to run.

A console application is attached to a console session, a resource
that's hosted by an instance of "conhost.exe" or (in a tab of Windows
Terminal) "openconsole.exe".  It connects to the console session via
files on the ConDrv device (e.g. "\Device\ConDrv\Connect",
"\Device\ConDrv\Input", "\Device\ConDrv\Output").

If an application executable is flagged as a console application, such
as "python.exe", and no console session was inherited from the parent
process, and the process wasn't spawned with the DETACHED_PROCESS
flag, then the Windows base API automatically allocates and attaches
to a new console session at startup during the initialization of
kernelbase.dll. If an application executable is flagged as a graphical
application, such as "pythonw.exe", then a console session is never
inherited and never automatically allocated, but one can be manually
allocated or acquired via AllocConsole() or
AttachConsole(dwProcessId).
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue47203] ImportError: DLL load failed while importing binascii: %1 is not a valid Win32 application.

[Eryk Sun]

Eryk Sun  added the comment:

There is something fundamentally wrong with the way modules built into the 
interpreter DLL (python3x.dll) are loaded if anything in sys.path or the system 
PATH can cause an import to fail.

--

___
Python tracker 
<https://bugs.python.org/issue47203>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue47203] ImportError: DLL load failed while importing binascii: %1 is not a valid Win32 application.

[Eryk Sun]

Eryk Sun  added the comment:

The user site packages directory is architecture specific starting with 3.10, 
e.g. "%AppData%\Python\Python310-32\site-packages". The PYTHONPATH environment 
variable is shared by all versions. 

However, I don't understand how the binascii module could be a problem because 
it's built into the interpreter DLL itself, i.e. it's in 
sys.builtin_module_names. There is no binascii.pyd, and, even if there were, it 
would be ignored.

--
nosy: +eryksun

___
Python tracker 
<https://bugs.python.org/issue47203>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue47198] os.stat on windows doesn't take an open file even though os.stat in os.supports_fd

[Eryk Sun]

Eryk Sun  added the comment:

You're mistaken about what `fd` is. It's a TextIOWrapper, which wraps a 
BufferedWriter, which buffers a FileIO raw file object, which accesses the open 
file number fd.fileno(). For example:

>>> f = open('tmp.tmp','w')
>>> os.stat(f.fileno()).st_size
0

--
nosy: +eryksun
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue47198>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue47170] py launcher on windows opens new terminal window when parsing python script with shebang

[Eryk Sun]

Eryk Sun  added the comment:

> This is Windows (shell) behaviour. To avoid this, you need to 
> add the .py extension to the PATHEXT environment variable.

PowerShell reuses the current console session only if .PY is set in PATHEXT. 
Otherwise Python gets executed with a flag that tells the system to allocate a 
new console session.

For CMD, PATHEXT only affects the file search, not how the file is executed. If 
the internal START command isn't used, CMD reuses the current console session. 
The START command defaults to making the system allocate a new console, unless 
the /B option is used.

--
nosy: +eryksun

___
Python tracker 
<https://bugs.python.org/issue47170>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue47161] pathlib method relative_to doesnt work with // in paths

[Eryk Sun]

Eryk Sun  added the comment:

> Hmm..., I get it, but Im not gonna lie it's pretty confusing given 
> that in other places `//` works as a substitute for `/`. Maybe it 
> should be mentioned in the documentation?

In Linux, the system resolves "//" as just "/". In other POSIX systems, such as 
Cygwin or MSYS2 (running in Windows), "//" is a UNC path of the form 
"//server/share/filepath". I would expect resolve() to handle this. For example:

Linux:

>>> p = pathlib.Path('//tmp')
>>> p
PosixPath('//tmp')
>>> p.resolve()
PosixPath('/tmp')

However, resolve() is broken for a UNC path in 3.9 under MSYS2:

>>> p = pathlib.Path('//localhost/C$/temp')
>>> p.exists()
True
>>> p.resolve()
PosixPath('/localhost/C$/temp')
>>> p.resolve().exists()
False

realpath() is also broken for this case in 3.9 under MSYS2:

>>> os.path.realpath(p)
'/localhost/C$/temp'

--
nosy: +eryksun

___
Python tracker 
<https://bugs.python.org/issue47161>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39090] Document various options for getting the absolute path from pathlib.Path objects

[Eryk Sun]

Eryk Sun  added the comment:

> Now a file that doesn't exist:
> >>> mike = Path("palin.jpg")
> >>> mike.resolve()
> WindowsPath('palin.jpg')

This is a bug in resolve(). It was fixed in 3.10+ by switching to 
ntpath.realpath(). I don't remember why a fix for 3.9 was never applied. Work 
on the PR may have stalled due to a minor disagreement.

> 'C:\Windows\..\Program Files' and '/usr/../bin' == relative path

No, a relative path depends on either the current working directory or, for a 
symlink target, the path of the directory that contains the symlink.

In Windows, a rooted path such as r"\spam" is a relative path because it 
depends on the drive of the current working directory. For example, if the 
current working directory is r"Z:\eggs", then r"\spam" resolves to r"Z:\spam". 
Also, a drive-relative paths such as "Z:spam" depends on the working directory 
of the given drive. Windows supports a separate working directory for each 
drive. For example, if the working directory of drive "Z:" is r"Z:\eggs", then 
"Z:spam" resolves to r"Z:\eggs\spam".

--
nosy: +eryksun

___
Python tracker 
<https://bugs.python.org/issue39090>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com