[issue43317] python -m gzip could use a larger buffer

[Ruben Vorderman]

Change by Ruben Vorderman :


--
type:  -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43316] python -m gzip handles error incorrectly

[Ruben Vorderman]

Change by Ruben Vorderman :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43303] ASAN : Memory leak in compilation

[Ramin Farajpour Cami]

Change by Ramin Farajpour Cami :


--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43318] pdb can't output the prompt message when successfully clear a breakpoint by "filename:lineno"

[huzhaojie]

Change by huzhaojie :


--
components: Library (Lib)
nosy: hjzin
priority: normal
severity: normal
status: open
title: pdb can't output the prompt message when successfully clear a breakpoint 
by "filename:lineno"
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

[Tzu-ping Chung]

Change by Tzu-ping Chung :


--
keywords: +patch
pull_requests: +23429
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24644

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

[Steve Dower]

Steve Dower  added the comment:

This seems like a good change to me. Making sure that the implementation is 
easily patchable sounds like a good plan, though I don't have any specific 
suggestions on what would be best.

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39758] StreamWriter.wait_closed() can hang indefinitely.

[Russell Owen]

Russell Owen  added the comment:

Regarding my previous comment: I have never seen this in Python 3.7 (though I 
see that this particular bug is listed as being present there) so it may be a 
different underlying issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39758] StreamWriter.wait_closed() can hang indefinitely.

[Russell Owen]

Russell Owen  added the comment:

I am also seeing this in Python 3.8.6. I am not using SSL, but am simply 
calling `await writer.wait_closed()` on an `asyncio.StreamWriter`. Sometimes it 
works quickly and sometimes it hangs indefinitely.

--
nosy: +r3owen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43290] [sqlite3] remove legacy code from pysqlite_step

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

Regarding test coverage:

_pysqlite_query_execute() has one coverage gap: lines 478-488 (if 
(self->statement->in_use)) are never executed. Except from that, the only 
missing are the hard-to-trigger goto error's (for example PyList_New and 
PyList_Append failures). I'm not sure if it's possible to trigger this, but I 
cannot see that it is relevant for our case here.

Coverage for the other users of pysqlite_step is as complete as it can be; only 
the standard corner error cases are missing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26111] On Windows, os.scandir will keep a handle on the directory until the iterator is exhausted

[Eryk Sun]

Eryk Sun  added the comment:

Issue 25994 added support for the context-manager protocol and close() method 
in 3.6. So it's at least much easier to ensure that the handle gets closed. 

The documentation of scandir() links to WinAPI FindFirstFile and FindNextFile, 
which at least mentions the "search handle". It's not made explicit that this 
encapsulates a handle for a kernel file object, nor are the operations (e.g. 
move, rename, delete) discussed that are allowed directly on the directory. 
Similarly, the directory stream that's returned by and used by POSIX opendir() 
and readdir() may or may not encapsulate a file descriptor. 

I don't think Python's documentation is the best place to discuss 
platform-specific implementation details in most cases. Exceptions should be 
made in some cases, but I don't think this is one of them because I can't even 
link to a document about the implementation details of FindNextFile. At a lower 
level I can link to documents about the NtQueryDirectoryFile[Ex] system call, 
but that's not much help in terms of officially documenting what FindNextFile 
does. Microsoft prefers to keep the Windows API details opaque, which gives 
them wiggle room.

FYI, in Windows 10, deleting files and directories now tries a POSIX delete (if 
supported by the filesystem) that immediately unlinks the name as soon as the 
handle that's used to perform the delete is closed, such as the handle that's 
opened to implement DeleteFile (os.unlink) and RemoveDirectory (os.rmdir). NTFS 
supports this feature by moving the file/directory to a reserved 
"\$Extend\$Deleted" directory:

>>> os.mkdir('spam')
>>> h = win32file.CreateFile('spam', 0, 0, None, 3, 0x0200_, None)
>>> print(win32file.GetFinalPathNameByHandle(h, 0))
\\?\C:\Temp\test\test\spam

>>> os.rmdir('spam')
>>> print(win32file.GetFinalPathNameByHandle(h, 0))
\\?\C:\$Extend\$Deleted\0010949A5E2FE5BB

Of course, none of the above is documented for RemoveDirectory().

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43290] [sqlite3] remove legacy code from pysqlite_step

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

pysqlite_cursor_iternext() has four users:
- sqlite3.Cursor.fetchone()
- sqlite3.Cursor.fetchall()
- sqlite3.Cursor.fetchmany()
- sqlite3.Cursor.__next__()

All of these methods pass self to pysqlite_cursor_iternext().

pysqlite_cursor_iternext() starts by checking the state of the cursor (is it 
initialised, it the database open, etc.). If there is a Statement object, 
pysqlite_step() is called with the sqlite3_stmt pointer from the Statement 
object (Cursor->statement->st).

The statement pointer of the Cursor object is set in _pysqlite_query_execute() 
– the last pysqlite_step() user – either from the LRU cache (line 470), or by 
creating a new Statement object (line 479). The latter only leaves a valid 
Cursor->statement->st pointer (sqlite3_stmt pointer) if the Statement object 
was successfully created, and the sqlite3_stmt successfully prepared. (I assume 
only valid Statement objects are added to the cache.) Before the main loop of 
_pysqlite_query_execute() starts, the statement is reset. In the loop, the next 
parameter set is fetched, the statement is (re)bound, and step is called. If 
Cursor.execute() called _pysqlite_query_execute(), the parameter list is 
initialised to a single-item list, and the loop is only run once. From what I 
can read, this function is also safe. (But it is very messy; for instance, if 
there's an active Statement, it is reset twice before the loop starts.)

I tried forcing an error by using an uninitialised cursor:
>>> cx = sqlite3.connect(":memory:")
>>> cu = sqlite3.Cursor.__new__(sqlite3.Cursor)
>>> sqlite3.Cursor.fetchone(cu)
Traceback (most recent call last):
  File "", line 1, in 
sqlite3.ProgrammingError: Base Cursor.__init__ not called.
>>> next(cu)
Traceback (most recent call last):
  File "", line 1, in 
sqlite3.ProgrammingError: Base Cursor.__init__ not called.

Ditto for fetchmany() and fetchall(). This is consistent with current behaviour.

Calling fetch*() without first executing a statement:
>>> cu = cx.cursor()
>>> cu.fetchone()
>>> cu.fetchmany()
[]
>>> cu.fetchall()
[]
>>> next(cu)
Traceback (most recent call last):
  File "", line 1, in 
StopIteration

This is consistent with current behaviour.



I might have missed something, but from what I can see, there are no paths that 
lead to pysqlite_step() being called with a NULL pointer.

Berker, Serhiy, please correct me if I'm wrong.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43293] Move note about GIL to top of threading module

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37146] opcode cache for LOAD_GLOBAL emits false alarm in memory leak hunting

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +23428
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/24643

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13368] Possible problem in documentation of module subprocess, method send_signal

[Eryk Sun]

Eryk Sun  added the comment:

Popen.send_signal() documents that sending CTRL_C_EVENT (cancel) to a process 
group is possible, which is clearly a true statement and easily demonstrated. 
OTOH, the Windows documentation of GenerateConsoleCtrlEvent() claims it's not 
possible. If you know what the system really does for this case, then the 
latter claim reads as inconsistent in spirit with the "remarks" section on the 
very same page, as well as the documentation of SetConsoleCtrlHandler(). It's 
also strangely worded to say the event isn't "received" by processes in the 
group, when it's actually about whether each individual process elects to 
ignore the cancel event that it receives.

The documentation of send_signal() could explain that the cancel event may be 
ignored in processes, which is initially the case for a new group and is 
inheritable by child processes. Sending the cancel event to all processes in 
the console session (process group 0) does nothing to resolve the general 
problem. It can help with a particular problem where the application has 
desired behavior for the cancel event and is known to be neutral about ignoring 
it, i.e. it never calls SetConsoleCtrlHandler(NULL, ...) to either enable or 
disable ignoring of the cancel event, and is known to not have been created as 
a new process group or by a parent process that ignores the cancel event.

It's worth discussing that CTRL_BREAK_EVENT can never be ignored at the process 
level. An application has to go out of its way to ignore the break event. It's 
the preferred event to send a console application when you need to terminate 
the process. Often it calls the default handler, which calls ExitProcess(), 
which at least gives shared libraries a chance to detach cleanly (i.e. 
DLL_PROCESS_DETACH). The C runtime maps the break event to SIGBREAK, and it 
also maps CTRL_CLOSE_EVENT to SIGBREAK. Thus if all you can set is a C signal 
handler, as is the case for Python scripts by default, then you need to handle 
SIGBREAK in order to exit gracefully for the cases of closing the console, 
manual Ctrl+Break, and a generated break event. This includes using Task 
Manager or taskkill.exe to non-forcefully kill the process that effectively 
owns the console session, which is implemented by sending WM_CLOSE to the 
console window.

--
versions: +Python 3.10, Python 3.9 -Python 2.7, Python 3.4, Python 3.5, Python 
3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37448] obmalloc: radix tree for tracking arena address ranges

[Neil Schemenauer]

Change by Neil Schemenauer :


Added file: https://bugs.python.org/file49834/perf_compare_radix4x.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37448] obmalloc: radix tree for tracking arena address ranges

[Neil Schemenauer]

Change by Neil Schemenauer :


Added file: https://bugs.python.org/file49833/perf_compare_noradix.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43290] [sqlite3] remove legacy code from pysqlite_step

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

There are six users of pysqlite_step():

$ grep -nrE "\" Modules/_sqlite
Modules/_sqlite/util.c:27:int pysqlite_step(sqlite3_stmt* statement, 
pysqlite_Connection* connection)
Modules/_sqlite/connection.c:393:rc = pysqlite_step(statement, self);
Modules/_sqlite/connection.c:442:rc = pysqlite_step(statement, self);
Modules/_sqlite/connection.c:493:rc = pysqlite_step(statement, self);
Modules/_sqlite/util.h:32:int pysqlite_step(sqlite3_stmt* statement, 
pysqlite_Connection* connection);
Modules/_sqlite/cursor.c:519:rc = pysqlite_step(self->statement->st, 
self->connection);
Modules/_sqlite/cursor.c:715:rc = pysqlite_step(statement, 
self->connection);
Modules/_sqlite/cursor.c:787:rc = pysqlite_step(self->statement->st, 
self->connection);

The three users in Modules/_sqlite/connection.c — _pysqlite_connection_begin(), 
pysqlite_connection_commit_impl(), and pysqlite_connection_rollback_impl() – 
are all ok, following this pattern:
1) prepare the statement
2) verify that prepare was successful, bail if not
3) call step

pysqlite_cursor_executescript() (line 715 in Modules/_sqlite/cursor.c) is also 
ok:
1) prepare the statement
2) verify that prepare was successful, bail if not
3) call step until there are no more rows

I need a little bit more time to verify _pysqlite_query_execute() and 
pysqlite_cursor_iternext().



Note to self: pysqlite_cursor_executescript() calls sqlite3_finalize() three 
times. It would have been better to break out of the loop when rc != 
SQLITE_ROW, immediately call sqlite3_finalize() (error code is preserved if 
sqlite3_step() failed), and then check rc and PyErr_Occurred(). It will make 
the code easier to follow, IMO.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42967] [CVE-2021-23336] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator

[Gregory P. Smith]

Gregory P. Smith  added the comment:

FYI - This was somewhat of an unfortuate API change.  I'm coming across code 
that relies on ; also being treated as a separator by parse_qs().  That code is 
now broken with no easy way around it.

And I'm only seeing things lucky enough to have an explicit test that happens 
to rely in some way on that behavior.  How much code doesn't?

It's been a mix of some clearly broken code (ex  appearing in the URI 
being parsed) and code where it is not so immediately obvious if there is a 
problem or not (up to the code owners to dive in and figure that out...).

The workarounds for people implementing "fixes" to previously working as 
intended rather than "oops that was a html charref" code are annoying.  Our new 
separator= parameter does not allow one to achieve the previous behavior if 
mixing and matching & And ; was intended to be allowed, as it is a single 
separator rather than a set of separators.

For security fixes, a way for people to explicitly opt-in to 
now-deemed-undesirable-by-default behavior they got from the API is desirable.  
We failed to provide that here.

Just a heads up with no suggested remediation for now.  I'm still unsure how 
big a problem this will turn out to be or not or if it is identifying actual 
worthwhile issues in code.  It's certainly a headache for a few.

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43293] Move note about GIL to top of threading module

[Quantum]

Change by Quantum :


--
pull_requests: +23427
pull_request: https://github.com/python/cpython/pull/24642

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43293] Move note about GIL to top of threading module

[miss-islington]

miss-islington  added the comment:


New changeset f82578ace103ec977cec3424b20e0b5f19cf720e by Miss Islington (bot) 
in branch '3.9':
bpo-43293: Doc: move note about GIL to top of threading module (GH-24622)
https://github.com/python/cpython/commit/f82578ace103ec977cec3424b20e0b5f19cf720e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43293] Move note about GIL to top of threading module

[miss-islington]

miss-islington  added the comment:


New changeset 2a9e6ab6adc82713e43cdf0aed09557d0d63e3bb by Miss Islington (bot) 
in branch '3.8':
bpo-43293: Doc: move note about GIL to top of threading module (GH-24622)
https://github.com/python/cpython/commit/2a9e6ab6adc82713e43cdf0aed09557d0d63e3bb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43293] Move note about GIL to top of threading module

[miss-islington]

Change by miss-islington :


--
pull_requests: +23426
pull_request: https://github.com/python/cpython/pull/24641

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43293] Move note about GIL to top of threading module

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 32181be6081f6c70a1e0bd0540050805c8e88e83 by Guanzhong Chen in 
branch 'master':
bpo-43293: Doc: move note about GIL to top of threading module (GH-24622)
https://github.com/python/cpython/commit/32181be6081f6c70a1e0bd0540050805c8e88e83


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43293] Move note about GIL to top of threading module

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +23425
pull_request: https://github.com/python/cpython/pull/24640

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24493] subprocess with env=os.environ doesn't preserve environment variables when calling a 32bit process on Windows 8.1

[Eryk Sun]

Change by Eryk Sun :


--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24977] shutil copy to non-existant directory

[Eryk Sun]

Change by Eryk Sun :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> shutil.copy raises IsADirectoryError when the directory does 
not actually exist

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39090] Document various options for getting the absolute path from pathlib.Path objects

[John Hennig]

John Hennig  added the comment:

@Floris:
> Not mentioning Path.resolve()'s behavior w.r.t. non-existing files since
that's documented in resolve() itself.

I don't see it mentioned in the documentation of `resolve()`, or anywhere else 
in the docs, that on Windows (but not on other platforms) `resolve()` does 
*not* resolve a relative path to an absolute path if the file does not exist. 
As opposed to `absolute()`, which works as expected on any platform.

Linux:
```
Python 3.6.9 (default, Oct  8 2020, 12:12:24)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> file = Path('new.txt')
>>> file.exists()
False
>>> file.resolve()
PosixPath('/home/user/new.txt')
```

Windows:
```
Python 3.9.2 (tags/v3.9.2:1a79785, Feb 19 2021, 13:44:55) [MSC v.1928 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> file = Path('new.txt')
>>> file.exists()
False
>>> file.resolve()
WindowsPath('new.txt')
>>> file.absolute()
WindowsPath('d:/home/new.txt')
>>> file.touch()
>>> file.resolve()
WindowsPath('D:/home/new.txt')
>>> file.unlink()
>>> file.resolve()
WindowsPath('new.txt')
>>>
```

--
nosy: +John-Hennig

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43310] Method __del__ with callable

[Mark Dickinson]

Mark Dickinson  added the comment:

Postscript: the "MethodType" that appears in the previous comment can be 
imported from the types module: "from types import MethodType".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43310] Method __del__ with callable

[Mark Dickinson]

Mark Dickinson  added the comment:

The key difference between the func_del function and the instance C() is that 
func_del is a (non-data) descriptor in addition to being callable, while the 
instance C() is not. That makes func_del usable as a method.

If you define C as follows, you'll see the behaviour you expect:


class C:
def __call__(self, *args, **kwargs):
print("call", args, kwargs)

def __get__(self, obj, objtype=None):
  return MethodType(self, obj)

Useful reference for descriptors: 
https://docs.python.org/3/howto/descriptor.html

I'll close here, since this is neither a bug nor a feature request.

--
nosy: +mark.dickinson
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22080] Add windows_helper module helper

[Eryk Sun]

Eryk Sun  added the comment:

I rewrote windows_helper.py with cleaner ctypes code, better error handling, 
the correct implementation of restoring the previous privilege state, and 
without leaking the handle for the process token. It's kind of limited as a 
"Windows helper" module. It's just handling privileges, at least for now. This 
helper was intended to help with testing bpo-21518 (add winreg.UnLoadKey), 
another contribution from Claudiu.

--
versions: +Python 3.10, Python 3.9 -Python 3.5, Python 3.6
Added file: https://bugs.python.org/file49832/windows_helper.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35077] Make TypeError message less ambiguous

[coyot linden]

coyot linden  added the comment:

3.8 has come and gone, resubmitting this for 3.10

--
versions: +Python 3.10 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43317] python -m gzip could use a larger buffer

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43235] Tools/scripts/stable_abi.py should also check PC/python3dll.c (Windows stable ABI)

[hai shi]

hai shi  added the comment:

> It would be great to have a tool to check that all symbols exported on Unix 
> are also exported on Windows.

I write a demo in PR 24639(compare the stable_abi.dat to python3dll.c).
And the output result:
```
Some symbols from the limited API on windows are missing:
PyMarshal_WriteObjectToFile, PyThread_allocate_lock, PyObject_GC_IsFinalized, 
PyThread_ReInitTLS, PyThread_acquire_lock, PyThread_GetInfo, 
PyDictRevIterValue_Type, PyThread_get_thread_native_id, 
PyThread_start_new_thread, PyThread_set_stacksize, PyDictRevIterKey_Type, 
PyOS_AfterFork_Child, PyThread_create_key, PyThread_get_thread_ident, 
PyOS_BeforeFork, Py_BytesMain, PyThread_get_key_value, PyCodec_Unregister, 
PyInterpreterState_Get, PyMember_GetOne, PyThread_init_thread, 
PyInterpreterState_GetDict, PyThread_set_key_value, PyThread_delete_key, 
PyMarshal_WriteObjectToString, PyOS_AfterFork_Parent, 
PyThread_delete_key_value, PyObject_GC_IsTracked, 
PyMarshal_ReadObjectFromString, PyMarshal_WriteLongToFile, 
PyThread_exit_thread, PyThread_acquire_lock_timed, PyThread_get_stacksize, 
PyMember_SetOne, PyDictRevIterItem_Type, PyThread_free_lock, 
PyThread_release_lock

This error normally means that there are some missing symbols on windows.

Check if this was a mistake and if not, update the file containing the limited
API symbols. This file is located at:

PC/python3dll.c

You can read more about the limited API and its contracts at:

https://docs.python.org/3/c-api/stable.html

And in PEP 384:

https://www.python.org/dev/peps/pep-0384/

```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43310] Method __del__ with callable

[Erik Soma]

Erik Soma  added the comment:

You can wrap your callable in a regular function:
```
def hack_c():
c = C()
def _(*args, **kwargs):
return c(*args, **kwargs)
return _
A.__del__ = hack_c()
```

Or (untested) make your callable an extension type with 
Py_TPFLAGS_METHOD_DESCRIPTOR.


Or instead of monkey-patching __del__ make __del__ call it:

```
class A:
   my_del = lambda *args, **kwargs: None
   def __del__(self):
   self.my_del(self)
A.my_del = C()
```



This doesn't just apply to __del__, other dunders exhibit this behavior as 
well. It is unintuitive, but I'm pretty sure it's not a bug.

--
nosy: +esoma

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43235] Tools/scripts/stable_abi.py should also check PC/python3dll.c (Windows stable ABI)

[hai shi]

Change by hai shi :


--
keywords: +patch
nosy: +shihai1991
nosy_count: 4.0 -> 5.0
pull_requests: +23424
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24639

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43290] [sqlite3] remove legacy code from pysqlite_step

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

I’ll check all uses and see if we’ve got everything covered by the test suite. 
This is one of the core functions of the sqlite3 module (all queries call step 
at least once, but often multiple times), so I’d expect coverage is pretty good.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43315] Decimal.__str__ has no way to force exact decimal representation

[Szymon]

Szymon  added the comment:

Thanks for the replies. The use of format works great. Maybe it's worth 
mentioning in the documentation because using "%f" % Decimal("0.0001") leds 
to loosing precision (cast to float64 I guess) while using 
f"{"Decimal('0.0001'):f}" does the thing correctly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43305] A typo in /Modules/_io/bufferedio.c

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +serhiy.storchaka, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43315] Decimal.__str__ has no way to force exact decimal representation

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Python 3.5 to 3.9 are all in feature freeze and can accept no new features; new 
features can only be added to 3.10.

But having said that, I agree with Mark that the correct solution here is to 
use format, not str. Szymon, unless you have a very argument against format, I 
think this should be closed.

--
nosy: +steven.daprano
versions:  -Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43317] python -m gzip could use a larger buffer

[Ruben Vorderman]

New submission from Ruben Vorderman :

python -m gzip reads in chunks of 1024 bytes: 
https://github.com/python/cpython/blob/1f433406bd46fbd00b88223ad64daea6bc9eaadc/Lib/gzip.py#L599

This hurts performance somewhat. Using io.DEFAULT_BUFFER_SIZE will improve it. 
Also 'io.DEFAULT_BUFFER_SIZE' is better than: 
'ARBITRARY_NUMBER_WITH_NO_COMMENT_EXPLAINING_WHY'.

With 1024 blocks
Decompression:
$ hyperfine -r 10 -w 3 'cat ~/test/50reads.fastq.gz | ./prefix/bin/python3 
-m gzip -d > /dev/null'
Benchmark #1: cat ~/test/50reads.fastq.gz | ./prefix/bin/python3 -m gzip -d 
> /dev/null
  Time (mean ± σ): 926.9 ms ±   7.7 ms[User: 901.2 ms, System: 59.1 ms]
  Range (min … max):   913.3 ms … 939.4 ms10 runs

Compression:
$ hyperfine -r 10 -w 3 'cat ~/test/50reads.fastq | ./prefix/bin/python3 -m 
gzip --fast > /dev/null'
Benchmark #1: cat ~/test/50reads.fastq | ./prefix/bin/python3 -m gzip 
--fast > /dev/null
  Time (mean ± σ):  2.514 s ±  0.030 s[User: 2.469 s, System: 0.125 s]
  Range (min … max):2.472 s …  2.563 s10 runs


with io.DEFAULT_BUFFER_SIZE
Decompression:
$ hyperfine -r 10 -w 3 'cat ~/test/50reads.fastq.gz | ./prefix/bin/python3 
-m gzip -d > /dev/null'
Benchmark #1: cat ~/test/50reads.fastq.gz | ./prefix/bin/python3 -m gzip -d 
> /dev/null
  Time (mean ± σ): 839.9 ms ±   7.3 ms[User: 816.0 ms, System: 57.3 ms]
  Range (min … max):   830.1 ms … 851.3 ms10 runs

Compression:
$ hyperfine -r 10 -w 3 'cat ~/test/50reads.fastq | ./prefix/bin/python3 -m 
gzip --fast > /dev/null'
Benchmark #1: cat ~/test/50reads.fastq | ./prefix/bin/python3 -m gzip 
--fast > /dev/null
  Time (mean ± σ):  2.275 s ±  0.024 s[User: 2.247 s, System: 0.096 s]
  Range (min … max):2.254 s …  2.322 s10 runs


Speedups: 
- Decompression 840 / 927 = 0.906 ~= 9% reduction in runtime
- Compression 2.275 / 2.514 = 0.905 ~= 9% reduction in runtime.

It is not stellar, but it is a quite nice improvement for such a tiny change.

--
components: Library (Lib)
messages: 387624
nosy: rhpvorderman
priority: normal
severity: normal
status: open
title: python -m gzip could use a larger buffer
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43290] [sqlite3] remove legacy code from pysqlite_step

[Dong-hee Na]

Dong-hee Na  added the comment:

Hmm by the way the current implementation returns SQLITE_OK if the statement is 
NULL, but it looks like return SQLITE_MISUSE if we apply this patch.

Does it not cause any behavior regression? if so we should add news also.

--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43316] python -m gzip handles error incorrectly

[Ruben Vorderman]

New submission from Ruben Vorderman :

`Python -m gzip -d myfile` will throw an error because myfile does not end in 
'.gz'. That is fair (even though a bit redundant, GzipFile contains a header 
check, so why bother checking the extension?).

The problem is how this error is thrown.
1. Error is printed to stdout instead of stderr
2. Tool exits with exit 0.

This is not the behaviour that is expected when using python -m gzip in a 
script.

The error is even codified in a test: 
https://github.com/python/cpython/blob/1f433406bd46fbd00b88223ad64daea6bc9eaadc/Lib/test/test_gzip.py#L776

def test_decompress_infile_outfile_error(self):
rc, out, err = assert_python_ok('-m', 'gzip', '-d', 'thisisatest.out')
self.assertIn(b"filename doesn't end in .gz:", out)
self.assertEqual(rc, 0)
self.assertEqual(err, b'')

This should be assert_python_failure, out and err should be swapped, and exit 
code should be something different than 0.

>From the zen of python: Errors should never pass silently.

I am willing to fix this in a PR, but first I would like some feedback on how 
to solve this exactly. 

I propose raising a ValueError("can not determine output filename: 'myfile' 
does not end in '.gz'").

--
components: Library (Lib)
messages: 387622
nosy: rhpvorderman
priority: normal
severity: normal
status: open
title: python -m gzip handles error incorrectly
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43308] subprocess.Popen leaks file descriptors opened for DEVNULL or PIPE stdin/stdout/stderr arguments

[Alexey Izbyshev]

Change by Alexey Izbyshev :


--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43315] Decimal.__str__ has no way to force exact decimal representation

[Mark Dickinson]

Mark Dickinson  added the comment:

This is by design: the choice of -6 is not arbitrary, but follows the standard 
that the Decimal class is based on. Quoting from 
http://speleotrove.com/decimal/daconvs.html#reftostr:

> If the exponent is less than or equal to zero and the adjusted exponent is 
> greater than or equal to -6, [...]

"str" isn't parameterizable (at least, not without a major change to the way 
that Python works), so if you're using it you have to accept the compromises it 
makes. If you want finer control over the string representation of Decimal 
objects, use Python's formatting capabilities:

>>> from decimal import Decimal
>>> x = Decimal("1e-8")
>>> format(x, 'f')
'0.0001'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43315] Decimal.__str__ has no way to force exact decimal representation

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +facundobatista, mark.dickinson, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43251] [sqlite3] sqlite3_column_name() failures should raise MemoryError

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

I believe we can proceed with this as planned. Serhiy, do you have additional 
comments or change requests?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23634] os.fdopen reopening a read-only fd on windows

[Eryk Sun]

Change by Eryk Sun :


--
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43290] [sqlite3] remove legacy code from pysqlite_step

[Erlend Egeberg Aasland]

Change by Erlend Egeberg Aasland :


--
keywords: +patch
pull_requests: +23423
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24638

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43290] [sqlite3] remove legacy code from pysqlite_step

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

>From the SQLite 3.5.3 changelog:
- sqlite3_step() returns SQLITE_MISUSE instead of crashing when called with a 
NULL parameter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43315] Decimal.__str__ has no way to force exact decimal representation

[Szymon]

New submission from Szymon :

str(Decimal("0.0001")) always returns "1E-8" and there is no way to 
directly get the "0.0001" string back without writting custom method for 
converting DecimalTuple to string. This is caused by arbitrary choice of `and 
leftdigits > -6` condition in 
https://github.com/python/cpython/blob/master/Lib/_pydecimal.py#L1052 .
The hardcoded value of -6 should be parametrizable. This can be done by adding 
it as argument to the __str__ method or maybe by adding it to the decimal 
context.

--
messages: 387618
nosy: sim1234
priority: normal
severity: normal
status: open
title: Decimal.__str__ has no way to force exact decimal representation
type: enhancement
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43314] [sqlite3] remove pre SQLite 3.7.7 support code

[Erlend Egeberg Aasland]

Change by Erlend Egeberg Aasland :


--
keywords: +patch
pull_requests: +23422
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24637

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

[Tzu-ping Chung]

Tzu-ping Chung  added the comment:

That would work as well (raising KeyError or ValueError if `name` is not valid?)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43314] [sqlite3] remove pre SQLite 3.7.7 support code

[Erlend Egeberg Aasland]

New submission from Erlend Egeberg Aasland :

I missed the SQLITE_OPEN_URI #ifdef in GH-24106. The "open URI" interface was 
added in 3.7.7. The dead code can safely be removed.

$ grep -r SQLITE_OPEN_URI sqlite-3.7.6/
$ grep -r SQLITE_OPEN_URI sqlite-3.7.7/
[...]
sqlite-3.7.7/sqlite3.h:#define SQLITE_OPEN_URI  0x0040  /* Ok 
for sqlite3_open_v2() */

--
components: Library (Lib)
messages: 387616
nosy: berker.peksag, erlendaasland, serhiy.storchaka
priority: normal
severity: normal
status: open
title: [sqlite3] remove pre SQLite 3.7.7 support code
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

[Christian Heimes]

Christian Heimes  added the comment:

Do you need all three items or would "get_preferred_scheme(name: str) -> str" 
be sufficient?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43313] feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state

[junyixie]

New submission from junyixie :

https://github.com/ericsnowcurrently/multi-core-python/issues/73

https://github.com/JunyiXie/cpython/commit/820954879fd546fcb29b654d10c424bd47da70ce
changes:
move pymalloc state in obmalloc.h
_is add pymalloc_state
pymalloc_allocxx api use subinterpreter pymalloc_state

--
components: Subinterpreters
messages: 387614
nosy: JunyiXie
priority: normal
severity: normal
status: open
title: feature: support pymalloc for subinterpreters. each subinterpreter has 
pymalloc_state

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43313] feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state

[junyixie]

Change by junyixie :


--
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

[mattip]

mattip  added the comment:

I wonder if the distro maintainers might be able to use this to reduce the 
patching they do for their schema?

--
nosy: +mattip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

[Tzu-ping Chung]

Tzu-ping Chung  added the comment:

Adding Christian to the nosy list since IIRC you said something related to this 
a while ago.

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

[Tzu-ping Chung]

New submission from Tzu-ping Chung :

While trying to migrate pip’s internal usages of distutils to sysconfig,[1] I 
noticed there isn’t a way for pip to select a scheme for sysconfig.get_paths() 
for `pip install --target` and `pip install --user`. I tried to implement some 
logic to "guess" a scheme ("posix_home" for `--home`, "nt_user" for `--user` 
when os.name is "nt", etc.), but eventually hit a wall trying to support 
alternative implementations.

PyPy, for example, adds additional schemes "pypy" and "pypy_nt", and it’s not 
clear whether pip should use then for `--home` or not. @mattip helped clear 
this up for PyPy (which also prompts bpo-43307), but we are worried that other 
implementations may introduce even more special rules that causes problems, and 
it’s also not a good idea for pip to implement special logic for every 
implementation.

I would propose two changes to sysconfig:

1. Make sysconfig._get_default_scheme() a public function. This function will 
be documented for implementations to return a default scheme to use when none 
is given to sysconfig.get_paths().
2. Add a new function sysconfig.get_preferred_schemes() for implementations to 
return preferred schemes for prefix, home, and user installations. This 
function should return a dict[str, str] with three keys "prefix", "home", and 
"user", and their values the scheme names to use.

I would be happy to work on a PR and iterate on the design if this sounds like 
a reasonable idea. For CPython, the implementation would be something like (to 
match distutils’s behaviour):

def get_preferred_schemes():
if os.name == "nt":
return {
"prefix": "nt",
"home": "posix_home",
"user": "nt_user",
}
return {
"prefix": "posix_prefix",
"home": "posix_home",
"user": "posix_user",
}


[1]: https://github.com/pypa/pip/pull/9626

--
components: Library (Lib)
messages: 387611
nosy: uranusjr
priority: normal
severity: normal
status: open
title: Interface to select preferred "user" or "home" sysconfig scheme for an 
environment
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43311] PyInterpreterState_New use thread tstate before set.

[junyixie]

Change by junyixie :


--
keywords: +patch
pull_requests: +23420
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24636

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43311] PyInterpreterState_New use thread tstate before set.

[junyixie]

New submission from junyixie :

PyInterpreterState_New use thread tstate before set.


PyInterpreterState_New use tstate. but tstate is not set.
tstate will set after PyInterpreterState_New.

PyInterpreterState *interp = PyInterpreterState_New();
if (interp == NULL) {
*tstate_p = NULL;
return _PyStatus_OK();
}

PyThreadState *tstate = PyThreadState_New(interp);
if (tstate == NULL) {
PyInterpreterState_Delete(interp);
*tstate_p = NULL;
return _PyStatus_OK();
}

PyThreadState *save_tstate = PyThreadState_Swap(tstate);

--
components: Subinterpreters
messages: 387610
nosy: JunyiXie
priority: normal
severity: normal
status: open
title: PyInterpreterState_New use thread tstate before set.
type: crash
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com