[issue44115] Improve conversions for fractions

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee: rhettinger -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44139] Unparenthesized tuple doc bug in what's new

[Michael Cuthbert]

New submission from Michael Cuthbert :

The What's New in Python 3.10 docs demonstrates a richer SyntaxError for this 
set comprehension:

>>> {x,y for x,y in range(100)}
  File "", line 1
{x,y for x,y in range(100)}
 ^
SyntaxError: did you forget parentheses around the comprehension target?


The problem (at least for beginners) is that there are two errors in the line.  
Parenthesizing the comprehension target gets "TypeError: cannot unpack 
non-iterable int object" since "x,y in range(...)" does not work.  I think it 
would be better to illustrate with code that works if not for the missing 
parentheses.  Something like this:

>>> {x,y for x,y in zip('abcd', 'efgh')}
  File "", line 1
{x,y for x,y in zip('abcd', 'efgh')}
 ^
SyntaxError: did you forget parentheses around the comprehension target?

Thanks for the great work on making error messages better! This prof. who often 
has to help students with Python errors heartily approves!

--
assignee: docs@python
components: Documentation
messages: 393704
nosy: docs@python, mscuthbert
priority: normal
severity: normal
status: open
title: Unparenthesized tuple doc bug in what's new
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44126] Cross Compile CPython Modules

[Jeff Moguillansky]

Jeff Moguillansky  added the comment:

Thanks for the info, I will forward the question to the setuptools mailing list

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44126] Cross Compile CPython Modules

[Jeff Moguillansky]

Change by Jeff Moguillansky :


--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44126] Cross Compile CPython Modules

[Jeff Moguillansky]

Change by Jeff Moguillansky :


--
title: Cross Compile Cython Modules -> Cross Compile CPython Modules

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30274] Rename 'name' to 'fullname' argument to ExtensionFileLoader

[Irit Katriel]

Irit Katriel  added the comment:

The PR was reviewed but it now has merge conflicts and possibly a test failure 
on MacOS.

--
components: +Library (Lib) -Documentation
keywords: +easy
nosy: +iritkatriel
title: Make importlib.abc.ExtensionFileLoader.__init__() documentation match 
code -> Rename 'name' to 'fullname' argument to ExtensionFileLoader
versions: +Python 3.11 -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44072] Doc: power operator (`**`) present for numbers.Complex, not added in numbers.Integral

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Thanks for the report and clean patch.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44107] HTTPServer can't close http client completely

[Terry J. Reedy]

Terry J. Reedy  added the comment:

3.8 only gets security fixes.  Please verify issue on a current release.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44081] ast.unparse: dont use redundant space separator for lambdas with no parameters

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Agreed. unparse is already correct if there is a parameter.
>>> ast.unparse(ast.parse("lambda x: 'hey!'"))
"lambda x: 'hey!'"

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44072] Doc: power operator (`**`) present for numbers.Complex, not added in numbers.Integral

[miss-islington]

miss-islington  added the comment:


New changeset fa9de0c383581936de3338c8dc6b58c10a6e27b8 by Miss Islington (bot) 
in branch '3.9':
bpo-44072: fix Complex, Integral docs for `**` (GH-25986)
https://github.com/python/cpython/commit/fa9de0c383581936de3338c8dc6b58c10a6e27b8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44072] Doc: power operator (`**`) present for numbers.Complex, not added in numbers.Integral

[miss-islington]

miss-islington  added the comment:


New changeset 2e99869f64bbd3c6590cb5ceaf9cf59e63689d63 by Miss Islington (bot) 
in branch '3.10':
bpo-44072: fix Complex, Integral docs for `**` (GH-25986)
https://github.com/python/cpython/commit/2e99869f64bbd3c6590cb5ceaf9cf59e63689d63


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40358] pathlib's relative_to should behave like os.path.relpath

[Terry J. Reedy]

Terry J. Reedy  added the comment:

On #44078 (closed as duplicate), Mark Hammond made a similar request.

--
nosy: +terry.reedy
versions: +Python 3.11 -Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44078] Output relative path when using PurePath.relative_to

[Terry J. Reedy]

Change by Terry J. Reedy :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> pathlib's relative_to should behave like os.path.relpath

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44138] multiprocessing documentation should note behavior when process with Lock dies

[Kushal Kumaran]

New submission from Kushal Kumaran :

If a process holding a multiprocessing.Lock dies, other waiters on the lock 
will be stuck.  This is mostly the same as with threads, but threads-users can 
usually avoid this with careful coding (dealing with errors, etc.), and 
significant crashes will normally take down the entire process anyway.  With 
multiprocessing, a process with lock held could get SIGKILLed with no recourse.

A simple program demonstrating the problem:
```
import multiprocessing
import os
import signal

lk = multiprocessing.Lock()

def f():
my_pid = os.getpid()
print("PID {} going to wait".format(my_pid))
with lk:
print("PID {} got the lock".format(my_pid))
os.kill(my_pid, signal.SIGKILL)

if __name__ == '__main__':
for i in range(5):
multiprocessing.Process(target=f).start()
```

Running this will have one of the processes acquiring the lock and dying; the 
other processes will wait forever.

The reason behind this behavior is obvious from the implementation that uses 
POSIX semaphores (I don't know how the win32 implementation behaves).

I don't think the behavior can be changed, since releasing the lock on process 
crash could leave other processes having to deal with unexpected state. A note 
in the documentation for the multiprocessing module is all I could think of. I 
don't see a way to use multiprocessing.Lock with safety against process 
crashes. If someone has a scenario where they can guarantee their data 
consistency in the face of process crash, they should use some alternative 
mechanism such as file-based locking.

--
components: Library (Lib)
messages: 393695
nosy: kushal-kumaran
priority: normal
severity: normal
status: open
title: multiprocessing documentation should note behavior when process with 
Lock dies
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 
3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44072] Doc: power operator (`**`) present for numbers.Complex, not added in numbers.Integral

[miss-islington]

Change by miss-islington :


--
pull_requests: +24774
pull_request: https://github.com/python/cpython/pull/26140

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44072] Doc: power operator (`**`) present for numbers.Complex, not added in numbers.Integral

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +24773
pull_request: https://github.com/python/cpython/pull/26139

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42862] Use functools.lru_cache iso. _sqlite.Cache in sqlite3 module

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
versions: +Python 3.11 -Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44072] Doc: power operator (`**`) present for numbers.Complex, not added in numbers.Integral

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Yes, complex ** complex works fine, and "TypeError: pow() 3rd argument not 
allowed unless all arguments are integers".

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44133] "Py_FrozenMain" symbol is not exported

[STINNER Victor]

STINNER Victor  added the comment:

The symbol is not exported on Windows neither. My PR 26126 ("Test 
Py_FrozenMain()") fails on Windows with:

"_testembed.obj : error LNK2001: unresolved external symbol __imp_Py_FrozenMain"

("__imp_" prefix is added by Windows DLL loader.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43650] MemoryError on zip.read in shutil._unpack_zipfile

[Igor Bolshakov]

Igor Bolshakov  added the comment:

pls review

--
resolution:  -> remind

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38768] [feature request] Add lldb equivalent to Tools/gdb

[Ned Deily]

Change by Ned Deily :


--
components:  -macOS

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43729] Tutorial Documentation for 3.1.1. Numbers missing "result"

[Carol Willing]

Carol Willing  added the comment:


New changeset c90642b0d48e5ab40d05b526311ba19bb87f3233 by Miss Islington (bot) 
in branch '3.9':
bpo-43729: Clarify comment in tutorial example (GH-25191) (GH-26137)
https://github.com/python/cpython/commit/c90642b0d48e5ab40d05b526311ba19bb87f3233


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43729] Tutorial Documentation for 3.1.1. Numbers missing "result"

[Carol Willing]

Carol Willing  added the comment:


New changeset dd4f7d96cb78b424ef8fa705aaee15a837b2d240 by Miss Islington (bot) 
in branch '3.10':
bpo-43729: Clarify comment in tutorial example (GH-25191) (#26136)
https://github.com/python/cpython/commit/dd4f7d96cb78b424ef8fa705aaee15a837b2d240


--
nosy: +willingc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38768] [feature request] Add lldb equivalent to Tools/gdb

[Gregory P. Smith]

Gregory P. Smith  added the comment:

This would be useful beyond macOS.  Clang/LLVM is the compiler toolchain of 
choice for a lot of the world these days.  Including lldb.  gdb cannot be 
assumed to be available or even work everywhere.  This would be a great thing 
to have.

--
components: +Demos and Tools
nosy: +gregory.p.smith
stage:  -> needs patch
type:  -> enhancement
versions:  -Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34903] strptime %d handling of single digit day of month

[Huziy Oleksandr]

Huziy Oleksandr  added the comment:

This is a bit unfortunate for the formats without separators, that could give 
wrong results silently:

```
In [26]: datetime.strptime("2020010112", "%Y%m%d%H")
Out[26]: datetime.datetime(2020, 1, 1, 12, 0)

In [27]: datetime.strptime("2020010112", "%Y%m%d%H%M")
Out[27]: datetime.datetime(2020, 1, 1, 1, 2)
```

--
nosy: +guziy.sasha
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37658] In some cases asyncio.wait_for can lead to socket leak.

[nmatravolgyi]

nmatravolgyi  added the comment:

I did a little research on how the `wait_for` could be improved. There are four 
alternate implementations you can find here: 
https://github.com/Traktormaster/wait-for2/blob/issue37658/wait_for2/impl.py

The repository (on the linked branch) has tox set-up and a test that asserts 
the behaviour of builtin and alternate implementations.

Quick summary:
  - wait_for (builtin): either leaks resources or ignores cancellation
  - wait_for (preferred alt): works as expected (I'm going to discuss this 
below)
  - wait_for_special_raise: works, but has pitfalls
  - wait_for_no_waiter*: These were proposed by @aaliddell on the related PR: 
https://github.com/python/cpython/pull/26097#issuecomment-840497455 I could not 
make them work, but I might be missing something. I think the fact that the 
inner coroutine gets wrapped into a Future introduces the race-condition, but I 
don't know how to test that. The general idea of this would be the best if an 
implementation was possible.


About the actually working alternate implementation I made:
In my opinion there is no way to implicitly handle losing a result properly in 
case of a cancellation, since it arbitrarily breaks the flow of execution. I'd 
look into having `wait_for(...)` support cleanup callbacks when a cancellation 
and completion happens at the same time. Something like:

```python

async def create_something():
# this acquires some resource that needs explicit cleanup after some work
return object()

def cleanup_something(inst):
inst.close()

t = asyncio.ensure_future(create_something())
x = await asyncio.wait_for(t, timeout=10, cancel_handler=cleanup_something)
try:
pass  # normal work with x
finally:
cleanup_something(x)  # cleanup at normal execution
```

The inner task is still responsible for handling the resource before it 
returns, which means if the inner task is cancelled, there must be no leak if 
the implementation is correct. If no cancellation happens, everything is 
"fine". Finally, the waiter task would be able to handle its cancellation and 
the simultaneous completion of the inner task if the caller code provides a 
callback.

Unfortunately this requires the user of `wait_for` to be aware of this 
race-condition. However it makes it possible to be handled properly when the 
waited future's result requires a cleanup.

--
nosy: +nmatravolgyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37788] fix for bpo-36402 (threading._shutdown() race condition) causes reference leak

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 14.0 -> 15.0
pull_requests: +24772
pull_request: https://github.com/python/cpython/pull/26138

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44137] importlib.resources.path raises RuntimeError when FileNotFoundError is raise in context manager

[Nils Kattenbeck]

Change by Nils Kattenbeck :


--
title: importlib.resources.path raises RuntimeError import FileNotFoundError is 
raise in context manager -> importlib.resources.path raises RuntimeError when 
FileNotFoundError is raise in context manager

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44137] importlib.resources.path raises RuntimeError import FileNotFoundError is raise in context manager

[Nils Kattenbeck]

New submission from Nils Kattenbeck :

When a FileNotFoundError is raised inside while the importlib.resources.path 
context manager is active a RuntimeError is raised.
Looking at the (3.8) code it seems that FileNotFound exceptions are handled 
specially from all other exceptions which may lead to this behaviour. While the 
code in 3.9 changed significantly the same behaviour can be observed.

Files:
.
└── my_package
├── data.txt (empty)
├── __init__.py (empty)
└── test.py

Content of test.py:
import importlib.resources
def main():
with importlib.resources.path('my_package', 'data.txt') as p:
raise FileNotFoundError()
if __name__ == '__main__':
main()

Exact error message:
RuntimeError: generator didn't stop after throw()

--
components: Library (Lib)
messages: 393686
nosy: Nils Kattenbeck, brett.cannon, jaraco
priority: normal
severity: normal
status: open
title: importlib.resources.path raises RuntimeError import FileNotFoundError is 
raise in context manager
type: behavior
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43729] Tutorial Documentation for 3.1.1. Numbers missing "result"

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 5.0 -> 6.0
pull_requests: +24770
pull_request: https://github.com/python/cpython/pull/26136

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43729] Tutorial Documentation for 3.1.1. Numbers missing "result"

[miss-islington]

Change by miss-islington :


--
pull_requests: +24771
pull_request: https://github.com/python/cpython/pull/26137

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43729] Tutorial Documentation for 3.1.1. Numbers missing "result"

[Irit Katriel]

Change by Irit Katriel :


--
versions: +Python 3.10, Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44136] Remove pathlib flavours

[Barney Gale]

New submission from Barney Gale :

Following bpo-39899, bpo-43757 and bpo-43012, `pathlib._Flavour` and its 
subclasses are looking a bit pointless.

The implementations of `is_reserved()` and `make_uri()` (~`as_uri()`) can be 
readily moved to into `PurePosixPath` and `PureWindowsPath`, which removes some 
indirection. This follows the pattern of OS-specific stuff in `PosixPath` and 
`WindowsPath`.

The remaining methods, such as `splitroot()`, can be pulled into `Pure*Path` 
with an underscore prefix.

I'm generally a believer in composition over inheritance, but in this case 
`_Flavour` seems too small and too similar to `PurePath` to separate out into 3 
extra classes.

There should be no impact on public APIs or performance.

--
components: Library (Lib)
messages: 393685
nosy: barneygale
priority: normal
severity: normal
status: open
title: Remove pathlib flavours
type: behavior
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44135] issubclass documentation doesn't explain tuple semantic

[João Henrique Pimentel]

New submission from João Henrique Pimentel :

The second parameter (classinfo) of the issubclass built-in function can be a 
Tuple and, starting from 3.10, it can be a Union Type as well.

The documentation states that in these cases "every entry in classinfo will be 
checked", but it doesn't explain if the check is AND (all) or OR (any). In 
contrast, the documentation of isinstance is clear: "return True if object is 
an instance of *any* of the types".


Here's a possible rewriting that reduces the odds of incorrect interpretations, 
based on the text of isinstance:
ORIGINAL: "in which case every entry in classinfo will be checked"
PROPOSAL: "in which case return True if class is a subclass of any entry in 
classinfo"

--
assignee: docs@python
components: Documentation
messages: 393684
nosy: docs@python, joaozinho
priority: normal
severity: normal
status: open
title: issubclass documentation doesn't explain tuple semantic
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 
3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44095] Add suffix property to zipfile.Path

[Jason R. Coombs]

Jason R. Coombs  added the comment:

Thanks for the excellent patch. I've created 
https://github.com/jaraco/zipp/issues/74 to track the backport of these changes.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44095] Add suffix property to zipfile.Path

[Jason R. Coombs]

Jason R. Coombs  added the comment:

Sounds good to me. In general, it's easier to contribute first to 
https://github.com/jaraco/zipp and those changes can be readily merged into 
CPython. No worries, though. If these changes can be accepted here, I should be 
able to graft them onto jaraco/zipp as well, and thus provide the functionality 
for older Pythons.

At this point, Python 3.10 is frozen for features, which this change would be, 
so I'm updating the target version to 3.11.

> It's not meant to be 100% consistent with pathlib.Path.

That's right. It's meant to provide whatever consistency makes sense, but a 
subset of what's available on pathlib.Path. Happy to accept any contributions 
that would be useful.

>> A pathlib-compatible interface
> only a few methods are supported

Indeed, perhaps the documentation should be more clear about what is and isn't 
supported.

--
versions: +Python 3.11 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44134] lzma: stream padding in xz files

[rogdham]

Change by rogdham :


Added file: https://bugs.python.org/file50045/example2.xz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44134] lzma: stream padding in xz files

[rogdham]

New submission from rogdham :

Hello,

The lzma module does not works well with XZ stream padding. Depending on the 
case, it may work; or it may stops the stream prematurely without error; or an 
error may be raised; or no error may be raised when it must.


In the XZ file format, stream padding is a number of null bytes (multiple of 4) 
that can be between and after streams.

>From the specification (section 2.2):

> Only the decoders that support decoding of concatenated Streams MUST support 
> Stream Padding.

Since the lzma module supports decoding of concatenated streams, it must 
support stream padding as well.



 Examples to reproduce the issue:

1. example1.xz:
- made of one stream followed by 4 null bytes:
$ (echo 'Hi!' | xz; head -c 4 /dev/zero) > example1.xz
- will raise an exception in both modes (FORMAT_AUTO and FORMAT_XZ)

>>> with lzma.open('/example1.xz', format=lzma.FORMAT_AUTO) as f:
... f.read()
...
Traceback (most recent call last):
  File "", line 2, in 
  File "/usr/lib/python3.9/lzma.py", line 200, in read
return self._buffer.read(size)
  File "/usr/lib/python3.9/_compression.py", line 99, in read
raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached
>>> with lzma.open('/example1.xz', format=lzma.FORMAT_XZ) as f:
... f.read()
...
Traceback (most recent call last):
  File "", line 2, in 
  File "/usr/lib/python3.9/lzma.py", line 200, in read
return self._buffer.read(size)
  File "/usr/lib/python3.9/_compression.py", line 99, in read
raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached


2. example2.xz:
- made of two streams with 18 null bytes of stream padding between them
$ (echo 'Hi!' | xz; head -c 18 /dev/zero; echo 'Second stream' | xz) > 
example2.xz
- second stream will be ignored with FORMAT_XZ
- the two streams will be decoded with FORMAT_AUTO, where it should raise 
an error (18 null bytes is not multiple of 4, so the stream padding is invalid 
according to the XZ specification and the decoder “MUST indicate an error”)

>>> with lzma.open('/tmp/example2.xz', format=lzma.FORMAT_AUTO) as f:
... f.read()
...
b'Hi!\nSecond stream\n'
>>> with lzma.open('/tmp/example2.xz', format=lzma.FORMAT_XZ) as f:
... f.read()
...
b'Hi!\n'



 Analysis

This issue comes from the relation between _lzma and _compression. In _lzma, 
the C library is called without the LZMA_CONCATENATED flag, which means that 
multiple streams and stream padding must be supported in Python.

In _compression, when a LZMADecompressor is done (.eof is True), an other one 
is created to decompress from that point. If the new one fails to decompress 
the remaining data, the LZMAError is ignored and we assume we reached the end.

So the behavior seen above can be explained as follows:
 - In FORMAT_AUTO, it seems that .eof is False while we haven't read 18 bytes
 - In FORMAT_AUTO, 18 null bytes will be decompressed as b'' with .eof being 
True afterwards
 - In FORMAT_XZ, it seems that .eof is False while we haven't read 12 bytes
 - In FORMAT_XZ, no stream padding is valid, so as soon as we have more than 12 
bytes an LZMAError is raised



 Possible solution

A possible solution would be to add a finish method on the decompressor 
interface, and support it appropriately in _compression when we reached EOF on 
the input. Then, in LZMADecompressor implementation, use the LZMA_CONCATENATED 
flag, and implement the finish method to call lzma_code with LZMA_FINISH as 
action.

I think this would be preferred than trying to solve the issue in Python, 
because if the format is FORMAT_AUTO we don't know if the format is XZ (and we 
should support stream padding) or not.

--
components: Library (Lib)
files: example1.xz
messages: 393681
nosy: nadeem.vawda, rogdham
priority: normal
severity: normal
status: open
title: lzma: stream padding in xz files
type: behavior
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 
3.9
Added file: https://bugs.python.org/file50044/example1.xz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44123] make function parameter sentinel value true singletons

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Parameters are not passed to a function. Arguments are passed. And what you 
need is just to not pass to the wrapped function arguments which were not 
passed to wrapper. *args and **kwargs do not contain arguments which were not 
passed.

I understand that it would be pleasant to have better some singletons and 
sentinels (with better repr, and sometimes pickleable), but I don't think that 
the problem with pickling default values of function parameters even exists.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32133] documentation: numbers module nitpick

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Here's a minimal change:

- None of the types defined in this module can be instantiated.
+ None of the types defined in this module are intended to be instantiated.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44109] missing dataclass decorator in match-statement example

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> Thoughts from others?

As it stands, the Point class is weird and atypical by only using class 
variables.  The example would be improved by adding @dataclass or inheriting 
from typing.NamedTuple.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44133] "Py_FrozenMain" symbol is not exported

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +24769
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26130

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37891] Exceptions tutorial page does not mention raise from

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Thanks for looking. This issue does appear to be out of date.

--
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44133] "Py_FrozenMain" symbol is not exported

[STINNER Victor]

STINNER Victor  added the comment:

The "python" binary exports 1610 symbols. With the attached fix, it exports 
1611 symbols (+1): "Py_FrozenMain" is also exported :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44133] "Py_FrozenMain" symbol is not exported

[STINNER Victor]

New submission from STINNER Victor :

When Python is built without --enable-shared, the "Py_FrozenMain" is not 
exported.

When Python is built with --enable-shared, the "Py_FrozenMain" is exported as 
expected.

I can reproduce the issue with attached reproduce.tar.gz example:

* build.sh exports "func1" symbol
* build_ar.sh doesn't export the "func1" symbol

The difference is that build.sh links directly two object files (.o) into a 
binary, whereas build_ar.sh creates a static library (.a) and then link the 
static library into a binary.

Python is always built with a static library (libpythonVERSION.a) which causes 
the issue.

A solution is to pass -Wl,--whole-archive option to the linker to export *all* 
symbols exported by all object files, and not only export symbols of the object 
files which are used.

I'm not sure why, but "Py_FrozenMain" is the *only* impacted symbol of the 
whole C API.

This issue was discovered by Petr Viktorin who works on the stable ABI:
https://mail.python.org/archives/list/capi-...@python.org/thread/5QLI3NUP3OSGLCCIBAQOTX4GEJQBWJ6F/

--
components: C API
files: reproduce.tar.gz
messages: 393675
nosy: vstinner
priority: normal
severity: normal
status: open
title: "Py_FrozenMain" symbol is not exported
versions: Python 3.11
Added file: https://bugs.python.org/file50043/reproduce.tar.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44108] [sqlite3] normalise SQL quoted literals in sqlite3 test suite

[Dong-hee Na]

Change by Dong-hee Na :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44123] make function parameter sentinel value true singletons

[Tal Einat]

Tal Einat  added the comment:

> I do not understand the problem with pickling sentinel values used as default 
> values for function parameters. Could you please show an example?

A category of relevant uses I can think of is when wrapping a function and 
storing the parameters it was called with for some later use. Some examples of 
this are logging, caching and RPC.

Specifically for RPC, using pickle to serialize/deserialize the parameters and 
then call the function with them seems reasonable. RPyC[1] does this in some 
cases, though I haven't yet checked how it handles these kinds of sentinel 
objects specifically.

I'll try to find a good concrete example.

[1] https://rpyc.readthedocs.io/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32133] documentation: numbers module nitpick

[Bhaskara Aditya Sriram]

Bhaskara Aditya Sriram  added the comment:

I would like work on this, but I'm very new to fixing bugs. Could someone 
please help me on how to proceed with this bug. Thank You in advance

--
nosy: +adityasriram.b

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

[Stefan Behnel]

Change by Stefan Behnel :


--
Removed message: https://bugs.python.org/msg393667

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44126] Cross Compile Cython Modules

[Ned Deily]

Ned Deily  added the comment:

Ah, that makes more sense.

Alas, while there has been some support for various cross-compiling scenarios 
added to cPython itself, including targeting for Android, for the most part 
they are not well-documented if at all and are not regularly tested as part of 
our release process. Further, while Distutils has been a part of the cPython 
standard library for many years, it is now deprecated for building modules not 
in the standard library. The PyPA recommends using setuptools instead of 
Distutils, in general, and there are other build tools out there. How well each 
of them support cross-compiling would be a good thing to research and ask. 
https://packaging.python.org/key_projects/?highlight=setuptools#distutils

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44095] Add suffix property to zipfile.Path

[Miguel Brito]

Miguel Brito  added the comment:

I prototyped something by adding `suffix`, `suffixes`, and `stem`. To me these 
are the most obvious ones and would be useful right away.

Regarding the others I'm not so sure. It seems that zipfile.Path is just a 
convenient class to provide basic navigational operations. It's not meant to be 
100% consistent with pathlib.Path. 

In any case, it'd nice to discuss it.


Here it is: https://github.com/python/cpython/pull/26129

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Shreyan Avigyan]

Change by Shreyan Avigyan :


--
resolution:  -> wont fix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Shreyan Avigyan]

Shreyan Avigyan  added the comment:

I suspected right. This was just CPU instability. In fact the patch is 1x-2x 
slower as reported by pyperf. Therefore I'm closing this issue. 

Though I'm working on another "yet to be failed attempt" to remove or rewrite 
the GIL to improve performance. Of course, any help would be appreciated there.

Repo - https://github.com/shreyanavigyan/pygilremove

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44095] Add suffix property to zipfile.Path

[Miguel Brito]

Change by Miguel Brito :


--
keywords: +patch
pull_requests: +24768
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26129

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44132] Local import conflict with system import

[Oscar]

New submission from Oscar :

Not sure if this is the right place to post this, but I stumble upon the 
following error.

I have the following directory structure

$ tree /F
Folder PATH listing
Volume serial number is C100 B8C8:3DC4
C:.
│   requirements.txt
│   run_repro.bat
│
└───testdriver
│   jira_utils.py
│   main.py
│   __init__.py
│
└───xml
xml_base.py
__init__.py

and I am getting the following error when I run 

(venv) > C:\depot\bitbucket\python_bug
$ python testdriver\main.py
Traceback (most recent call last):
  File "testdriver\main.py", line 1, in 
from testdriver.jira_utils import JiraUtils
  File "C:\depot\bitbucket\python_bug\testdriver\jira_utils.py", line 1, in 

from jira import JIRA, JIRAError
  File "C:\depot\bitbucket\python_bug\venv\lib\site-packages\jira\__init__.py", 
line 10, in 
from jira.client import JIRA  # noqa: E402
  File "C:\depot\bitbucket\python_bug\venv\lib\site-packages\jira\client.py", 
line 29, in 
from pkg_resources import parse_version
  File 
"C:\depot\bitbucket\python_bug\venv\lib\site-packages\pkg_resources\__init__.py",
 line 32, in 
import plistlib
  File "C:\Python37\Lib\plistlib.py", line 65, in 
from xml.parsers.expat import ParserCreate
ModuleNotFoundError: No module named 'xml.parsers'

but if I run 
(venv) otrejoso@OTREJOSO-TLALOC C:\depot\bitbucket\python_bug
$ python -c "from testdriver.jira_utils import JiraUtils"

(venv) otrejoso@OTREJOSO-TLALOC C:\depot\bitbucket\python_bug

There is no error.

I wonder if this is some kind of bug with importing packages or is it a way to 
fix this by importing differently?

I have attached a zip file that should be able to repro the issue. It is pretty 
simple since it just imports packages.
There is a run_repro.bat script to make things easier to repro

Steps of bat script:
set PYTHONPATH=.
python testdriver\main.py
python -c "from testdriver.jira_utils import JiraUtils"

Note: the zip contains the minimal files that showed the issue on bigger scale 
project that we have.

Thanks in advance.

--
files: error.zip
messages: 393669
nosy: otrejoso
priority: normal
severity: normal
status: open
title: Local import conflict with system import
versions: Python 3.7
Added file: https://bugs.python.org/file50042/error.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42972] [C API] Heap types (PyType_FromSpec) must fully implement the GC protocol

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

Is there a deterministic way to test these changes? Will something a la this be 
sufficient:

import gc
import sys

gc.collect()
before = sys.gettotalrefcount()

import somemod
del sys.modules['somemod']
del somemod

gc.collect()
after = sys.gettotalrefcount()

assert after == before

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44108] [sqlite3] normalise SQL quoted literals in sqlite3 test suite

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
pull_requests: +24767
pull_request: https://github.com/python/cpython/pull/26128

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

[Stefan Behnel]

Stefan Behnel  added the comment:

Should we allow empty CFrame entries (with a NULL Frame pointer) that only use 
their "use_tracing" flag? That would make it easy for code to append and remove 
such an entry in order to temporarily disable tracing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

[Stefan Behnel]

Stefan Behnel  added the comment:

I just noticed that new C-API functions are probably useless for Cython since I 
think it will have to maintain the CFrame stack, so not to enable "use_tracing" 
for the (Python) caller but the current (Cython) function. This then means that 
we own The current CFrame as well as its "use_tracing" field and don't need any 
help from CPython in order to change the state.

I'm not sure if this is any different for other users of the "use_tracing" 
field.

--
type:  -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[STINNER Victor]

STINNER Victor  added the comment:

> Mean +- std dev: 455 ms +- 13 ms

A single benchmark is useless without a reference. Please run the benchmark on 
unpatched python, then run it again on a patched Python, and compare results. 
You can store benchmark results using -o option. Example:

# Unpatched python
python pyperf_benchmark.py -o ref.json
# Patched python
python pyperf_benchmark.py -o new_gil.json
# Compare
python -m pyperf compare_to ref.json new_gil.json

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Shreyan Avigyan]

Shreyan Avigyan  added the comment:

pyperf results -

Mean +- std dev: 455 ms +- 13 ms

Looks a little bit faster than current 3.11, 3.10 and 3.9 versions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[STINNER Victor]

STINNER Victor  added the comment:

> I attached the pyperf benchmark also.

Can you please provide pyperf_benchmark.py results?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44098] Remove ParamSpec from __parameters__ of most typing generics

[Ken Jin]

Ken Jin  added the comment:

All patches have landed. Thanks Guido!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42109] Use hypothesis for testing the standard library, falling back to stubs

[Paul Ganssle]

Paul Ganssle  added the comment:

@Terry

> The problem with random input tests in not that they are 'flakey', but that 
> they are useless unless someone is going to pay attention to failures and try 
> to find the cause.  This touches on the difference between regression testing 
> and bug-finding tests.  CPython CI is the former, and marred at that by buggy 
> randomly failing tests.

> My conclusion: bug testing would likely be a good idea, but should be done 
> separate from the CI test suite.  Such testing should only be done for 
> modules with an active maintainer who would welcome failure reports.

Are you saying that random input tests are flaky but that that is not the big 
problem? In my experience using hypothesis, in practice it is not the case that 
you get tests that fail randomly. The majority of the time if your code doesn't 
violate one of the properties, the tests fail the first time you run the test 
suite (this is particularly true for strategies where hypothesis deliberately 
makes it more likely that you'll get a "nasty" input by biasing the random 
selection algorithm in that direction). In a smaller number of cases, I see 
failures that happen on the second, third or fourth run.

That said, if it were a concern that every run of the tests is using different 
inputs (and thus you might see a bug that only appears once in every 20 runs), 
it is possible to run hypothesis in a configuration where you specify the seed, 
making it so that hypothesis always runs the same set of inputs for the same 
tests. We can disable that on a separate non-CI run for hypothesis "fuzzing" 
that would run the test suite for longer (or indefinitely) looking for 
long-tail violations of these properties.

I feel that if we don't at least run some form of the hypothesis tests in CI, 
there will likely be bit rot and the tests will decay in usefulness. Consider 
the case where someone accidentally breaks an edge case that makes it so that 
`json.loads(json.dumps(o))` no longer works for some obscure value of `o`. With 
hypothesis tests running in CI, we are MUCH more likely to find this bug / 
regression during the initial PR that would break the edge case than if we run 
it separately and report it later. If we run the hypothesis tests in a 
build-bot, the process would be:

1. Contributor makes PR with passing CI.
2. Core dev review passes, PR is merged.
3. Buildbot run occurs and the buildbot watch is notified.
4. Buildbot maintainers track down the PR responsible and either file a new bug 
or comment on the old bug.
5. Someone makes a NEW PR adding a regression test and the fix for the old PR.
6. Core dev review passes, second PR is merged.

If we run it in CI, the process would be:

1. Contributor makes PR, CI breaks.
2. If the contributor doesn't notice the broken CI, core dev points it out and 
it is fixed (or the PR is scrapped as unworkable).

Note that in the non-CI process, we need TWO core dev reviews, we need TWO PRs 
(people are not always super motivated to fix bugs that don't affect them that 
they the caused when fixing a bug that does affect them), and we need time and 
effort from the buildbot maintainers (note the same applies even if the 
"buildbot" is actually a separate process run by Zac out of a github repo).

Even if the bug only appears in one out of every 4 CI runs, it's highly likely 
that it will be found and fixed before it makes it into production, or at least 
much more quickly, considering that most PRs go through a few edit cycles, and 
a good fraction of them are backported to 2-3 branches, all with separate CI 
runs. It's a much quicker feedback loop.

I think there's an argument to be made that incorporating more third-party 
libraries (in general) into our CI build might cause headaches, but I think 
that is not a problem specific to hypothesis, and I think its one where we can 
find a reasonable balance that allows us to use hypothesis in one form or 
another in the standard library.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Shreyan Avigyan]

Shreyan Avigyan  added the comment:

I attached the pyperf benchmark also.

--
Added file: https://bugs.python.org/file50041/benchmark.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37891] Exceptions tutorial page does not mention raise from

[Bonifacio]

Bonifacio  added the comment:

Version 3.9 of the docs already cover the syntax Neils talks about in his 
original message. 
https://docs.python.org/3.9/tutorial/errors.html#exception-chaining
Is there anything left to be done here? If so, I'm glad to help with them, but 
I would need directions.

--
nosy: +Bonifacio2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Shreyan Avigyan]

Shreyan Avigyan  added the comment:

Sure

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Christian Heimes]

Christian Heimes  added the comment:

The benchmark code has multiple issues. For example it uses a bad clock 
sources, time.time(). Could you please rewrite the benchmark to use 
https://pypi.org/project/pyperf/ ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Shreyan Avigyan]

Shreyan Avigyan  added the comment:

> "That difference is 0.02 seconds, or about 1%, correct?"

I mistook 1.98 as 1.88 in my calculation. Sorry for that.

> The information is not very helpful to understand what you are doing. Please 
> provide your system specs (arch, platform, CPU spec, memory), the commands 
> you used to compile Python and the commands you used to get the numbers

My System is x86_64 Windows with 4 cores and 8 gigs of memory. I also tried it 
with my WSL1.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Shreyan Avigyan]

Shreyan Avigyan  added the comment:

Test suite passed and also I'm attaching the benchmark I used to measure. (This 
is the benchmark used by David Beazley in one of his famous GIL blog posts and 
talks.)

--
Added file: https://bugs.python.org/file50040/benchmark.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Eric V. Smith]

Eric V. Smith  added the comment:

"If it ran in 1.98 secs then the new patch would theoretically measure 
1.96 (original - 0.22)"

That difference is 0.02 seconds, or about 1%, correct?

I'm just trying to understand what we're measuring here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Christian Heimes]

Christian Heimes  added the comment:

The information is not very helpful to understand what you are doing. Please 
provide your system specs (arch, platform, CPU spec, memory), the commands you 
used to compile Python and the commands you used to get the numbers

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Shreyan Avigyan]

Shreyan Avigyan  added the comment:

I took the measurement like this. If it ran in 1.98 secs then the new patch 
would theoretically measure 1.96 (original - 0.22). The patch isn't very 
useful but it can sometimes be faster than the present code especially in debug 
mode.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Eric V. Smith]

Eric V. Smith  added the comment:

"there's an increase in speed by 0.22 or something like that"

0.22 what? Seconds? Percent? A factor of 0.22?

What are you measuring, and how are you measuring it?

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44095] Add suffix property to zipfile.Path

[Jakub Nowak]

Jakub Nowak  added the comment:

Thank for your interest Miguel. I have no experience with CPython codebase so 
I'll be glad if you work on this.

I think treating ZipFile's like a separate drive would be the best. If we agree 
on this then probably all of the properties and methods from pathlib could be 
implemented. But without this at least `suffix`, `suffixes`, `stem`, 
`with_name()`, `with_stem()`, and `with_suffix()` makes perfect sense.

If some of the properties/methods won't be implemented on zipfile.Path then it 
would be useful to raise `NotImplementedError`. Currently, trying to use suffix 
property, raises the error `AttributeError: 'Path' object has no attribute 
'suffix'`. This is confusing as the name of the class is the same as in 
`pathlib.Path` so it's not immediately obvious that it meant `zipfile.Path`.

The use-case for mimicking those two interfaces closely is to use both 
interchangeably (thanks to duck-typing).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Shreyan Avigyan]

Shreyan Avigyan  added the comment:

Forgot to do that. I'll report back with the test suite result.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44113] [C API] Deprecate legacy API for configure Python initialization

[STINNER Victor]

STINNER Victor  added the comment:

> Well, Py_FrozenMain() should be rewritten with PyConfig, I want to do that 
> for a long time, but I don't know how to test it. It would be nice to have a 
> very basic test for Py_FrozenMain().

I created bpo-44131: [C API] Add tests on Py_FrozenMain().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Christian Heimes]

Christian Heimes  added the comment:

Does Python's unit test suite pass?

--
nosy: +christian.heimes, pitrou, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44113] [C API] Deprecate legacy API for configure Python initialization

[STINNER Victor]

STINNER Victor  added the comment:

Thanks for the help Dong-hee ;-) I close the issue, functions are now 
deprecated.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24871] freeze.py doesn't work on x86_64 Linux out of the box

[STINNER Victor]

STINNER Victor  added the comment:

I consider that the issue is now fixed. If it's not the case, please reopen the 
issue (or open a new issue).

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Shreyan Avigyan]

Shreyan Avigyan  added the comment:

Instability is more likely. But I'll run the benchmark once more and see what 
is happening.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44131] [C API] Add tests on Py_FrozenMain()

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +24766
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26126

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44131] [C API] Add tests on Py_FrozenMain()

[STINNER Victor]

New submission from STINNER Victor :

I would like to change Py_FrozenMain() using the new PyConfig C API, but I 
would prefer to first write tests of its current behavior. Currently, 
Py_FrozenMain() has no test.

Py_FrozenMain() is used by Tools/freeze/freeze.py. Old freeze.py issues:

* bpo-42613: freeze.py doesn't support sys.platlibdir different than lib nor 
multiarch
* bpo-40954: freeze.py aborts on macOS
* bpo-32217: freeze.py fails to work.
* bpo-27727: Update Tools/freeze to use .vcxproj files 
* bpo-27566: Tools/freeze/winmakemakefile.py clean target should use 'del' 
instead of 'rm' 
* bpo-26271: freeze.py makefile uses the wrong flags variables
* bpo-25504: undefined name 'modules' in Tools/freeze/checkextensions_win32.py 
* bpo-24871: freeze.py doesn't work on x86_64 Linux out of the box
* bpo-23405: Tools/freeze "make" gets missing file error with unix shared 
builds 
* bpo-21502: freeze.py not working properly with OS X framework builds
* bpo-16047: Tools/freeze no longer works in Python 3
* bpo-11824: freeze.py broken due to ABI flags
* bpo-7517: freeze.py not ported to python3
* bpo-6078: freeze.py doesn't work 
* bpo-1985: Bug/Patch: Problem with xml/__init__.py when using freeze.py 
* bpo-588452: $DEBUG_PYTHON -O freeze.py broken 

I never used freeze.py, but since we get bug reports, it seems like it's used 
by users :-)

Note: freeze.py has no test neither.


Py_FrozenMain() has been modified recently in bpo-44113:

commit 7565586724692e2ad164d770af9675f7a261fe3a
Author: Dong-hee Na 
Date:   Thu May 13 10:19:46 2021 +0900

bpo-44113: Update fromzenmain not to use Py_SetProgramName (GH-26085)

See also: [capi-sig] Py_FrozenMain and the stable ABI
https://mail.python.org/archives/list/capi-...@python.org/thread/5QLI3NUP3OSGLCCIBAQOTX4GEJQBWJ6F/

See also: [Python-Dev] What's the story on Py_FrozenMain?
https://mail.python.org/pipermail/python-dev/2013-November/130280.html

--
components: C API
messages: 393643
nosy: corona10, petr.viktorin, vstinner
priority: normal
severity: normal
status: open
title: [C API] Add tests on Py_FrozenMain()
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44130] GIL Improvement

[Shreyan Avigyan]

New submission from Shreyan Avigyan :

Today while working on an attempt to improve the GIL (either by modifying it or 
removing it) I noticed that in destroy_gil function first we unlock the mutex 
and then we set the gil->locked to -1 using _Py_atomic_store_explicit. 
Therefore the cycle is, "Unlock -> Atomic_Set_Value" which closely evaluates 
to, "Unlock -> Lock -> Set_Value -> Unlock"


I tweaked around a little and when I changed the cycle to, "Atomic_Set_Value -> 
Unlock" I noticed by running David Beazley's famous benchmarks that there's an 
increase in speed by 0.22 or something like that. I don't know if it's because 
of CPU instability or because of the tweak but it looks like an improvement. 
Should this patch be applied to cpython?

--
components: C API
messages: 393642
nosy: shreyanavigyan
priority: normal
severity: normal
status: open
title: GIL Improvement
type: performance
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44113] [C API] Deprecate legacy API for configure Python initialization

[STINNER Victor]

STINNER Victor  added the comment:

commit 7565586724692e2ad164d770af9675f7a261fe3a
Author: Dong-hee Na 
Date:   Thu May 13 10:19:46 2021 +0900

bpo-44113: Update fromzenmain not to use Py_SetProgramName (GH-26085)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] [C API] Convert a few stdlib extensions to the limited C API (PEP 384)

[hai shi]

Change by hai shi :


--
versions: +Python 3.11 -Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] [C API] Convert a few stdlib extensions to the limited C API (PEP 384)

[hai shi]

hai shi  added the comment:

> METH_VARARGS and METH_KEYWORDS are part of the stable ABI.
  Copy that, thanks, petr.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1662] [patch] assert tp_traverse in PyType_GenericAlloc()

[Irit Katriel]

Irit Katriel  added the comment:

tp_traverse is optional, so we should not add this assertion.

--
resolution:  -> rejected
stage: test needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44108] [sqlite3] normalise SQL quoted literals in sqlite3 test suite

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset f28bac428d5fff23fe5e95aa02b903ee2dd5b3de by Miss Islington (bot) 
in branch '3.10':
bpo-44108: sqlite3 test suite now works with SQLITE_DQS=0 (GH-26032) (GH-26125)
https://github.com/python/cpython/commit/f28bac428d5fff23fe5e95aa02b903ee2dd5b3de


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44108] [sqlite3] normalise SQL quoted literals in sqlite3 test suite

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 2.0 -> 3.0
pull_requests: +24765
pull_request: https://github.com/python/cpython/pull/26125

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44095] Add suffix property to zipfile.Path

[Miguel Brito]

Miguel Brito  added the comment:

+1. 

The docstring says:
```
class Path:
"""
A pathlib-compatible interface for zip files.
```
but only a few methods are supported. It'd be nice to have at least `stem`, 
`parents`, `suffixes`, `parts`, which IMHO would make sense here.

I'd love to work on this if the Core Devs think it makes sense and if OP is not 
doing so also.

--
nosy: +miguendes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32133] documentation: numbers module nitpick

[Miguel Brito]

Miguel Brito  added the comment:

Seen that nobody is working on this I transformed the suggestion into a PR.

--
nosy: +miguendes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43795] Implement PEP 652 -- Maintaining the Stable ABI

[miss-islington]

Change by miss-islington :


--
pull_requests: +24764
pull_request: https://github.com/python/cpython/pull/26123

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] [C API] Convert a few stdlib extensions to the limited C API (PEP 384)

[Petr Viktorin]

Petr Viktorin  added the comment:

METH_VARARGS and METH_KEYWORDS are part of the stable ABI.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44123] make function parameter sentinel value true singletons

[STINNER Victor]

STINNER Victor  added the comment:

Tal: "This is also being discussed on python-dev."

https://mail.python.org/archives/list/python-...@python.org/thread/ZLVPD2OISI7M4POMTR2FCQTE6TPMPTO3/

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42109] Use hypothesis for testing the standard library, falling back to stubs

[Terry J. Reedy]

Terry J. Reedy  added the comment:

This issue confuses two different testing issues.  First is selection of test 
data before the test versus generation while testing (usually 'randomly')  
Second is how the function is tested.  All tests test some property of the 
inputs and function, but it is useful to distinguish between testing that a 
function produces a specific expected output versus testing something more 
abstract or defined in terms of the inputs.

Example 1: 2 + 3 = 5 (a + b = (known) c) versus 2 + 3 = 3 + 2 (a + b = b + a) 
and (2 + 3) + 4 = 2 + (3 + 4).

Example 2: sorted([3,5,1,4]) = [1,3,4,5] versus is_sorted(sorted([random 
list]).  

The latter are what hypothesis people mean by 'property tests'.  Such property 
tests can be done with either fixed or random pairs in unittest module tests.  
The test that a module compiles is a weak property test, but better than 
nothing.  A test that that all lines of code runs at least once without an 
unexpected exception a stronger property test.  I consider it roughly on a par 
with min(s) <= mean(s) <= max(s).

The connection between randomized input testing, such as with hypothesis, and 
property tests is that with random inputs, one cannot test f(input) against an 
'expected' value unless one has an alternate means of producing the expected 
value.  It is often easier to test a parameterized equation or other properties.

The problem with random input tests in not that they are 'flakey', but that 
they are useless unless someone is going to pay attention to failures and try 
to find the cause.  This touches on the difference between regression testing 
and bug-finding tests.  CPython CI is the former, and marred at that by buggy 
randomly failing tests.

My conclusion: bug testing would likely be a good idea, but should be done 
separate from the CI test suite.  Such testing should only be done for modules 
with an active maintainer who would welcome failure reports.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44123] make function parameter sentinel value true singletons

[Tal Einat]

Tal Einat  added the comment:

> This needs a good deal more discussion before sweeping through the code and 
> change a long standing Python idiom that has stood the test of time.

I agree this should be discussed, which is why I created this issue about it, 
as place for that discussion to take place. This is also being discussed on 
python-dev.

The examples I've mentioned are both rather new, and they have implemented 
sentinel values using three different methods, with those in dataclasses even 
being inconsistent among themselves. I fail to see how this can be considered 
"chang[ing] a long standing Python idiom that has stood the test of time".

> You "recently noticing this" doesn't mean that it is wrong.

I find this wording makes things unnecessarily personal and negative. Let's 
discuss things based on technical merit.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26032] Use scandir() to speed up pathlib globbing

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28146] Confusing error messages in str.format()

[STINNER Victor]

STINNER Victor  added the comment:

commit 4aeee0b47b3a2b604bbac37040320ffc88c291f2
Author: Irit Katriel 
Date:   Thu May 13 21:55:55 2021 +0100

bpo-28146: Fix a confusing error message in str.format() (GH-24213)



Automerge-Triggered-By: GH:pitrou

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44117] [C API] Remove deprecated PyEval_InitThreads()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Should not they be kept for binary compatibility, so you can link with 
extensions compiled for older Python versions? They can be removed from 
headers, so newly compiled extensions will not be able to use them.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com