[issue23316] Incorrect evaluation order of function arguments with *args

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy:  -serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44439] PickleBuffer doesn't have __len__ method

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Oh, LZMAFile.write() should not use len() directly on input data because it 
does not always work correctly with memoryview and other objects supporting the 
buffer protocol. It should use memoryview(data).nbytes or data = 
memoryview(data).cast('B') if other byte-oriented operations (indexing, 
slicing) are used. See for example Lib/gzip.py, Lib/_pyio.py, 
Lib/_compression.py, Lib/ssl.py, Lib/socketserver.py, Lib/wave.py.

--
nosy: +nadeem.vawda, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
versions: +Python 3.11 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Would not be be better in long term to get rid of irregularities? It would make 
the grammar simpler and the documentation clearer.

The only use case of such syntax in wrappers, but they always can be rewritten 
in other style, with natural order of arguments evaluation.

def wrapper(*args, **kw):
return wrapped_fn(*args, some_arg=1, **kw)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44439] PickleBuffer doesn't have __len__ method

[Ma Lin]

New submission from Ma Lin :

If run this code, it will raise an exception: 

import pickle
import lzma
import pandas as pd
with lzma.open("test.xz", "wb") as file:
pickle.dump(pd.DataFrame(range(1_000_000)), file, protocol=5)

The exception:

Traceback (most recent call last):
  File "E:\testlen.py", line 7, in 
pickle.dump(pd.DataFrame(range(1_000_000)), file, protocol=5)
  File "D:\Python39\lib\lzma.py", line 234, in write
self._pos += len(data)
TypeError: object of type 'pickle.PickleBuffer' has no len()

The exception is raised in lzma.LZMAFile.write() method:
https://github.com/python/cpython/blob/v3.10.0b2/Lib/lzma.py#L238

PickleBuffer doesn't have .__len__ method, is it intended?

--
messages: 395971
nosy: malin, pitrou
priority: normal
severity: normal
status: open
title: PickleBuffer doesn't have __len__ method

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Neil Girdhar]

Neil Girdhar  added the comment:

FYI: https://github.com/PyCQA/pylint/issues/4586

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44318] Asyncio classes missing __slots__

[Andrei Kulakov]

Andrei Kulakov  added the comment:

Bluenix, can you describe your use case in more detail?

--
nosy: +andrei.avk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Guido van Rossum]

Guido van Rossum  added the comment:

Well, it seems we're stuck -- we can't make the grammar more restrictive (at 
least, I don't think we should, since it is a backwards incompatibility), so 
we'll have to live with the exception. Since it is now clear what the rule is, 
we can update the docs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44431] Add command-line functionality to uuid module

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
nosy: +erlendaasland

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44431] Add command-line functionality to uuid module

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I think it would be a useful feature.

But would not be better to make interface similar to interface of standard 
Linux command uuidgen? In any case it should support the -h or --help options.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44438] argparser documentation error

[Raymond Hettinger]

Raymond Hettinger  added the comment:

If you read the text afterwards, you'll see that raising a TypeError was the 
intended purpose of this example.  Just afterward, it shows how to use *type* 
to make the code correct.

--
assignee:  -> rhettinger
nosy: +rhettinger
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

See also thread "Order of positional and keyword arguments" on the Python-Dev 
mailing list 
(https://mail.python.org/archives/list/python-...@python.org/thread/I6AUYV6DVEMP7XVYVDWC62N6PK6FHX3H/).

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23394] No garbage collection at end of main thread

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44438] argparser documentation error

[Arman Sargsyan]

Arman Sargsyan  added the comment:

I have signed the CLA

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44438] argparser documentation error

[Arman Sargsyan]

New submission from Arman Sargsyan :

URL - https://docs.python.org/3/howto/argparse.html

The following code will return a type error:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument("square", help="display a square of a given number")
args = parser.parse_args()
print(args.square**2)

This should be changed to:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument("square", help="display a square of a given number")
args = parser.parse_args()
print(int(args.square)**2)

--
components: Parser
messages: 395963
nosy: arsar7, lys.nikolaou, pablogsal
priority: normal
pull_requests: 25349
severity: normal
status: open
title: argparser documentation error
type: resource usage
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 
3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The following does not consider keyword-only args with or without defaults.  My 
understanding is the the compiler does not know or considered the signature of 
the function when evaluating the args.

from dis import dis
from itertools import permutations as pm  # My first use of this.

for a1, a2, a3, a4 in pm(('a', 'b=b', '*g', '**k')):
s = f"e({a1}, {a2}, {a3}, {a4})"
print(s)
try:
cd = compile(s, '', 'eval')
except SyntaxError as e:
print(e)
continue
dis(cd)

Since positional arg cannot follow keyword arg or keyword arg unpacking and 
positional arg unpacking cannot follow keyword arg unpacking, only 5 orders of 
'a', 'b=b', '*g', and '**k' are legal.

e(a, b=b, *g, **k)  g before b
e(a, *g, b=b, **k)  normal
e(a, *g, **k, b=b)  normal
e(*g, a, b=b, **k)  normal
e(*g, a, **k, b=b)  normal

For whatever reason, the byte code is more complicated without function 
arguments.

e(a, b=b, *g, **k)
  1   0 LOAD_NAME0 (e)
  2 LOAD_NAME1 (a)
  4 BUILD_LIST   1
  6 LOAD_NAME2 (g)
  8 LIST_EXTEND  1
 10 LIST_TO_TUPLE
 12 LOAD_CONST   0 ('b')
 14 LOAD_NAME3 (b)
 16 BUILD_MAP1
 18 LOAD_NAME4 (k)
 20 DICT_MERGE   1
 22 CALL_FUNCTION_EX 1
 24 RETURN_VALUE

The exceptional case essentially says that positional arg unpacking cannot 
follow keyword args.  This is consistent with the other 3 prohibitions.  
Arguments passed by position form one group, those passed by name another.  The 
nice, simple, and consistent rule is that positional args (and unpacking 
thereof) cannot follow keyword args (and unpacking there).

But in this one case, instead of raising SyntaxError like the other 3 
prohibited orders, the compiler in effect rewrites* the code in a legal order 
-- and then compiles as if written that way.  How often does it rewrite code to 
make it correct?

I think it would have been better to have stuck with 'positional before 
keyword'.  Can we consider deprecating something rare and easy to get wrong?

Thinking more, I might prefer leaving the Evaluation Order section alone, as it 
is correct with respect to the internal rewrite, and revise the explanation of 
the quirk in the function doc. We could deprecate the mis-ordering at least in 
the doc.  

* After writing this, I notice Neil's suggestion that PEP 8 'prohibit' named 
params before iterable (positional) unpacking, so that a fixer program could 
literally rewrite in the correct order.  I am suggesting that it is useful to 
take the viewpoint that the compiler is doing this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44437] Add multimap() function similar to map(), but with multiprocessing functionality to the multiprocessing module

[Raymond Hettinger]

New submission from Raymond Hettinger :

Marking as closed for the reasons mentioned in the PR.  I you want to go 
forward, consider starting a thread on python-ideas.

--
nosy: +rhettinger
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31772] SourceLoader uses stale bytecode in case of equal mtime seconds

[Irit Katriel]

Irit Katriel  added the comment:

Looks like this was merged here: 
https://github.com/akvadrako/cpython/commit/fe9b43a0928a5ef80a4daf3a50af300e5eaeda9a

Can we close?

--
nosy: +iritkatriel
resolution:  -> fixed
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43259] argparse: allow add_mutually_exclusive_group on add_argument_group

[Francis Charette Migneault]

Francis Charette Migneault  added the 
comment:

I have found that the only thing argparse is actually missing is to forward the 
actions to the generated mutually exclusive group in _add_container_actions 
method.  


class SubArgumentParserFixedMutExtGroups(argparse.ArgumentParser):
def _add_container_actions(self, container):
groups = container._mutually_exclusive_groups
container._mutually_exclusive_groups = []  # temporary override 
just so it is not processed
super(SubArgumentParserFixedMutexGroups, 
self)._add_container_actions(container)
# same as original loop, but with extra append of actions in 
created mutex
for group in groups:
mutex_group = 
self.add_mutually_exclusive_group(required=group.required)
for action in group._group_actions:
mutex_group._group_actions.append(action)


When printing help, this resolves correctly. 
The actions can be found because they are already added when parsing the group 
of the parent parser that contains the mutex.
Snippet below. 

# same as other comment examples
parser = argparse.ArgumentParser()
parser_group = parser.add_argument_group("INPUT OPTIONS")
parser_group_mutually_exclusive = 
parser_group.add_mutually_exclusive_group(required=False)
parser_group_mutually_exclusive.add_argument("--from-args")
parser_group_mutually_exclusive.add_argument("--from-files")
parser_group_mutually_exclusive.add_argument("--from-stdin")
parser_group.add_argument("-0", help="null delimited pathnames")
parser.print_help()

usage: pydevconsole.py [-h]
   [--from-args FROM_ARGS | --from-files FROM_FILES | 
--from-stdin FROM_STDIN]
   [-0 0]
optional arguments:
  -h, --helpshow this help message and exit
INPUT OPTIONS:
  --from-args FROM_ARGS
  --from-files FROM_FILES
  --from-stdin FROM_STDIN
  -0 0  null delimited pathnames

# now add the subparser with tweaked ArgumentParser
parent = SubArgumentParserFixedMutexGroups()
subpar = parent.add_subparsers()
subpar.add_parser("test", add_help=False, parents=[parser])
parent.parse_args(["test", "--help"])

usage: pydevconsole.py test [-h]
[--from-args FROM_ARGS | --from-files 
FROM_FILES | --from-stdin FROM_STDIN]
[-0 0]
optional arguments:
  -h, --helpshow this help message and exit
INPUT OPTIONS:
  --from-args FROM_ARGS
  --from-files FROM_FILES
  --from-stdin FROM_STDIN
  -0 0  null delimited pathnames

--
nosy: +fmigneault

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23786] test_unaligned_buffers (test.test_hash.HashEqualityTestCase) ... Fatal Python error: Bus error

[Peter]

Peter  added the comment:

We've migrated our python process off Solaris.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44437] Add multimap() function similar to map(), but with multiprocessing functionality to the multiprocessing module

[Ryan Rudes]

Change by Ryan Rudes :


--
components: +Library (Lib) -Tkinter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44437] Add multimap() function similar to map(), but with multiprocessing functionality to the multiprocessing module

[Ryan Rudes]

Change by Ryan Rudes :


--
keywords: +patch
pull_requests: +25347
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26762

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44437] Add multimap() function similar to map(), but with multiprocessing functionality to the multiprocessing module

[Ryan Rudes]

Change by Ryan Rudes :


--
components: Tkinter
nosy: Ryan-Rudes
priority: normal
severity: normal
status: open
title: Add multimap() function similar to map(), but with multiprocessing 
functionality to the multiprocessing module
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44426] Docs fail to build with Sphinx 4 due to Invalid C declaration

[Mark Dickinson]

Mark Dickinson  added the comment:

> and we can certainly change it

Done. Closing here, but I've opened a Sphinx issue at 
https://github.com/sphinx-doc/sphinx/issues/9354

--
nosy:  -miss-islington
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44426] Docs fail to build with Sphinx 4 due to Invalid C declaration

[Mark Dickinson]

Mark Dickinson  added the comment:


New changeset c689e0a7e2a25621da82f22cc64d089eae05e753 by Miss Islington (bot) 
in branch '3.10':
bpo-44426: Use of 'complex' as a C variable name confuses Sphinx; change it to 
'num'. (GH-26744) (GH-26760)
https://github.com/python/cpython/commit/c689e0a7e2a25621da82f22cc64d089eae05e753


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44426] Docs fail to build with Sphinx 4 due to Invalid C declaration

[Mark Dickinson]

Mark Dickinson  added the comment:


New changeset 686c6f303a6e9e54b50401be0ae3dc6aa2fcf05a by Miss Islington (bot) 
in branch '3.9':
bpo-44426: Use of 'complex' as a C variable name confuses Sphinx; change it to 
'num'. (GH-26744) (GH-26761)
https://github.com/python/cpython/commit/686c6f303a6e9e54b50401be0ae3dc6aa2fcf05a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23786] test_unaligned_buffers (test.test_hash.HashEqualityTestCase) ... Fatal Python error: Bus error

[Irit Katriel]

Irit Katriel  added the comment:

Can this be close (as third party?) or is there anything left to do?

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Guido van Rossum]

Guido van Rossum  added the comment:

> "The one exception is in function calls with *expression after a keyword 
> argument: f(x=expr2, *expr1)."

So the question before us is whether to add this phrase to the docs, or to 
tweak the compiler to fix it. It is certainly convenient to update the docs 
rather than expend the resources to change the compiler for what looks like a 
rare corner case.

How certain are we that this is truly the only exception? Can someone read the 
compiler source code related to argument order, or experiment with all the 
different permutations of positional args, keyword args, *args, and **kwargs? 
(Fortunately the evaluation order is independent from the function definition, 
so it's really just all permutations of those four things.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44426] Docs fail to build with Sphinx 4 due to Invalid C declaration

[miss-islington]

Change by miss-islington :


--
pull_requests: +25346
pull_request: https://github.com/python/cpython/pull/26761

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44426] Docs fail to build with Sphinx 4 due to Invalid C declaration

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +25345
pull_request: https://github.com/python/cpython/pull/26760

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44426] Docs fail to build with Sphinx 4 due to Invalid C declaration

[Mark Dickinson]

Mark Dickinson  added the comment:


New changeset 7247f6f433846c6e37308a550e8e5eb6be379856 by Mark Dickinson in 
branch 'main':
bpo-44426: Use of 'complex' as a C variable name confuses Sphinx; change it to 
'num'. (GH-26744)
https://github.com/python/cpython/commit/7247f6f433846c6e37308a550e8e5eb6be379856


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The particular example of left-to-right function evaluation in
https://docs.python.org/3/reference/expressions.html#evaluation-order
is "expr1(expr2, expr3, *expr4, **expr5)".

Joshua's claim, without evidence, that "'f(*a(), b=b())' will evaluate b() 
before a()" was false, as shown by Neil with dis and is now, as shown by Irit 
with a code example.

But Neil also showed, again with dis, that a revised discrepancy claim, 
""'f(b=b(), *a())' will evaluate a() before b()", is true, because reversing 
the order of these particular arguments does not reverse the evaluation order.  
The dis today is equivalent to the one 7 years ago, with CALL_FUNCTION_VAR 
expanded to two operations.

dis.dis('f(b=b(), *a())')
  1   0 LOAD_NAME0 (f)
  2 LOAD_NAME1 (a)
  4 CALL_FUNCTION0
  6 LOAD_CONST   0 ('b')
  8 LOAD_NAME2 (b)
 10 CALL_FUNCTION0
 12 BUILD_MAP1
 14 CALL_FUNCTION_EX 1
 16 RETURN_VALUE

Irit's example, carried one step further confirms this.

>>> f(b=b(), *a())
a
b
f

Although I considered SilentGhost's reading of
https://docs.python.org/3/reference/expressions.html#calls
as too 'loose' (evaluation order of function argument *is* defined in general), 
that section address this exact case.

"A consequence of this is that although the *expression syntax may appear after 
explicit keyword arguments, it is processed before the keyword arguments ..."

I read this as an acknowledgement that this order violates the general rule.

I do wonder whether exception is specific to using a stack machine and 
implementation convenience or whether *all* implementations must follow this 
order.  If C-Python specific, it should be labelled as such.

The doc continues with an example call, for a different 'f', where, unlike 
Irit's 'f', the result is "TypeError: f() got multiple values for keyword 
argument 'a'".  It concludes "It is unusual for both keyword arguments and the 
*expression syntax to be used in the same call, so in practice this confusion 
does not arise."

The problem with mixing unpacking and named values might be even clearer if 
"f(b=1, *(2,1))" were added to the example box.  Perhaps *exp after x=exp 
should be prohibited.

If the exception is not removed from the implementation, then perhaps it should 
be added to the evaluation order section.  Something like

"The one exception is in function calls with *expression after a keyword 
argument: f(x=expr2, *expr1)."

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44433] processes created by subprocess.Popen is not terminating

[Jack DeVries]

Jack DeVries  added the comment:

For future reference, here are some good guidelines for submitting bug reports 
and asking for help to debug your code:

https://stackoverflow.com/help/minimal-reproducible-example

http://www.sscce.org/

--
nosy: +jack__d
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44411] regrtests fail with segfault after failing calls to malloc

[Jack DeVries]

Jack DeVries  added the comment:

I'm closing this because I have the sense that this is a problem with my 
system, but I'm continuing to investigate.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44411] regrtests fail with segfault after failing calls to malloc

[Jack DeVries]

Change by Jack DeVries :


--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44436] [Windows] _thread.start_new_thread() should close the thread handle

[STINNER Victor]

STINNER Victor  added the comment:

Eryk:
> We already close the handle in PyThread_start_new_thread() in 
> Python/thread_nt.h

Oh right! I completely missed this call. Thanks for double checking the issue 
;-) I close the issue as "not a bug".

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44436] [Windows] _thread.start_new_thread() should close the thread handle

[STINNER Victor]

STINNER Victor  added the comment:

> [Windows] _thread.start_new_thread() should close the thread handle

So far, I'm not convinced that Python must be changed.

I modified my Python locally so thread_run() writes GetCurrentThread() to 
stderr.

Example:
---
SLEEP = 5

import ctypes
import _thread
import time
import os

HANDLE = ctypes.c_voidp
DWORD = ctypes.c_ulong

ResumeThread = ctypes.windll.kernel32.ResumeThread
ResumeThread.argtypes = (HANDLE,)
ResumeThread.restype = DWORD

SuspendThread = ctypes.windll.kernel32.SuspendThread
SuspendThread.argtypes = (HANDLE,)
SuspendThread.restype = DWORD

def func():
time.sleep(SLEEP)
print("--thread exit--")

def check_handle(handle):
x = SuspendThread(handle)
print(f"SuspendThread({handle}) -> {x}")
x = ResumeThread(handle)
print(f"ResumeThread({handle}) -> {x}")

handle = _thread.start_new_thread(func, ())
print(f"start_new_thread() -> {handle}")
check_handle(handle)
time.sleep(SLEEP+1)
check_handle(handle)
---


Output:
---
start_new_thread() -> 2436
SuspendThread(2436) -> 4294967295
ResumeThread(2436) -> 4294967295
thread_run: GetCurrentThread() -> -2
--thread exit--
SuspendThread(2436) -> 4294967295
ResumeThread(2436) -> 4294967295
---

It seems like the handle returned by _beginthreadex() cannot be used with 
SuspendThread() or ResumeThread(): both fail. GetHandleInformation() also fails 
on this handle (I tried os.get_handle_inheritable()).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44433] processes created by subprocess.Popen is not terminating

[Eric V. Smith]

Eric V. Smith  added the comment:

This is probably not a python bug. But if you can demonstrate it without 
needing to install the Oracle software, we can take a look at it. You've also 
not explained how you're using multiprocessing. We need an complete, short, 
working example program to determine if it's a python bug.

I suggest you ask this question on Stack Overflow.

Also, I'd suggest subprocess.run instead of .communicate, if you can.

--
nosy: +eric.smith
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44436] [Windows] _thread.start_new_thread() should close the thread handle

[STINNER Victor]

STINNER Victor  added the comment:

> CloseHandle(GetCurrentThread())

This is useless. GetCurrentThread() returns a pseudo handle. The 
GetCurrentThread() documentation says:

"The pseudo handle need not be closed when it is no longer needed. Calling the 
CloseHandle function with this handle has no effect."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44436] [Windows] _thread.start_new_thread() should close the thread handle

[Eryk Sun]

Eryk Sun  added the comment:

> Would it be safe to close the handle just after PyThread_start_new_thread() 
> success?

We already close the handle in PyThread_start_new_thread() in 
Python/thread_nt.h:

if (hThread == 0) {
// ...
}
else {
dprintf(("%lu: PyThread_start_new_thread succeeded: %p\n",
 PyThread_get_thread_ident(), (void*)hThread));
CloseHandle(hThread);
}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44434] _thread module: Remove redundant PyThread_exit_thread() call to avoid glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
nosy: +erlendaasland

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44430] [sqlite3] refactor threading tests

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

The current code duplication has several negative impacts, IMO:

- Readability
  Currently the ThreadTests class spans 173 lines. Refactored, it spans 51 
lines (should fit in a screenful), making it easy to see what's actually being 
tested.
  Currently, the tests differ by only one line; the remaining 15 lines are 
boilerplate code. That's 15 lines of boilerplate code per test method.

- Maintainability
  Currently, if the boilerplate code needs tuning (or bugfixing), we must 
adjust it in _all_ the test methods. For example, adding the 
@threading_helper.reap_threads decorator to every single test. After 
refactoring, the boilerplate code is refactored out; adding the reap threads 
decorator needs only be done in one place; modifying the boilerplate code needs 
only be done in one place.

- Fewer lines of code; improved maintainability
  diff stat:
1 file changed, 37 insertions(+), 140 deletions(-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16437] issubclass doc improvement

[Ken Jin]

Ken Jin  added the comment:

@irit, you may be interested in issue44135, there's an open PR there and it 
seems to fix what this issue describes.

--
nosy: +kj

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44434] _thread module: Remove redundant PyThread_exit_thread() call to avoid glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-44436 "[Windows] _thread.start_new_thread() should close the 
thread handle".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44392] Py_GenericAlias is not documented

[Ken Jin]

Ken Jin  added the comment:

I'm closing this issue as all the relevant PRs have landed. Thanks everyone!

@Ronald
> Why should this be deprecated at all?

I can't speak for others, but personally I think it's here to stay (for the 
many reasons raised in the previous messages).

Thanks for catching and reporting this. Improving docs is always fun :).

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44436] [Windows] _thread.start_new_thread() should close the thread handle

[STINNER Victor]

New submission from STINNER Victor :

_thread.start_new_thread() is implemented by calling _beginthreadex(). 

Currently, when the thread completes: PyThread_exit_thread() is called which 
calls "_endthreadex(0)" on Windows. I proposed to no longer call it explicitly 
in bpo-44434.

_endthreadex() documentation says that the thread handle must be closed 
explicitly (with CloseHandle()), same applies for ExitThread().

"Unlike _endthread, the _endthreadex function does not close the thread handle, 
thereby allowing other threads to block on this one without fear that the 
handle will be freed out from under the system."

_endthread() closes the thread handle, but Python uses _endthreadex().

Should Python be modified to close explicitly the thread handle once the thread 
terminated?

_endthread() and _endthreadex() documentation:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/endthread-endthreadex?view=msvc-160

Example closing the thread handle in the thread which created the thread:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/beginthread-beginthreadex?view=msvc-160

Simplified code:
---
hThread = (HANDLE)_beginthreadex( NULL, 0, , NULL, 0, 
 );
WaitForSingleObject( hThread, INFINITE );
CloseHandle( hThread );
---


Would it be safe to close the handle just after PyThread_start_new_thread() 
success?

Or should it be done in the C function thread_run(), just before existing, 
CloseHandle(GetCurrentThread())?


To debug this issue, GetHandleInformation() can be used to check if the handle 
is still open or not. For example, call os.get_handle_inheritable(handle) in 
Python.

--
components: Windows
messages: 395939
nosy: eryksun, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: [Windows] _thread.start_new_thread() should close the thread handle
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20115] NUL bytes in commented lines

[Guido van Rossum]

Guido van Rossum  added the comment:

Yeah, null bytes should just be rejected. If someone comes up with a fix for 
this we'll accept it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44434] _thread module: Remove redundant PyThread_exit_thread() call to avoid glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work

[STINNER Victor]

STINNER Victor  added the comment:

Unix pthread_create() manual page.
https://man7.org/linux/man-pages/man3/pthread_create.3.html

The new thread terminates in one of the following ways:

(...)
* It  returns  from start_routine().  This is equivalent to calling 
pthread_exit(3) with the value supplied in
 the return statement.
(...)

Calling pthread_exit(0) is optional.

--


MSDN _beginthreadex() documentation:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/beginthread-beginthreadex?view=msvc-160

"When the thread returns from that routine, it is terminated automatically."

Calling _endthreadex(0) is optional.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44392] Py_GenericAlias is not documented

[miss-islington]

miss-islington  added the comment:


New changeset c7e95715ec2f2a16eace7aa35a1eb2f18e8d06ed by Ken Jin in branch 
'3.9':
[3.9] bpo-44392: Add Py_GenericAlias to C API docs (GH-26724) (GH-26757)
https://github.com/python/cpython/commit/c7e95715ec2f2a16eace7aa35a1eb2f18e8d06ed


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44435] There is no description of PY_SSIZE_T_CLEAN in docs

[Jack DeVries]

Change by Jack DeVries :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44435] There is no description of PY_SSIZE_T_CLEAN in docs

[Jack DeVries]

New submission from Jack DeVries :

In the intro to the C API 
(https://docs.python.org/3/c-api/intro.html#include-files), it says, "see 
Parsing arguments and building values for a description of this macro." There 
is a link which leads to ``arg.rst``, but there is no description of the macro 
there, just a note to include the macro like elsewhere.

I propose to remove this sentence from the docs, since it is my understanding 
that there is no need to document the details of why this macro must be 
defined, only to ensure that users define it.

Alternatively, add a description of the macro in the appropriate place in the C 
API docs, and fix the link in this blurb.

I'm happy to submit a documentation patch, but I'd need someone to advise on 
which of these options are more desired.

--
messages: 395935
nosy: jack__d
priority: normal
severity: normal
status: open
title: There is no description of PY_SSIZE_T_CLEAN in docs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44429] Tkinter Flow Geometry Manager

[Gary Davenport]

Gary Davenport  added the comment:

Thank you so much for the kind words and help in giving me some direction with 
these projects.  I am relatively new to Python, object oriented programming and 
open source collaboration.

I think the 3rd party module probably makes the most sense, because this is how 
I am currently using it.  The beginning of my code typically looks like this:

from tkinter import *
from tkinterFlow import *

But I will have to look at how to make or publish a module and so forth.  Then 
I would maintain the module with new releases.

I am also going to study Tk more.  I am fairly certain I can add the flow 
manager at that level.  Really the flow geometry manager as I have made isn't 
really a manager on its own.  It uses the grid managers methods.  (it could use 
the place instead of grid, but I chose grid).  

The main question I suppose would be does the community want a .flow method 
added to the Widgets in Tk or a 3rd party module?  I think I could make either 
one, given some time and study.

For now I will look into making the 3rd party module.  This is a skill I need 
to develop notwithstanding this flow manager project.

My main goal is just to make it easy for someone to implement flow behaviour.

Thanks again to everyone for your time and direction.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23394] No garbage collection at end of main thread

[Irit Katriel]

Irit Katriel  added the comment:

For the use case you describe, you could add a close() function to your library 
so that callers can make shutdown explicit, or if you want it to remain 
implicit you could use daemon threads which terminate when the main thread 
exits.

Relying on GC to release objects which then close non-daemonic threads from 
__del__ doesn't sound like a robust design to me.

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20115] NUL bytes in commented lines

[Terry J. Reedy]

Terry J. Reedy  added the comment:

https://docs.python.org/3/reference/toplevel_components.html#file-input
says that file input and exec input (should) have the same grammar. This 
implies that the divergence is a bug.

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44434] _thread module: Remove redundant PyThread_exit_thread() call to avoid glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work

[STINNER Victor]

STINNER Victor  added the comment:

_thread.start_new_thread() always called "exit thread", since the function was 
added to Python:

commit 1984f1e1c6306d4e8073c28d2395638f80ea509b
Author: Guido van Rossum 
Date:   Tue Aug 4 12:41:02 1992 +

* Makefile adapted to changes below.
* split pythonmain.c in two: most stuff goes to pythonrun.c, in the library.
* new optional built-in threadmodule.c, build upon Sjoerd's thread.{c,h}.
* new module from Sjoerd: mmmodule.c (dynamically loaded).
* new module from Sjoerd: sv (svgen.py, svmodule.c.proto).
* new files thread.{c,h} (from Sjoerd).
* new xxmodule.c (example only).
* myselect.h: bzero -> memset
* select.c: bzero -> memset; removed global variable

static void
t_bootstrap(args_raw)
void *args_raw;
{
object *args = (object *) args_raw;
object *func, *arg, *res;

restore_thread((void *)NULL);
func = gettupleitem(args, 0);
arg = gettupleitem(args, 1);
res = call_object(func, arg);
DECREF(arg); /* Matches the INCREF(arg) in thread_start_new_thread */
if (res == NULL) {
fprintf(stderr, "Unhandled exception in thread:\n");
print_error(); /* From pythonmain.c */
fprintf(stderr, "Exiting the entire program\n");
goaway(1);
}
(void) save_thread();
exit_thread();
}

exit_thread() was partially replaced with PyThread_exit_thread() in:

commit bcc207484a0f8f27a684e11194e7430c0710f66d
Author: Guido van Rossum 
Date:   Tue Aug 4 22:53:56 1998 +

Changes for BeOS, QNX and long long, by Chris Herborth.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44434] _thread module: Remove redundant PyThread_exit_thread() call to avoid glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work

[STINNER Victor]

STINNER Victor  added the comment:

PyThread_exit_thread() was modified in 2011 to fix daemon threads:

commit 0d5e52d3469a310001afe50689f77ddba6d554d1
Author: Antoine Pitrou 
Date:   Wed May 4 20:02:30 2011 +0200

Issue #1856: Avoid crashes and lockups when daemon threads run while the
interpreter is shutting down; instead, these threads are now killed when
they try to take the GIL.

 PyThread_exit_thread(void)
 {
 dprintf(("PyThread_exit_thread called\n"));
-if (!initialized) {
+if (!initialized)
 exit(0);
-}
+pthread_exit(0);
 }


This change remains important for Python/ceval.c. When a daemon thread tries to 
acquire the GIL, it calls PyThread_exit_thread() if Python already exited to 
exit immediately the thread. Example from take_gil():

if (tstate_must_exit(tstate)) {
/* bpo-39877: If Py_Finalize() has been called and tstate is not the
   thread which called Py_Finalize(), exit immediately the thread.

   This code path can be reached by a daemon thread after Py_Finalize()
   completes. In this case, tstate is a dangling pointer: points to
   PyThreadState freed memory. */
PyThread_exit_thread();
}

See also my articles on daemon threads fixes:

* https://vstinner.github.io/gil-bugfixes-daemon-threads-python39.html
* https://vstinner.github.io/daemon-threads-python-finalization-python32.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Irit Katriel]

Irit Katriel  added the comment:

I don't think this is true anymore:

>>> def a(): 
...   print('a')
...   return (1,2)
... 
>>> def b():
...   print('b')
...   return (3,4)
... 
>>> def f(*args, b=None):
...   print('f')
... 
>>> f(*a(), b=b())
a
b
f
>>>

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44337] Port LOAD_ATTR to adaptive interpreter

[Mark Shannon]

Change by Mark Shannon :


--
pull_requests: +25344
pull_request: https://github.com/python/cpython/pull/26759

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44434] _thread module: Remove redundant PyThread_exit_thread() call to avoid glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +25343
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26758

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44392] Py_GenericAlias is not documented

[Ken Jin]

Change by Ken Jin :


--
pull_requests: +25342
pull_request: https://github.com/python/cpython/pull/26757

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20115] NUL bytes in commented lines

[Irit Katriel]

Irit Katriel  added the comment:

This is still the same in 3.11. I added another line to the example's file, 
which shows more clearly what's happening:

>>> open('x.py', 'wb').write(b'#\x00\na\nb\n')

% ./python.exe x.py
Traceback (most recent call last):
  File "x.py", line 2, in 
a
NameError: name 'b' is not defined

--
nosy: +iritkatriel
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.7, Python 3.3, Python 
3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44392] Py_GenericAlias is not documented

[miss-islington]

miss-islington  added the comment:


New changeset 84ce737f73136c1de7c4dc92bc096ed6c963e42d by Miss Islington (bot) 
in branch '3.10':
bpo-44392: Add Py_GenericAlias to C API docs (GH-26724)
https://github.com/python/cpython/commit/84ce737f73136c1de7c4dc92bc096ed6c963e42d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28937] str.split(): allow removing empty strings (when sep is not None)

[Andrei Kulakov]

Andrei Kulakov  added the comment:

Just to sum up the current state the way I see it, as well as the history of 
the discussion, I think there were 2 initial requests based on experience and 
one additional, more theoretical "nice to have":

A. ''.split() => ['']
B. ''.split(sep) => []  # where sep!=None

C. a way to get the current semantics of sep=None, but with specific whitespace 
separators like just spaces or just tabs. 'ab'.split(' ') => ['a','b']

The idea was to cover all 3 enhancements with the current patch.

As I pointed out in the comments above, current patch does not "cleanly" cover 
case B, potentially leading to confusion and/or bugs.

My suggestion was to cover cases A and B, and leave out C, potentially for some 
future patch.

If we go with the current patch, there will be no practical way to fix the 
issue with B later, other than adding a new `str.split2`. Conversely, it would 
be possible to add a new flag to handle C in the future.

This leads to a few questions:
- will the issue I brought up not really be a problem in practice?
- what's more important, B or C?
- if both B and C are important, can we leave C for a future patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44434] _thread module: Remove redundant PyThread_exit_thread() call to avoid glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-18748 "io.IOBase destructor silence I/O error on close() by 
default" which was caused by a bug in an application, the application closed 
the libgcc_s file descriptor by mistake. It closed the same file decriptor 
twice, whereas the FD was reused by dlopen() in the meanwhile. But the result 
was the same, the process aborted with this error message:

"libgcc_s.so.1 must be installed for pthread_cancel to work"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44434] _thread module: Remove redundant PyThread_exit_thread() call to avoid glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work

[STINNER Victor]

New submission from STINNER Victor :

The glibc pthread_exit() functions loads an unwind function from libgcc_s.so.1 
using dlopen(). dlopen() can fail to open libgcc_s.so.1 file to various 
reasons, but the most likely seems to be that the process is out of available 
file descriptor (EMFILE error).

If the glibc pthread_exit() fails to open libgcc_s.so.1, it aborts the process. 
Extract of pthread_cancel():

  /* Trigger an error if libgcc_s cannot be loaded.  */
  {
struct unwind_link *unwind_link = __libc_unwind_link_get ();
if (unwind_link == NULL)
  __libc_fatal (LIBGCC_S_SO
" must be installed for pthread_cancel to work\n");
  }

Sometimes, libgcc_s.so.1 library is loaded early in Python startup. Sometimes, 
it only loaded when the first Python thread exits.

Hitting in a multithreaded real world application, dlopen() failing with EMFILE 
is not deterministic. It depends on precise timing and in which order threads 
are running. It is unlikely in a small application, but it is more likely on a 
network server which has thousands of open sockets (file descriptors).

--

Attached scripts reproduces the issue. You may need to run the scripts 
(especially pthread_cancel_emfile.py) multiple times to trigger the issue. 
Sometimes libgcc_s library is loaded early for an unknown reason, it works 
around the issue.

(1) pthread_cancel_bug.py 

$ python3.10 pthread_cancel_bug.py 
libgcc_s.so.1 must be installed for pthread_cancel to work
Abandon (core dumped)


(2) pthread_cancel_emfile.py:

$ python3.10 ~/pthread_cancel_emfile.py 
spawn thread
os.open failed: OSError(24, 'Too many open files')
FDs open by the thread: 2 (max FD: 4)
fd 0 valid? True
fd 1 valid? True
fd 2 valid? True
fd 3 valid? True
fd 4 valid? True
libgcc_s.so.1 must be installed for pthread_cancel to work
Abandon (core dumped)

--

Example of real world issue on RHEL8:
https://bugzilla.redhat.com/show_bug.cgi?id=1972293

The RHEL reproducer uses a very low RLIMIT_NOFILE (5 file descriptors) to 
trigger the bug faster. It simulates a busy server application.

--

There are different options:

(*) Modify thread_run() of Modules/_threadmodule.c to remove the *redundant* 
PyThread_exit_thread() call.

This is the most simple option and it sounds perfectly safe to me. I'm not sure 
why PyThread_exit_thread() is called explicitly. We don't pass any parameter to 
the function.


(*) Link the Python _thread extension on libgcc_s.so if Python it built with 
the glibc.

Checking if Python is linked to the glibc is non trivial and we have hardcode 
the "libgcc_s" library name. I expect painful maintenance burden with this 
option.

(*) Load explicitly the libgcc_s.so library in _thread.start_new_thread(): when 
the first thread is created.

We need to detect that we are running the glibc at runtime, by calling 
confstr('CS_GNU_LIBC_VERSION') for example. The problem is that "libgcc_s.so.1" 
filename may change depending on the Linux distribution. It will likely have a 
different filename on macOS (".dynlib"). In short, it's tricky to get it right.

(*) Fix the glibc!

I discussed with glibc developers who explained me that there are good reasons 
to keep the unwind code in the compiler (GCC), and so load it dynamically in 
the glibc. In short, this is not going to change.

--

Attached PR implements the most straightforward option: remove the redundant 
PyThread_exit_thread() call in thread_run().

--
components: Library (Lib)
files: pthread_cancel_bug.py
messages: 395924
nosy: vstinner
priority: normal
severity: normal
status: open
title: _thread module: Remove redundant PyThread_exit_thread() call to avoid 
glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work
versions: Python 3.11
Added file: https://bugs.python.org/file50112/pthread_cancel_bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44434] _thread module: Remove redundant PyThread_exit_thread() call to avoid glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work

[STINNER Victor]

Change by STINNER Victor :


Added file: https://bugs.python.org/file50113/pthread_cancel_emfile.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44392] Py_GenericAlias is not documented

[Guido van Rossum]

Guido van Rossum  added the comment:

Does it need a backport to 3.9 too, since GenericAlias was introduced there?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44392] Py_GenericAlias is not documented

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 7.0 -> 8.0
pull_requests: +25341
pull_request: https://github.com/python/cpython/pull/26756

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44392] Py_GenericAlias is not documented

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset 6773c3eaa735b5061b4a97f2c730703a32d8c9ff by Ken Jin in branch 
'main':
bpo-44392: Add Py_GenericAlias to C API docs (GH-26724)
https://github.com/python/cpython/commit/6773c3eaa735b5061b4a97f2c730703a32d8c9ff


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14879] invalid docs for subprocess exceptions with shell=True

[Jack DeVries]

Change by Jack DeVries :


--
keywords: +patch
nosy: +jack__d
nosy_count: 6.0 -> 7.0
pull_requests: +25340
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26755

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16437] issubclass doc improvement

[Irit Katriel]

Irit Katriel  added the comment:

Whether the recursive nature of the check should be documented or not, it seems 
inconsistent that it's documented for isinstance but not for issubclass.

--
nosy: +iritkatriel
versions: +Python 3.11 -Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14879] invalid docs for subprocess exceptions with shell=True

[Irit Katriel]

Change by Irit Katriel :


--
keywords: +easy
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.7, Python 3.2, Python 
3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44107] HTTPServer can't close http client completely

[Gregory P. Smith]

Change by Gregory P. Smith :


--
resolution: later -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38211] clean up type_init()

[Mark Shannon]

Mark Shannon  added the comment:


New changeset ab030d6f9d73e7f6c2213c2e308d1ceb04761485 by Sergey Fedoseev in 
branch 'main':
bpo-38211: Clean up type_init() (GH-16257)
https://github.com/python/cpython/commit/ab030d6f9d73e7f6c2213c2e308d1ceb04761485


--
nosy: +Mark.Shannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44389] Modules/_ssl.c, repeated 'SSL_OP_NO_TLSv1_2'

[Josh Jiang]

Change by Josh Jiang :


--
nosy: +johnj
nosy_count: 5.0 -> 6.0
pull_requests: +25339
pull_request: https://github.com/python/cpython/pull/26754

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44107] HTTPServer can't close http client completely

[ueJone]

ueJone <775844...@qq.com> added the comment:

Sorry, I've been busy with other things recently.

I think that the problem is't caused by OS, because the other TCP servers were 
disconnected normally and TCP client can access to these servers.

My question is how to make the client connect to the server normally every 
time, instead of why the abnormal disconnect.

The attachment:
   Capture packets of wireshark(HTTPServer listen on port 20245)

--
resolution: not a bug -> later
status: closed -> open
Added file: https://bugs.python.org/file50111/tcpPort_20245.pcapng

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44357] Add math.cbrt() function: Cube Root

[Mark Dickinson]

Change by Mark Dickinson :


--
assignee:  -> mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43475] Worst-case behaviour of hash collision with float NaN

[Mark Dickinson]

Mark Dickinson  added the comment:

I think this can be closed now.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14445] Providing more fine-grained control over assert statements

[Irit Katriel]

Irit Katriel  added the comment:

I am closing this because there was no followup for over 9 years. If you are 
still interested in pursuing this, please raise it for discussion on 
python-ideas and open a new issue once there is agreement on how to proceed.

--
nosy: +iritkatriel
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44429] Tkinter Flow Geometry Manager

[E. Paine]

E. Paine  added the comment:

> But we do not want to risk conflicting with Tk.
I think it's worth noting that tkinter does already have some 'extra' 
functionality over just what Tk offers (such as the ttk extension widgets 
`LabeledScale` and `OptionMenu`).


While this would (IMO) be a useful addition, it may be better suited as a 
third-party library.

--
nosy: +epaine

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43795] Implement PEP 652 -- Maintaining the Stable ABI

[miss-islington]

miss-islington  added the comment:


New changeset 7fd40101b7130016fc98f05ce34746fed68ab542 by Miss Islington (bot) 
in branch '3.10':
bpo-43795: Don't list private names in the limited API (GH-26740)
https://github.com/python/cpython/commit/7fd40101b7130016fc98f05ce34746fed68ab542


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44422] threading.enumerate(): reentrant call during a GC collection hangs

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44422] threading.enumerate(): reentrant call during a GC collection hangs

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 0729694246174a5c2f0ae197f2e0dbea61b90c9f by Victor Stinner in 
branch 'main':
bpo-44422: threading.Thread reuses the _delete() method (GH-26741)
https://github.com/python/cpython/commit/0729694246174a5c2f0ae197f2e0dbea61b90c9f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43795] Implement PEP 652 -- Maintaining the Stable ABI

[Petr Viktorin]

Petr Viktorin  added the comment:


New changeset 7cad9cb51bdae2144cbab330f13a607ba3471742 by Petr Viktorin in 
branch 'main':
bpo-43795: Don't list private names in the limited API (GH-26740)
https://github.com/python/cpython/commit/7cad9cb51bdae2144cbab330f13a607ba3471742


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43795] Implement PEP 652 -- Maintaining the Stable ABI

[miss-islington]

Change by miss-islington :


--
pull_requests: +25338
pull_request: https://github.com/python/cpython/pull/26753

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44310] Document that lru_cache uses hard references

[Henk-Jaap Wagenaar]

Henk-Jaap Wagenaar  added the comment:

PR 26731 looks very good to me. My only comment, which I am not sure is worthy 
of adding/is a general lru_cache thing, that "instances
are kept alive until they age out of the cache or until the cache is
cleared", if you are creating instances and calling this method all the time, 
will lead to an infinite memory leak.

Not sure whether that's too specific to the problem we encountered and we are 
all consenting adults and should infer this or it is helpful: leave it up to 
your/other people's judgement.

P.S. In the programming.rst there is also the "Why are default values shared 
between objects?" section which actually uses default values to make its own 
poor version of a cache. It should probably at least mention lru_cache could be 
used (unless you particularly need callers to be able to pass their own cache).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44429] Tkinter Flow Geometry Manager

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I concur with Zachary. If the flow geometry manager be added in Tk we will add 
its support in Tkinter. We can even add pure Python implementation for older Tk 
versions. But we do not want to risk conflicting with Tk.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39679] functools: singledispatchmethod doesn't work with classmethod

[Dmitry Kulazhenko]

Change by Dmitry Kulazhenko :


--
nosy: +EugenePY, FFY00, Viktor Roytman, glyph, levkivskyi, lukasz.langa, 
markgrandi, mental, ncoghlan, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39679] functools: singledispatchmethod doesn't work with classmethod

[Dmitry Kulazhenko]

Dmitry Kulazhenko  added the comment:

Based on what I've read, workaround:


from functools import singledispatchmethod


def _register(self, cls, method=None):
if hasattr(cls, "__func__"):
setattr(cls, "__annotations__", cls.__func__.__annotations__)
return self.dispatcher.register(cls, func=method)


singledispatchmethod.register = _register

--
nosy: +dmkulazhenko -EugenePY, FFY00, Viktor Roytman, glyph, levkivskyi, 
lukasz.langa, markgrandi, mental, ncoghlan, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44433] processes created by subprocess.Popen is not terminating

[Rajeev Chaurasia]

New submission from Rajeev Chaurasia :

I am using Linux OS and using multiprocessing in my application.

The child process execution get completed but the processes are not terminating.


childprocess = "/opt/oracle.ahf/exachk/exachk -silentforce -showpass -dball 
-nocvu -autorun_id DEFAULT -localonly -sudo_remoterun 0"

session = subprocess.Popen(childprocess, shell = True)
session.communicate()  

>From the log I can see execution of childprocess scripts(exachk and exachk.py) 
>is completed successfully but the process remains alive forever. Also the 
>parent process from where I am using subprocess.Popen that also remains alive.

# ps -ef|grep exachk
(P1) 278988 247699  0 18:46 ?00:00:00 sh 
/opt/oracle.ahf/exachk/exachk -silentforce -showpass -dball -nocvu -autorun_id 
DEFAULT -localonly -sudo_remoterun 0
(P2) 279873 278988 39 18:46 ?00:12:51 
/opt/oracle.ahf/python/bin/python /opt/oracle.ahf/exachk/exachk.py -silentforce 
-showpass -dball -nocvu -autorun_id DEFAULT -localonly


Please help!
-Rajeev

--
components: Library (Lib)
messages: 395909
nosy: rajeevkchaurasia
priority: normal
severity: normal
status: open
title: processes created by subprocess.Popen is not terminating
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44432] SourceFileLoader.load_module() is mixing content of previosly loaded files

[Andrew Johnson]

New submission from Andrew Johnson :

Hi,

I'm loading multiple files in same convention - same filename, as a part of 
creating a build system. I can compare the case to the loading multple 
configuration files for various plugins in the application, the file contents 
of multiple files from different directories are mixed together which looks 
dangerous.

Here is a reproducible case for Python 3.9.4 (running on Arch Linux)
https://github.com/blackandred/python-bug-sourcefileloader

--
hgrepos: 406
messages: 395908
nosy: blackandred
priority: normal
severity: normal
status: open
title: SourceFileLoader.load_module() is mixing content of previosly loaded 
files
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44415] sys.stdout.flush and print() hanging

[Rajeev Chaurasia]

Rajeev Chaurasia  added the comment:

Thanks a lot Serhiy.
Your suggested fix resolved the issue.
-Rajeev

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com