[issue36963] PyDict_GetItem SegFaults on simple dictionary lookup when using Ctypes

[Apoorv Reddy]

Change by Apoorv Reddy :


--
nosy: +amaury.forgeotdarc, belopolsky, benjamin.peterson, christian.heimes, 
duaneg, ebarry, georg.brandl, inada.naoki, larry, meador.inge, ned.deily, 
rhettinger, serhiy.storchaka, tehybel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36963] PyDict_GetItem SegFaults on simple dictionary lookup when using Ctypes

[Apoorv Reddy]

New submission from Apoorv Reddy :

I'm trying to use ctypes to speed up an internal function in my project. 
However, I'm getting a segmentation fault on a simple dictionary lookup in my C 
Code, on PyDict_GetItem(dict, key).

I have supplied a minimal version of the code I'm trying in C, which segfaults, 
with the Makefile and the Python driver code which loads the shared library.I 
have created a simple conda environment for Python 3.6 for the same.

--
components: ctypes
files: bug.tar.gz
messages: 342834
nosy: apoorvreddy
priority: normal
severity: normal
status: open
title: PyDict_GetItem SegFaults on simple dictionary lookup when using Ctypes
type: crash
versions: Python 3.6
Added file: https://bugs.python.org/file48336/bug.tar.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20410] Argument Clinic: add 'self' return converter

[Batuhan]

Batuhan  added the comment:

Doesn't clinic have a `self_converter` class? 
https://github.com/python/cpython/blame/master/Tools/clinic/clinic.py#L3465

--
nosy: +BTaskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36949] WeakSet.__repr__ and __str__ do not show contents of the set

[Batuhan]

Change by Batuhan :


--
keywords: +patch
pull_requests: +13326
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36962] Cant sort ModuleInfo instances

[Batuhan]

Change by Batuhan :


--
keywords: +patch
pull_requests: +13325
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36962] Cant sort ModuleInfo instances

[Batuhan]

Batuhan  added the comment:

I think dataclasses can be used to do it with order, frozen true parameters. 
Also a __getitem__ is required for doesnt break anything.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36919] Exception from 'compile' reports a newline char not present in input

[Anthony Sottile]

Anthony Sottile  added the comment:

still not able to answer the why, but at least I can answer the what here:

https://github.com/python/cpython/blob/f665b96e92a6a6943e312e2c606f348db95939ab/Parser/tokenizer.c#L984-L987

cpython adds a newline during tokenization if the file does not end in a newline

--
nosy: +Anthony Sottile

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36962] Cant sort ModuleInfo instances

[Batuhan]

New submission from Batuhan :

I can't sort the result of iter_modules;
>>> import random, pkgutil
>>> modules = list(pkgutil.iter_modules(None))
>>> random.shuffle(modules)
>>> sorted(modules)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: '<' not supported between instances of 'FileFinder' and 'FileFinder'

--
components: Library (Lib)
messages: 342830
nosy: BTaskaya
priority: normal
severity: normal
status: open
title: Cant sort ModuleInfo instances
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36783] No documentation for _FromXandFold C API functions

[miss-islington]

miss-islington  added the comment:


New changeset 951b161857a840d4d14de0a5a6610e212d78ab68 by Miss Islington (bot) 
in branch '3.7':
bpo-36783: Add new references for C API Documentation changes (GH-13204)
https://github.com/python/cpython/commit/951b161857a840d4d14de0a5a6610e212d78ab68


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36783] No documentation for _FromXandFold C API functions

[miss-islington]

Change by miss-islington :


--
pull_requests: +13324

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36959] ISO date errors in _strptime are jumbled

[Gordon P. Hemsley]

Gordon P. Hemsley  added the comment:

Ah yes, to be clear, I wasn't trying to suggest that the error messages 
themselves were wrong—just that they weren't triggering when the tests were 
expecting them to.

Some of the existing tests currently trigger the "unconverted data remains" 
ValueError from earlier in the method, but because the messages are not 
checked, that is not immediately obvious. I've also added new tests for 
additional scenarios that would presumably also be considered invalid, based on 
the existing ones.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36917] ast.NodeVisitor no longer calls visit_Str

[Batuhan]

Batuhan  added the comment:

What about adding visit_Constant to NodeVisitor for at least one relase period 
and call visit_Str, visit_Num etc?

--
nosy: +BTaskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33519] Should MutableSequence provide .copy()?

[Cheryl Sabella]

Change by Cheryl Sabella :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33519] Should MutableSequence provide .copy()?

[Cheryl Sabella]

Cheryl Sabella  added the comment:


New changeset 9892f454d11b7ea9ba394a115b3e6f48ef6f78fe by Cheryl Sabella (Jelle 
Zijlstra) in branch 'master':
bpo-33519: clarify that .copy() is not part of the MutableSequence ABC (GH-6965)
https://github.com/python/cpython/commit/9892f454d11b7ea9ba394a115b3e6f48ef6f78fe


--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36691] SystemExit & sys.exit : Allow both exit status and message

[Eric V. Smith]

Eric V. Smith  added the comment:

You could add a flag in the exception to tell the default handler where to 
write the message. It would default to stderr.

This would make it possible to catch the exception before it was written out, 
and customize the behavior.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32397] textwrap output may change if you wrap a paragraph twice

[Cheryl Sabella]

Cheryl Sabella  added the comment:

@larry, it looks like this was close to being merged pending some review 
comments by Serhiy.  Although this is considered a bug and not a new feature, 
it might be nice to try to get this in for 3.8.  Thanks!

--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36958] IDLE should print exit message or status if one is provided

[Terry J. Reedy]

Terry J. Reedy  added the comment:

https://docs.python.org/3/library/exceptions.html#SystemExit says: skip 
traceback, convert default None to 0, print any other non-int and convert to 1, 
and pass unprinted int to C exit().

I agree that IDLE should also print non-ints.  The IDLE doc should mention its 
non-exit behavior.

[https://docs.python.org/3/library/sys.html#sys.exit adds more about int return 
codes and that exit() only exits the process in the main thread.  Neither is 
relevant here.]

I believe the relevant code is the following, from run.Executive.runcode:

except SystemExit:
# Scripts that raise SystemExit should just
# return to the interactive prompt
pass
except:
self.usr_exc_info = sys.exc_info()
if quitting:
exit()
# even print a user code SystemExit exception, continue
print_exception()
jit = self.rpchandler.console.getvar("<>")
if jit:
self.rpchandler.interp.open_remote_stack_viewer()

The bare except clause, including the comment, is from 2003.  The 'except 
SystemExit' clause was added 2013 June 11 in #18196, a follow-up of comments in 
#5492.  The obsoleted comment should have been deleted.  The behavior 
suppressed was always printing traceback + message.  What should have been 
retained was printing non-int messages, but I don't think that either Roger or 
I were aware of that behavior.

One could argue that SystemExit in user code should trigger an exit from the 
user execution process, which would trigger a Shell Restart.  This would be 
closer to the standard behavior.  But it does not hurt, and may be better, to 
keep the process and let the user trigger a restart when wanted.  In the case 
that inspired #18196, the SystemExit came from site.py, not user code, and 
definitely should not cause a restart.

I will try something like the following:

except SystemExit as e:
ob = e.args[0]
if not isinstance(ob, (type(None), int)):
print('SystemExit: ' + str(ob), file=sys.stderr)

Since the message will be followed by a normal prompt rather than an exit, I 
want to 'enhance' the message with the prefix and error color.

--
stage:  -> needs patch
type:  -> behavior
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36961] ast_unparser.c doesn't handle PEP570

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset da6129e821099c1372d511a11d18af83d6d5d128 by Pablo Galindo in 
branch 'master':
bpo-36961: Handle positional-only arguments in uparse.c (GH-13412)
https://github.com/python/cpython/commit/da6129e821099c1372d511a11d18af83d6d5d128


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36961] ast_unparser.c doesn't handle PEP570

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35893] distutils fails to build extension on windows when it is a package.__init__

[Evandro Coan]

Evandro Coan  added the comment:

It is missing the import on:

#START
from distutils.command import build_ext

def get_export_symbols(self, ext):
parts = ext.name.split(".")
print('parts', parts)
if parts[-1] == "__init__":
initfunc_name = "PyInit_" + parts[-2]
else:
initfunc_name = "PyInit_" + parts[-1]

build_ext.build_ext.get_export_symbols = get_export_symbols
#END

--
nosy: +evandrocoan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36826] ast_unparser.c doesn't handle := expressions

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Thanks for the patch Batuhan!

--
nosy: +pablogsal
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36961] ast_unparser.c doesn't handle PEP570

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +13323
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36961] ast_unparser.c doesn't handle PEP570

[Pablo Galindo Salgado]

New submission from Pablo Galindo Salgado :

Any line in test_annotations() containing positional-only markers
in Lib/test/test_future.py will fail:

eq('lambda x, /: x')

Run tests sequentially
0:00:00 load avg: 0.71 [1/1] test_future
test test_future failed -- Traceback (most recent call last):
  File "/home/pablogsal/github/cpython/Lib/test/test_future.py", line 186, in 
test_annotations
eq("lambda a, /, b, c=True, *vararg, d, e='str', **kwargs: a + b")
  File "/home/pablogsal/github/cpython/Lib/test/test_future.py", line 141, in 
assertAnnotationEqual
self.assertEqual(actual, expected)
AssertionError: "lambda b, c=True, *vararg, d, e='str', **kwargs: a + b" != 
"lambda a, /, b, c=True, *vararg, d, e='str', **kwargs: a + b"
- lambda b, c=True, *vararg, d, e='str', **kwargs: a + b
+ lambda a, /, b, c=True, *vararg, d, e='str', **kwargs: a + b
?++


test_future failed

== Tests result: FAILURE ==

1 test failed:
test_future

Total duration: 46 ms
Tests result: FAILURE

--
assignee: pablogsal
components: Interpreter Core
messages: 342819
nosy: pablogsal
priority: normal
severity: normal
status: open
title: ast_unparser.c doesn't handle PEP570
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36567] DOC: manpage directive doesn't create hyperlink

[Berker Peksag]

Berker Peksag  added the comment:


New changeset eab99650799699f766c2660f4cfa8ff3f9e8457f by Berker Peksag 
(Batuhan Taşkaya) in branch 'master':
bpo-36567: Use manpages_url to create links for man pages (GH-13339)
https://github.com/python/cpython/commit/eab99650799699f766c2660f4cfa8ff3f9e8457f


--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2180] tokenize: mishandles line joining

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Thanks for figuring this one out Anthony! :)

--
resolution:  -> fixed
stage: patch review -> commit review
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36959] ISO date errors in _strptime are jumbled

[Paul Ganssle]

Paul Ganssle  added the comment:

Hm, I was a bit confused by your wording here, because I am able to trigger all 
the errors just fine even before this PR, but I do think that even though this 
isn't necessarily fixing inaccurate error messages (all the error messages 
*are* accurate), you're right that the text of the messages does seem to 
indicate that the original authors intended a cascade ordering more like the 
one you've proposed.

I'll give this a more thorough review a bit later, thanks for working on this!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19376] document that strptime() does not support the Feb 29 if the format does not contain the year

[Cheryl Sabella]

Change by Cheryl Sabella :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19376] document that strptime() does not support the Feb 29 if the format does not contain the year

[Cheryl Sabella]

Cheryl Sabella  added the comment:


New changeset 56027ccd6b9dab4a090e4fef8574933fb9a36ff2 by Cheryl Sabella 
(Abhishek Kumar Singh) in branch 'master':
bpo-19376: Added doc mentioning `datetime.strptime()` without a year fails for 
Feb 29. (GH-10243)
https://github.com/python/cpython/commit/56027ccd6b9dab4a090e4fef8574933fb9a36ff2


--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36960] Make datetime docs more user-friendly

[Cheryl Sabella]

Cheryl Sabella  added the comment:

See also issue8822.

--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36960] Make datetime docs more user-friendly

[Cheryl Sabella]

Change by Cheryl Sabella :


--
nosy: +belopolsky, p-ganssle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35849] Added thousands separators to Lib/pstats.py final report

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +13322

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36960] Make datetime docs more user-friendly

[Brad Solomon]

New submission from Brad Solomon :

The datetime docs are chalk full of detail.  This is a positive aspect,
and represents a huge amount of work by Tim Peters and A.M. Kuchling.
However, it also may function as an obstacle for beginner readers and
those simply seeking to answer a basic question or see a straightforward
usage example.  Rather than seeing an example-based explanation of
a common use-case, they are bombarded with technical detail and
edge cases.

I propose some restructuring of the datetime docs with the goal of
making them more reader-friendly.  The goal is not to eliminate any
of the detail, but to restructure things so as to bring the "everyday"
parts into more prominent real estate.

The changes here all make an effort to reflect what's espoused by
"Documenting Python" at https://devguide.python.org/documenting/.

I have some additional changes in mind but wanted to put this here now
to gauge receptiveness to the existing changes.

--
assignee: docs@python
components: Documentation
messages: 342813
nosy: bsolomon1124, docs@python
priority: normal
pull_requests: 13321
severity: normal
status: open
title: Make datetime docs more user-friendly
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25988] collections.abc.Indexable

[Matthias Bussonnier]

Change by Matthias Bussonnier :


--
pull_requests: +13320

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36959] ISO date errors in _strptime are jumbled

[Gordon P. Hemsley]

Gordon P. Hemsley  added the comment:

I've created a PR that fixes the issue, which I discovered while evaluating the 
test coverage for _strptime.

Certain scenarios of error messages were never being hit because the cascade 
was out of order, and the tests were not showing that because they were 
throwing a different ValueError than the one they were expecting to throw.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36959] ISO date errors in _strptime are jumbled

[Gordon P. Hemsley]

Change by Gordon P. Hemsley :


--
keywords: +patch
pull_requests: +13319
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36959] ISO date errors in _strptime are jumbled

[Paul Ganssle]

Paul Ganssle  added the comment:

@gphelmsley Can you clarify what you mean by this?

Do you have a minimal reproducing example that shows what's happening and what 
you are expecting?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36959] ISO date errors in _strptime are jumbled

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +p-ganssle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36959] ISO date errors in _strptime are jumbled

[Gordon P. Hemsley]

New submission from Gordon P. Hemsley :

This has not been apparent because the tests for this code are not testing what 
they think they're testing.

--
components: Library (Lib), Tests
messages: 342810
nosy: gphemsley
priority: normal
severity: normal
status: open
title: ISO date errors in _strptime are jumbled
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36958] IDLE should print exit message or status if one is provided

[Random832]

New submission from Random832 :

IDLE currently just returns to the interactive prompt when a script exits, even 
if SystemExit has arguments. This can be confusing to new users if they are 
using sys.exit to print a message.

--
assignee: terry.reedy
components: IDLE
files: run.py.patch
keywords: patch
messages: 342809
nosy: Random832, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE should print exit message or status if one is provided
versions: Python 3.7
Added file: https://bugs.python.org/file48335/run.py.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36950] test.support: add an helper to wait for an event with a timeout

[Pierre Glaser]

Pierre Glaser  added the comment:

Just did so.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36950] test.support: add an helper to wait for an event with a timeout

[Pierre Glaser]

Change by Pierre Glaser :


--
keywords: +patch
pull_requests: +13318
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2180] tokenize: mishandles line joining

[miss-islington]

miss-islington  added the comment:


New changeset abea73bf4a320ff658c9a98fef3d948a142e61a9 by Miss Islington (bot) 
(Anthony Sottile) in branch 'master':
bpo-2180: Treat line continuation at EOF as a `SyntaxError` (GH-13401)
https://github.com/python/cpython/commit/abea73bf4a320ff658c9a98fef3d948a142e61a9


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36953] Remove collections ABCs?

[Matthias Bussonnier]

Matthias Bussonnier  added the comment:

Should it still raise an informative error message with ImportError:

> ImportError: cannot import name 'XXX' from 'collections', please import it 
> from 'collections.abc'.

or just the "cannot import name ''" without the  "please import it from 
'collections.abc'." ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36950] test.support: add an helper to wait for an event with a timeout

[STINNER Victor]

STINNER Victor  added the comment:

Would you be interested to propose a PR? Maybe raise an assertion error by
default, but allow to customize it in the API?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36950] test.support: add an helper to wait for an event with a timeout

[Pierre Glaser]

Pierre Glaser  added the comment:

Lib/test/test_asyncio/utils.py defines a similar helper:

def run_until(loop, pred, timeout=30):
deadline = time.monotonic() + timeout
while not pred():
if timeout is not None:
timeout = deadline - time.monotonic()
if timeout <= 0:
raise futures.TimeoutError()
loop.run_until_complete(tasks.sleep(0.001))

If we trim the ``loop`` usage, we have a rather simple helper can be used to 
rewrite a decent number of tests, such as:

- _test_multiprocessing.py _TestBarrier._test_reset_f
- _test_multiprocessing.py _TestPoolWorkerLifetime.test_pool_worker_lifetime
- _test_multiprocessing.py TestSyncManagerTypes.test_wait_proc_exit
- fork_wait. ForkWait.test_wait
- test_concurrent_futures.py FailingInitializerMixin.test_initializer
- test_fork1.py ForkTest.waitimpl
- test_logging.py HandlerTests.test_post_fork_child_no_deadlock
- test_os. Win32KillTests._kill
- test_signal.py StressTest.test_stress_delivery_dependent
- test_signal.py StressTest.test_stress_delivery_simulatenous
- test_wait4.py Wait4Test.wait_impl

As well as some top-level commands in:
- test_multiprocessing_main_handling.py some top level instructions
- subprocessdata/sigchlild_ignore.py
- support/__init__.py


I also witnessed some slightly more complex patterns that does not easily fit 
into the asyncio helper:

# eintr_tester.py FNTREINTLTest._lock
while True:  # synchronize the subprocess
dt = time.monotonic() - start_time
if dt > 60.0:
raise Exception("failed to sync child in %.1f sec" %
dt)
try:
lock_func(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
lock_func(f, fcntl.LOCK_UN)
time.sleep(0.01)
except BlockingIOError:
break

Which is also (IMO) the case for the lines quoted by Victor.

However, such more complex structures do not seem to appear that often, so 
sticking to run_until and moving it to test.support.script_helper may be enough.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36546] Add quantiles() to the statistics module

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset e917f2ed9af044fe808fc9b4ddc6c5eb99003500 by Raymond Hettinger in 
branch 'master':
bpo-36546: Add more tests and expand docs (#13406)
https://github.com/python/cpython/commit/e917f2ed9af044fe808fc9b4ddc6c5eb99003500


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24932] Use proper command line parsing in _testembed

[Eric Snow]

Change by Eric Snow :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36957] Speed up math.isqrt

[Mark Dickinson]

Mark Dickinson  added the comment:

> Did you try the floating point implementation?

The aim here was to use exactly the same algorithm, but speed it up by working 
with C integers where possible; that's a fairly simple change.

Using floating-point would require more complex changes. Again, the biggest 
issue with using a floating-point sqrt as an initial value is that we can't 
assume either IEEE 754 floating-point format, *or* a correctly rounded libm 
sqrt, even though both those things are highly likely on a typical modern 
machine. So any use of floating-point would also have to have an accuracy check 
and a fallback integer-only implementation for the case where the 
floating-point fails. It's possible to make those changes, but I think we'd end 
up crossing the threshold to "too complicated" for the implementation of a 
simple function.

It's a bit of a shame, because if we _are_ allowed to assume IEEE 754, and a 
correctly-rounded sqrt implementation (using round-ties-to-even), then it turns 
out that one can prove that for any value `n` smaller than 2**106 and `a := 
int(math.sqrt(float(n)))` (assuming that the `int` and `float` conversions are 
*also* correctly rounded), we have (a - 1)**2 < n < (a + 1)**2, which is 
exactly the loop invariant that the current algorithm needs to maintain.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36546] Add quantiles() to the statistics module

[Raymond Hettinger]

Change by Raymond Hettinger :


--
pull_requests: +13317

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36957] Speed up math.isqrt

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Did you try the floating point implementation?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36957] Speed up math.isqrt

[Mark Dickinson]

Mark Dickinson  added the comment:

> introduce in GH-36887

Sorry, that should have been: introduced in GH-13244. #36887 was the 
corresponding b.p.o. issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36957] Speed up math.isqrt

[Mark Dickinson]

Change by Mark Dickinson :


--
keywords: +patch
pull_requests: +13316
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36957] Speed up math.isqrt

[Mark Dickinson]

New submission from Mark Dickinson :

The `math.isqrt` algorithm introduce in GH-36887 currently works entirely with 
Python long integers. That's unnecessarily inefficient for small inputs.

For n < 2**64, `math.isqrt(n)` can be computed, via exactly the same algorithm, 
using entirely C integer arithmetic.

For larger n, the first 5 iterations of the algorithm can similarly be 
performed entirely in C integer arithmetic, and we can then switch to long 
integer arithmetic for subsequent iterations.

On my machine, these simple changes make a substantial difference (4x faster) 
for small inputs, and a significant but less substantial difference (70% 
speedup) for inputs not much larger than 2**64. The speedup for huge integers 
is likely to be much smaller, percentage-wise.

Some timings:

Unpatched
-
lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt" 
"[isqrt(n) for n in range(2, 1000)]"
1000 loops, best of 5: 327 usec per loop
lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt; 
x = range(2**63-1000, 2**63+1000)" "[isqrt(n) for n in x]"
200 loops, best of 5: 1.44 msec per loop
lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt; 
x = range(2**95-1000, 2**95+1000)" "[isqrt(n) for n in x]"
200 loops, best of 5: 1.64 msec per loop

Patched (PR imminent)
---
lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt" 
"[isqrt(n) for n in range(2, 1000)]"
5000 loops, best of 5: 78.1 usec per loop
lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt; 
x = range(2**63-1000, 2**63+1000)" "[isqrt(n) for n in x]"
1000 loops, best of 5: 355 usec per loop
lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt; 
x = range(2**95-1000, 2**95+1000)" "[isqrt(n) for n in x]"
500 loops, best of 5: 954 usec per loop

--
assignee: mark.dickinson
components: Extension Modules
messages: 342799
nosy: mark.dickinson
priority: normal
severity: normal
stage: needs patch
status: open
title: Speed up math.isqrt
type: performance
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24882] ThreadPoolExecutor doesn't reuse threads until #threads == max_workers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Thomas, I think that's a good argument, so perhaps we should do this (strive to 
reuse threads) after all.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36956] Calling "functions" used to implement generators/comps easily cause crash

[Dan Snider]

New submission from Dan Snider :

As far as I know, generators, set comprehensions, list comprehensions, and dict 
comprehensions, (along with their asynchronous variants) are implemented by 
first calling the GET_(A)ITER opcode and then building and calling a function 
that acepts the resulting iterator as its sole argument.

Assigning the code object used to make that function (or using it in the 
types.FunctionType constructor) and then calling it with a non-iterator 
argument will obviously cause a crash since the FOR_ITER opcode rightly expects 
that it will never have to deal with non-iterators and calls tp_iternext 
without checking if it exists.

The 4-liner demonstrates the crash:

if 1:
  fn = lambda: None
  gi = (i for i in ())
  fn.__code__ = gi.gi_code
  [*fn("abc")]

--
messages: 342797
nosy: bup
priority: normal
severity: normal
status: open
title: Calling "functions" used to implement generators/comps easily cause crash
type: crash
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36887] Add integer square root, math.isqrt

[Mark Dickinson]

Change by Mark Dickinson :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36950] test.support: add an helper to wait for an event with a timeout

[Pierre Glaser]

Change by Pierre Glaser :


--
nosy: +pierreglaser

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24882] ThreadPoolExecutor doesn't reuse threads until #threads == max_workers

[Pierre Glaser]

Change by Pierre Glaser :


--
nosy: +pierreglaser

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24882] ThreadPoolExecutor doesn't reuse threads until #threads == max_workers

[Thomas]

Thomas  added the comment:

We ran into this issue in the context of asyncio which uses an internal 
ThreadPoolExecutor to provide an asynchronous getaddrinfo / getnameinfo.

We observed an async application spawned more and more threads through several 
reconnects. With a maximum of 5 x CPUs these were dozens of threads which 
easily looked like a resource leak.

At least in this scenario I would strongly prefer to correctly reuse idle 
threads. 

Spawning all possible threads on initialization in such a transparent case 
would be quite bad. Imagine having a process-parallel daemon that running a 
apparently single-threaded asyncio loop but then getting these executors for 
doing a single asyncio.getaddrinfo. Now you run 80 instances on an 80 core 
machine you get 32.000 extra implicit threads.

Now you can argue whether the default executor in asyncio is good as is, but if 
the executors properly reuse threads, it would be quite unlikely to be a 
practical problem.

--
nosy: +tilsche

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Change by Pierre Glaser :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36368] server process of shared_memory shuts down if KeyboardInterrupt

[Pierre Glaser]

Change by Pierre Glaser :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35989] ipaddress.IPv4Network allows prefix > 32

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

See also issue36845 that seems to have fixed this.

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24564] shutil.copytree fails when copying NFS to NFS

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Nosying Hynek and Larry, as they had done the original code.

--
nosy: +cheryl.sabella, hynek, larry
versions:  -Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36887] Add integer square root, math.isqrt

[Mark Dickinson]

Mark Dickinson  added the comment:


New changeset 73934b9da07daefb203e7d26089e7486a1ce4fdf by Mark Dickinson in 
branch 'master':
bpo-36887: add math.isqrt (GH-13244)
https://github.com/python/cpython/commit/73934b9da07daefb203e7d26089e7486a1ce4fdf


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33608] Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.

[Pavel Kostyuchenko]

Pavel Kostyuchenko  added the comment:

Also it might be viable to add some assertion to verify the take_gil is not 
called with uninitialized interpreter.
I used the changes in the attachment (take_gil.assert.patch), but it produced 
errors during test_tracemalloc with f13c5c8b9401a9dc19e95d8b420ee100ac022208 . 
It happens because, during startup with invalid arguments, the interpreter is 
finalized with pymain_main->pymain_free->_PyRuntime_Finalize before the error 
is printed. However, the problem seems to be fixed for me in the last revisions 
of master branch, so I upload the diff against it.

--
Added file: https://bugs.python.org/file48334/take_gil.assert.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33608] Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.

[Pavel Kostyuchenko]

Pavel Kostyuchenko  added the comment:

I was able to reproduce the error with version 
f13c5c8b9401a9dc19e95d8b420ee100ac022208 on FreeBSD 12.0 VM. The error seems to 
be caused not by those changes, but by lack of synchronization in the 
multiprocessing.managers.Server.
The failure happens when running the 
"test_shared_memory_SharedMemoryManager_basics" with high CPU load and frequent 
interrupts e.g. moving some window during test. Mostly I used the "python -m 
test --fail-env-changed test_multiprocessing_spawn -m 'WithProcessesTestS[hu]*' 
-F" command to reproduce the crash.
By analyzing core dumps I deduced that the crash happens during this call from 
the parent test process:

class BaseManager(object):
def _finalize_manager(process, address, authkey, state, _Client):
...
try:
conn = _Client(address, authkey=authkey)
try:
dispatch(conn, None, 'shutdown')
finally:
conn.close()
except Exception:
pass

Main thread in the multiprocessing child:

class Server(object):
def serve_forever(self):
...
try:
accepter = threading.Thread(target=self.accepter)
accepter.daemon = True
accepter.start()
try:
while not self.stop_event.is_set():
self.stop_event.wait(1)
except (KeyboardInterrupt, SystemExit):
pass
finally:
...
sys.exit(0)  << main thread have finished and destroyed the 
interpreter

Worker thread in the multiprocessing child.
Locals:
File "/usr/home/user/cpython/Lib/multiprocessing/managers.py", line 214, in 
handle_request
c.send(msg)
self = 
funcname = 'shutdown'
result = None
request = (None, 'shutdown', (), {})
ignore = None
args = ()
kwds = {}
msg = ('#RETURN', None)

Listing:
class Server(object):
def handle_request(self, c):
...
try:
result = func(c, *args, **kwds)  << calls Server.shutdown method
except Exception:
msg = ('#TRACEBACK', format_exc())
else:
msg = ('#RETURN', result)
try:
c.send(msg)  << crashes with SIGBUS in _send_bytes -> write -> 
take_gil -> SET_GIL_DROP_REQUEST(tstate->interp)
except Exception as e:
try:
c.send(('#TRACEBACK', format_exc()))
except Exception:
pass
...
def shutdown(self, c):
...
try:
util.debug('manager received shutdown message')
c.send(('#RETURN', None))
except:
import traceback
traceback.print_exc()
finally:
self.stop_event.set()

Worker thread is daemonic and is not terminated during the interpreter 
finalization, thus it might still be running and is terminated silently when 
the process exits. The connection (c) has different implementations on several 
platforms, so we cannot be sure whether the connection is closed during 
shutdown or not, whether the last "c.send(msg)" blocks until the end of the 
process, returns instantly, or fails inconsistently.
The error was there for a long time, but for two reasons it didn't cause much 
trouble:
- the race condition is hard to trigger;
- SET_GIL_DROP_REQUEST used to ignore the errorneous state of interpreter, but 
introduction of tstate->interp argument by Eric manifested SIGBUS on FreeBSD.

I haven't managed to find a nice clean test to reproduce the bug automatically. 
I suggest the changes for the multiprocessing/managers.py in the attachment.

--
nosy: +shprotx
Added file: https://bugs.python.org/file48333/managers.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36955] Python3 - mulltiprocessing

[SilentGhost]

SilentGhost  added the comment:

multiprocessing module that is available on PyPi is a backport of stdlib module 
to python 2.4 and 2.5. multiprocessing module is available in stdlib for all 
version after that, so you don't need to install it with pip.

--
nosy: +SilentGhost
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36955] Python3 - mulltiprocessing

[Frode Byrkjeland]

New submission from Frode Byrkjeland :

multiprocessing-2.6.2.1.tar.gz contains setup.py with python2 print statement 
in stead of python3 print( ). This results in an error when trying to install. 
Found on Ubuntu 18.04 x86_64 running python3.6.7
error seems to be ok after doing changes on the print statements in that file.

install cmd: "python3 -m pip install multiprocessing"

--
messages: 342789
nosy: fbyrkjeland
priority: normal
severity: normal
status: open
title: Python3 - mulltiprocessing

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2180] tokenize: mishandles line joining

[Gregory P. Smith]

Change by Gregory P. Smith :


--
assignee:  -> gregory.p.smith
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36951] Wrong types for PyMemberDefs in Objects/typeobject.c

[Zackery Spytz]

Zackery Spytz  added the comment:

I agree that this should be fixed.

--
nosy: +ZackerySpytz
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36951] Wrong types for PyMemberDefs in Objects/typeobject.c

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
pull_requests: +13315
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com