date:20200701



miss-islington  added the comment:


New changeset df59293bf0d815fe37743025d639a63a78e0c771 by Miss Islington (bot) 
in branch '3.9':
bpo-40967: Remove deprecated asyncio.Task.current_task() and 
asyncio.Task.all_tasks() (GH-20874)
https://github.com/python/cpython/commit/df59293bf0d815fe37743025d639a63a78e0c771


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40967] asyncio.Task.all_tasks() and asyncio.Task.current_task() must be removed in 3.9


miss-islington  added the comment:


New changeset 004e64e8059fe68a72890314673282f2e60d5ce1 by Rémi Lapeyre in 
branch 'master':
bpo-40967: Remove deprecated asyncio.Task.current_task() and 
asyncio.Task.all_tasks() (GH-20874)
https://github.com/python/cpython/commit/004e64e8059fe68a72890314673282f2e60d5ce1


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40967] asyncio.Task.all_tasks() and asyncio.Task.current_task() must be removed in 3.9



Change by miss-islington :


--
pull_requests: +20422
pull_request: https://github.com/python/cpython/pull/21272

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41180] marshal load bypass code.new audit event

2020-07-01 Thread Yunfan Zhan



Change by Yunfan Zhan :


--
keywords: +patch
pull_requests: +20421
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21271

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41137] pdb uses the locale encoding for .pdbrc

2020-07-01 Thread Emmanuel Arias



Change by Emmanuel Arias :


--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41100] Build failure on macOS 11 (beta)

2020-07-01 Thread Emmanuel Arias



Change by Emmanuel Arias :


--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39579] Attribute node in a decorator has wrong end_col_offset

2020-07-01 Thread Francis Herne



Francis Herne  added the comment:

Sorry, on further inspection it was the other AST range change 
https://bugs.python.org/issue39474 , not this one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41133] Insufficient description of cyclic garbage collector for C API

2020-07-01 Thread Pablo Galindo Salgado



Pablo Galindo Salgado  added the comment:

I agree with what Tim said. We also cannot foresee every possible bug of 
someone misusing the C API, and the garbage collector interacts with many 
systems of the VM, and having gigantic documentation has shown to be always 
detrimental. 

> I think it would be helpful to have something as troubleshooting information 
> on the garbage collector. If you've got a bug in your C module,

"A bug in your C module" can be absolutely anything (even more things given 
that we are talking about C!). Many people also blame the GC immediately just 
because the crash happens during visitation when the bug already happened much 
before. I don't think that a "general debugging section" about the GC will be 
very as helpful as improving the clarity of the contract.

> For that matter, the documentation for tp_traverse doesn't mention that you 
> have to call Py_VISIT on each object exactly as many times as references you 
> hold to it; the phrasing sounds like it's fine so long as you call it at 
> least once on any object you hold a reference to.

A note in the docs about this sounds reasonable as is well-scoped. If you want, 
you can do a PR and add me there and I will review it. Otherwise, I will try to 
put something together myself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit



Change by STINNER Victor :


--
pull_requests: +20420
pull_request: https://github.com/python/cpython/pull/21270

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit



STINNER Victor  added the comment:


New changeset 666ecfb0957a2fa0df5e2bd03804195de74bdfbf by Victor Stinner in 
branch 'master':
bpo-1635741: Release Unicode interned strings at exit (GH-21269)
https://github.com/python/cpython/commit/666ecfb0957a2fa0df5e2bd03804195de74bdfbf


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39579] Attribute node in a decorator has wrong end_col_offset

2020-07-01 Thread Francis Herne



Francis Herne  added the comment:

Note: this change causes a regression in kdev-python because our code was based 
on (and worked around) the long-standing previous behaviour; it produces broken 
results given the 'fixed' offset.

I shall have to write a patch with a version-check before our next stable 
release.

While the new behaviour is more logical, I'm not convinced that deliberate API 
breakage in a stable release - for something that's not a recent regression - 
was a good idea.

--
nosy: +flherne

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit



Change by STINNER Victor :


--
pull_requests: +20419
pull_request: https://github.com/python/cpython/pull/21269

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter



STINNER Victor  added the comment:


New changeset 90db4653ae37ef90754cfd2cd6ec6857b87a88e6 by Victor Stinner in 
branch 'master':
bpo-40521: Cleanup finalize_interp_types() (GH-21265)
https://github.com/python/cpython/commit/90db4653ae37ef90754cfd2cd6ec6857b87a88e6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39385] Add an assertNoLogs context manager to unittest TestCase

2020-07-01 Thread Vinay Sajip



Change by Vinay Sajip :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39385] Add an assertNoLogs context manager to unittest TestCase

2020-07-01 Thread Vinay Sajip



Vinay Sajip  added the comment:


New changeset 6b34d7b51e33fcb21b8827d927474ce9ed1f605c by Kit Choi in branch 
'master':
bpo-39385: Add an assertNoLogs context manager to unittest.TestCase (GH-18067)
https://github.com/python/cpython/commit/6b34d7b51e33fcb21b8827d927474ce9ed1f605c


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41043] Escape the literal part of the path for glob()

2020-07-01 Thread Petr Viktorin



Petr Viktorin  added the comment:

Would it be worth it to add a "base" keyword argument to glob.glob?

--
nosy: +petr.viktorin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41168] Lack of proper checking in PyObject_SetAttr leads to segmentation fault

2020-07-01 Thread Petr Viktorin



Petr Viktorin  added the comment:

What kind of security-related issues do you mean? A .pyc file is code; it can 
do anything. Never run untrusted code, be it .py or .pyc.

See also: Lib/test/crashers/bogus_code_obj.py

--
nosy: +petr.viktorin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41073] [C API] PyType_GetSlot() should accept static types

2020-07-01 Thread Petr Viktorin



Petr Viktorin  added the comment:

> If we extend PyType_GetSlot() to accept non-heaptype, we need find a way to 
> judge the max slot of non-heaptype.

Static types can have some sub-slots structs but not others. A "max slot" will 
not help for types that have tp_as_mapping but not tp_as_number, for example.
You'll probably need some table like typeslots.inc to record which sub-slots 
struct each slot belongs to.

--
nosy: +petr.viktorin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41191] PyType_FromModuleAndSpec is not mentioned in 3.9 What's new

2020-07-01 Thread Matthieu Dartiailh



New submission from Matthieu Dartiailh :

Looking at the What's new for Python 3.9 I noticed that there was no mention of 
PEP 573. The added functions are properly documented and should probably be 
mentioned in the What's new.

--
assignee: docs@python
components: Documentation
messages: 372790
nosy: docs@python, mdartiailh
priority: normal
severity: normal
status: open
title: PyType_FromModuleAndSpec is not mentioned in 3.9 What's new
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41189] An exploitable segmentation fault in _PyEval_EvalFrameDefault



Iman Sharafodin  added the comment:

Thank you for the response.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41189] An exploitable segmentation fault in _PyEval_EvalFrameDefault



Ned Deily  added the comment:

> my only goal was to help Python community (which I love it) to improve the 
> code quality

Thanks for trying to improve things, we do appreciate it!

The idea here is that to be able to exploit the crashing pyc file, you need to 
be able to run an arbitrary pyc file on the web service and to do that the 
attacker has to have access somehow to the interpreter. If the web service has 
a hole to allow that, many bad things are possible. That's true for many other 
languages and tools, too.  So it's just not worth worrying about being able to 
crash with a fuzzed pyc file since, if you can exploit that, you can exploit in 
much easier ways.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41189] An exploitable segmentation fault in _PyEval_EvalFrameDefault



Iman Sharafodin  added the comment:

You're right. But if someone uses the exact same code to decompile a pyc to a 
Python code, attacker doesn't have access to the interpreter and cannot even 
run the pyc file on the server, but the attacker can cause a crash and run the 
malicious exploit code. Anyway, my only goal was to help Python community 
(which I love it) to improve the code quality. Have a nice day.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41189] An exploitable segmentation fault in _PyEval_EvalFrameDefault



Ned Deily  added the comment:

If users have unrestricted access to the interpreter, there are easier ways to 
crash Python than with modified byte code, for example, as is documented with 
ctypes. As noted on the Python Security Team web page 
(https://www.python.org/dev/security/): "If you can already execute Python 
code, there are far worse things you can do than provoke a use-after-free or an 
interpreter crash."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41100] Build failure on macOS 11 (beta)

2020-07-01 Thread Mark Dickinson



Change by Mark Dickinson :


--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41133] Insufficient description of cyclic garbage collector for C API

2020-07-01 Thread Joannah Nanjekye



Change by Joannah Nanjekye :


--
nosy: +nanjekyejoannah, pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41100] Build failure on macOS 11 (beta)

2020-07-01 Thread Lawrence D'Anna



Change by Lawrence D'Anna :


--
pull_requests: +20418
pull_request: https://github.com/python/cpython/pull/21268

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37999] No longer use implicit convertion to int with loss

2020-07-01 Thread Hans Petter Jansson



Change by Hans Petter Jansson :


--
nosy: +hpj
nosy_count: 2.0 -> 3.0
pull_requests: +20417
pull_request: https://github.com/python/cpython/pull/17536

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41189] An exploitable segmentation fault in _PyEval_EvalFrameDefault



Iman Sharafodin  added the comment:

It could be potential dangerous, for example some services might use Python 
Core to decompile pyc files and they could be hacked or some other services 
could run restricted pyc files for users but using this bug they can escape the 
sandbox and run the malicious code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41187] Convert the _msi module to Argument Clinic



Serhiy Storchaka  added the comment:


New changeset 5d5c84ef78b19211671c2bfa68fe073485135eed by Serhiy Storchaka in 
branch 'master':
bpo-41187: Convert the _msi module to Argument Clinic (GH-21264)
https://github.com/python/cpython/commit/5d5c84ef78b19211671c2bfa68fe073485135eed


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41189] An exploitable segmentation fault in _PyEval_EvalFrameDefault



Ned Deily  added the comment:

Sorry, if you modified the pyc file, that is undefined behavior. We make no 
guarantees that you can't crash the interpreter with arbitrary byte code.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41189] An exploitable segmentation fault in _PyEval_EvalFrameDefault



Iman Sharafodin  added the comment:

I created a Python file with 12 lines of code and then changed the bytecode to 
make Python crash (I was testing Python to find security related bugs). I can 
send the original file, do you need that?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41189] An exploitable segmentation fault in _PyEval_EvalFrameDefault



Ned Deily  added the comment:

Thank you for the report. Can you please supply the Python code that was 
translated into the .pyc file you supplied?  If there is some reason that you 
don't want to post it to this issue, you can email it to secur...@python.org.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41158] IDLE: rewrite the code for handling file encoding



miss-islington  added the comment:


New changeset c3fa7534c7173d338880a1727f17795670518610 by Miss Islington (bot) 
in branch '3.8':
bpo-41158: IDLE: rewrite the code for handling file encoding (GH-21215)
https://github.com/python/cpython/commit/c3fa7534c7173d338880a1727f17795670518610


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41181] [macOS] Build macOS installer with LTO and PGO optimizations



Ned Deily  added the comment:

er, "macOS 11.0 Big Sur" :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38980] Compile libpython with -fno-semantic-interposition

2020-07-01 Thread Tianon



Change by Tianon :


--
nosy: +tianon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41181] [macOS] Build macOS installer with LTO and PGO optimizations



Ned Deily  added the comment:

I should have made it clearer that we expect to release a new installer variant 
for macOS 11.6 Big Sur that supports both Intel and Apple Silicon architectures 
later this year (i.e. in several months) when Big Sur releases. It will be much 
easier to support newer optimizations in that variant.  We are in the process 
right now of getting builds to work on the developer previews and on developer 
hardware. We will look at optimizations for that variant then.

Please drop the idea of trying to change how we build on 10.9 (and, yes, we are 
perfectly capable of finding newer compilers to run on 10.9 but that's not the 
point - we *only* support building installers with standard Apple Developer 
Tool chains and with good reason); hacking on 10.9 is not worth it at this 
point.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41189] An exploitable segmentation fault in _PyEval_EvalFrameDefault

2020-07-01 Thread SilentGhost



Change by SilentGhost :


--
nosy: +ned.deily
type:  -> security

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41190] msilib: SetProperty() accepts str, but GetProperty() returns bytes



New submission from Serhiy Storchaka :

There is an inconsistency in the _msi.SummaryInformation class. Its method 
SetProperty() accepts str, but GetProperty() returns bytes (encoded with the 
Windows ANSI encoding).

Since os.fsencode()/os.fsdecode() now use UTF-8 encoding, it is not so easy to 
convert between bytes and str. Also, the encoding with the Windows ANSI 
encoding is lossy, so it may be that there is a loss in GetProperty().

--
components: Windows
messages: 372777
nosy: paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: msilib: SetProperty() accepts str, but GetProperty() returns bytes
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41158] IDLE: rewrite the code for handling file encoding



Change by miss-islington :


--
pull_requests: +20416
pull_request: https://github.com/python/cpython/pull/21267

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41189] An exploitable segmentation fault in _PyEval_EvalFrameDefault



New submission from Iman Sharafodin :

Python 3.6 (June 27, 2020) 
(https://www.python.org/ftp/python/3.6.11/Python-3.6.11.tgz).

I found an exploitable segmentation fault in Python 3.6.11 (I validated that by 
using GDB's Exploitable plugin). Please find the attachment.

#0  0x00b63bf4 in _PyEval_EvalFrameDefault (f=, 
throwflag=) at Python/ceval.c:3667
#1  0x00b5bc5b in PyEval_EvalFrameEx (throwflag=0, f=0x77f66c50) at 
Python/ceval.c:754
#2  _PyEval_EvalCodeWithName (_co=_co@entry=0x77ef5030, 
globals=globals@entry=0x77f62168, locals=locals@entry=0x77f62168, 
args=args@entry=0x0, argcount=argcount@entry=0, kwnames=kwnames@entry=0x0, 
kwargs=0x0, kwcount=0, kwstep=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, 
name=0x0, qualname=0x0) at Python/ceval.c:4166
#3  0x00b6100b in PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, 
defcount=0, defs=0x0, kwcount=0, kws=0x0, argcount=0, args=0x0, 
locals=locals@entry=0x77f62168, globals=globals@entry=0x77f62168, 
_co=_co@entry=0x77ef5030) at Python/ceval.c:4187
#4  PyEval_EvalCode (co=co@entry=0x77ef5030, 
globals=globals@entry=0x77f62168, locals=locals@entry=0x77f62168) at 
Python/ceval.c:731

--
files: ExploitableCrash.pyc
messages: 372776
nosy: Iman Sharafodin
priority: normal
severity: normal
status: open
title: An exploitable segmentation fault in _PyEval_EvalFrameDefault
versions: Python 3.6
Added file: https://bugs.python.org/file49285/ExploitableCrash.pyc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41188] Prepare CPython for opaque PyObject structure.



Change by William Pickard :


--
keywords: +patch
pull_requests: +20415
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21262

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41188] Prepare CPython for opaque PyObject structure.



New submission from William Pickard :

The goal of issue 39573 is to make "PyObject" and opaque structure in the 
limited API.

To do that, a few mandatory changes will be required to CPython in order to 
allow for seamless implementation.

Namely:
1) User types need to get away from directly referencing PyObject.
- This can be done by adding a new variable to "PyTypeObject" who's value 
is an offset into the object's pointer to the type's internal structure.
-- Example: 'PyType_Type.tp_obj_offset' would be the value of 
"sizeof(PyVarObject)".
-- For custom types with another base: The value would be calculated 
from the base's "tp_obj_offset" + "tp_basicsize"

2) Create a linkable static library to facility method calls requiring internal 
implementation.
- This static library will be implementation defined, IE: using internal 
methods specific to the runtime that created it.
- Public facing methods will use generic names for example, 
"PyObject_GetType" will get the object's ob_type (or whatever the target 
runtime calls it).

--
components: C API, Interpreter Core
messages: 372775
nosy: WildCard65, vstinner
priority: normal
severity: normal
status: open
title: Prepare CPython for opaque PyObject structure.
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41184] Reconciling IDLE's Comment Out Region / Uncomment Region with PEP 8 guidelines for commenting

2020-07-01 Thread Guido van Rossum



Guido van Rossum  added the comment:

The IDLE feature should not change, for all the reasons Terry have.

I am quite done with changes to PEP in order to settle arguments, and I do not 
believe this convention needs mentioning in that PEP. Remember there is text in 
PEP 8 reminding readers to use your best judgment.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41100] Build failure on macOS 11 (beta)

2020-07-01 Thread Lawrence D'Anna



Change by Lawrence D'Anna :


--
pull_requests: +20414
pull_request: https://github.com/python/cpython/pull/21266

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter



Change by STINNER Victor :


--
pull_requests: +20413
pull_request: https://github.com/python/cpython/pull/21265

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40512] [subinterpreters] Meta issue: per-interpreter GIL



STINNER Victor  added the comment:

> Update of the EXPERIMENTAL_ISOLATED_SUBINTERPRETERS status.

Also:

* _PyLong_Zero and _PyLong_One singletons are shared
* Py_None, Py_True and Py_False singletons are shared: bpo-39511 and PR 18301
* Static types like PyUnicode_Type and PyLong_Type are shared: see bpo-40077 
and bpo-40601
* The dictionary of Unicode interned strings is shared: PR 20085
* context.c: _token_missing singleton is shared
* "struct _PyArg_Parser" generated by Argument Clinic is shared: see 
_PyArg_Fini()

Misc notes:

* init_interp_main(): if sys.warnoptions is not empty, "import warnings" is 
called to process these options, but not in subinterpreters: only in the main 
intepreter.
* _PyImport_FixupExtensionObject() contains code specific to the main 
interpreter. Maybe this function will not longer be needed once builtin 
extension modules will be converted to PEP 489 "multiphase initialization" API. 
I'm not sure.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41184] Reconciling IDLE's Comment Out Region / Uncomment Region with PEP 8 guidelines for commenting

2020-07-01 Thread Terry J. Reedy

[issue41180] marshal load bypass code.new audit event

2020-07-01 Thread Steve Dower



Steve Dower  added the comment:

I like using the existing event for unmarshalling code objects, assuming we 
have all the arguments available.

I'm not sure whether it's worth auditing all marshal.load() calls (just as we 
don't audit all pickle.load() calls). But depending on the code paths we may 
not have a choice.

Auditing the .pyc import twice (or more) is basically unavoidable, but it's not 
terrible anyway. When no hooks are set, it's negligible impact, and when 
someone adds a hook they are in control of how much work it does. Providing 
both events means they can choose to only handle one and still get the coverage.

Marking this as easy (C), since the hardest part is just going to be finding 
the right place to add the PySys_Audit call.

--
keywords: +easy (C)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41007] test_importlib logs ResourceWarning: test_path.CommonTests.test_importing_module_as_side_effect()



STINNER Victor  added the comment:

Alright, this issue was fixed by:

commit 2fb5f038f2a2e91a7293d62dfd5601e6eb500c55 (HEAD)
Author: Jason R. Coombs 
Date:   Mon Jun 29 16:59:22 2020 -0400

bpo-40924: Ensure importlib.resources.path returns an extant path (GH-20857)

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41187] Convert the _msi module to Argument Clinic



Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +20412
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21264

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41187] Convert the _msi module to Argument Clinic



New submission from Serhiy Storchaka :

The proposed PR converts the _msi module to Argument Clinic.

* Fixes deprecation warnings with the "u" format.
* Adds signatures.
* Adds meaningful docstrings.

--
components: Argument Clinic, Windows
messages: 372769
nosy: larry, paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Convert the _msi module to Argument Clinic
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41186] distutils.version epoch compatibility

2020-07-01 Thread Bar Harel



Bar Harel  added the comment:

Actually, it looks like distutils days are quite over and pip itself uses 
setuptools with pypa packaging.

I'll take the initiative and close this.

--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41137] pdb uses the locale encoding for .pdbrc


Change by Srinivas  Reddy Thatiparthy(శ్రీనివాస్ రెడ్డి తాటిపర్తి) 
:


--
keywords: +patch
nosy: +thatiparthy
nosy_count: 2.0 -> 3.0
pull_requests: +20411
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21263

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41181] [macOS] Build macOS installer with LTO and PGO optimizations



STINNER Victor  added the comment:

If clang 6.0 is a dead end for LTO, another option is to build a recent clang 
version on macOS 10.9. If I manage to do that, would it sound like an 
acceptable solution? I don't expect any API/ABI issue just by changing the 
clang version. Upgrading clang should not change the semantics.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41181] [macOS] Build macOS installer with LTO and PGO optimizations



STINNER Victor  added the comment:

>> Clang 6.0 doesn't support LTO and PGO?
> No, it appears not.

That's really surprising. I see LTO mentioned in LLVM 3.4 changelog for example:
https://releases.llvm.org/3.4/tools/clang/docs/ReleaseNotes.html#new-compiler-flags

Did you try to build Python with my PR? Which error message do you get? How can 
I try? I only own a macbook which runs a recent macOS version. Maybe I could 
try to get clang 6.0 on Linux.

If PGO is not available, just enabling LTO should already make Python faster 
significantly.

I understand why Python is built on macOS 10.9, and this issue and my PR 
doesn't change anything about that. I don't request to require newer CPU 
features or to require newer macOS API or syscall. LTO only changes how Python 
itself is built.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41186] distutils.version epoch compatibility

2020-07-01 Thread Bar Harel



New submission from Bar Harel :

Is distutils.version aware of the PEP440 epoch version modifier? I haven't seen 
any reference to this.

AFAIK pypa packaging does take it into consideration, but should the stdlib 
also care about it?

I would like to believe pip takes it into consideration but not sure if it uses 
distutils, or that setuptools have their own versioning scheme together with 
pkg_resources.

--
components: Distutils
messages: 372765
nosy: bar.harel, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: distutils.version epoch compatibility
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41179] find_library on macOS Big Sur



Change by Ned Deily :


--
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41181] [macOS] Build macOS installer with LTO and PGO optimizations


Ned Deily  added the comment:

> Clang 6.0 doesn't support LTO and PGO?

No, it appears not.  And it's not an oversight that we don't use the these 
options.

As Łukasz points out, for the current macOS installer variants we supply are 
designed to run on all Mac systems from macOS 10.9 on. To accomplish that 
safely, we build the Python binaries on macOS 10.9 system to ensure they will 
be compatible, in other words, we build on the oldest system support and rely 
on upward compatibility when running on newer systems. The other approach is to 
build on the newest systems available after adding runtime checks throughout 
the C code to test for the presence of newer features (i.e. runtime calls that 
have been added in an operating system release newer than the oldest one 
support).  While this ("weaklinking") can be a viable option, it's a lot more 
work to implement initially and then keep updated over each o/s release to 
avoid segfaults and other failures when users on older systems try to use newer 
features.  Eventually we would like to fully support weaklinking so that we 
could provide one installer variant for all supported o/s versions that has all 
features available at each o/s version, it's not a high priority item at the 
moment (for example, supporting the upcoming 11.0 Big Sur with Apple Silicon 
is) and the current practices have worked well for many years.

Keep in mind that the main goal of the python.org macOS installers is to 
provide a single installable binary that works correctly on a wide-range of 
macOS releases and hardware.  What we provide today works on all Macs capable 
of running macOS 10.9 or later.  In particular, it is *not* a goal to provide 
the most optimized configuration for a particular system.  In general, consider 
the range of hardware and operating system releases, that's not easy to do. I 
believe that the intended users for the python.org macOS pythons are (1) 
beginners (like in a teaching environment where ease of deployment and 
uniformity is key) and (2) third-party Mac applications developers who want an 
embeddable Python that will allow their applications to work on multiple levels 
of macOS. If you are looking for the highest performance for a particular use, 
like benchmarking, you should look elsewhere - like one of the third-party 
distributors who specialize in numeric Pythons - or build it yourself on your 
own system.

So, thanks for the suggestion but we won't be using it now. Sometime in the 
future, if and when we support weaklinking and/or use newer toolchains across 
the board we will look at adding and other optimizations.

--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41106] os.scandir() Windows bug dir_entry.stat() not works on file during writing.

2020-07-01 Thread Eryk Sun

Eryk Sun added the comment:

> As far as I know os.stat() resets d.stat() maybe should be added
> some option to d.stat() to force update(). d.stat(nt_force_update=True).

It depends on the filesystem. NTFS will update the directory entry as soon as
the link is accessed by CreateFileW. But that's relatively expensive, and
actually one of the more expensive steps in an os.stat call.

> I am not sure if os.path.getmtime() can reset d.stat().

genericpath.getmtime calls os.stat:

https://github.com/python/cpython/blob/d0981e61a5869c48e0a70a512967558391272a93/Lib/genericpath.py#L53

lexists, exists, getctime, getatime, getmtime, getsize, isdir, and isfile could
be modified to call WinAPI GetFileAttributesExW [1], which is implemented via
NtQueryFullAttributesFile [2], an optimized system call to get a file's
network-open information. This can be significantly faster than the sequence of
system calls that are required by os.stat. Note that this does not update the
NTFS directory entry for the accessed link, unlike CreateFileW, but it does
return updated information.

The GetFileAttributesExW result would be used if the call succeeds and the file
isn't a reparse point. Otherwise fall back on os.stat (win32_xstat_impl). If
passed an fd, try GetFileInformationByHandleEx to get the FileBasicInfo and
FileStandardInfo, or use a single system call via NTAPI NtQueryInformationFile:
FileNetworkOpenInformation, which is the same info that GetFileAttributesExW
returns.

This could be implemented in C as nt._basic_stat(filename,
follow_symlinks=True), where follow_symlinks means the expanded set of Windows
name-surrogate reparse points. The C implementation would fall back on
win32_xstat_impl. Note that a basic stat would not guarantee to return the
following fields: st_ino, st_dev, and st_nlink.

Alternatively, it could be implemented as a keyword-only basic=True option for
os.stat, which would be ignored by POSIX. This way the high-level functions
could continue to have a common implementation in genericpath.py.

[1]
https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getfileattributesexw
[2]
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-zwqueryfullattributesfile

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41007] test_importlib logs ResourceWarning: test_path.CommonTests.test_importing_module_as_side_effect()


Srinivas  Reddy Thatiparthy(శ్రీనివాస్ రెడ్డి తాటిపర్తి) 
 added the comment:

(allhub) ➜  cpython git:(master) ✗ ./python.exe -X tracemalloc=20 -m test  
test_importlib -m 
test.test_importlib.test_path.CommonTests.test_importing_module_as_side_effect 
-v
== CPython 3.10.0a0 (heads/master:d0981e61a5, Jul 1 2020, 21:25:34) [Clang 
11.0.3 (clang-1103.0.32.62)]
== macOS-10.15.5-x86_64-i386-64bit little-endian
== cwd: /Users/srini/workspace/consulting/cpython/build/test_python_1601æ
== CPU count: 12
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 2.06 Run tests sequentially
0:00:00 load avg: 2.06 [1/1] test_importlib
test_importing_module_as_side_effect 
(test.test_importlib.test_path.CommonTests) ... ok

--

Ran 1 test in 0.061s

OK

== Tests result: SUCCESS ==

1 test OK.

Total duration: 6.5 sec
Tests result: SUCCESS

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41007] test_importlib logs ResourceWarning: test_path.CommonTests.test_importing_module_as_side_effect()


Srinivas  Reddy Thatiparthy(శ్రీనివాస్ రెడ్డి తాటిపర్తి) 
 added the comment:

I can repro on  843c27765652e2322011fb3e5d88f4837de38c06  but I have tried this 
on latest master d0981e6 and I do not see the warning. 

can be closed?

--
nosy: +thatiparthy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41185] lib2to3 generation of pickle files is racy

2020-07-01 Thread Marco Barisione



New submission from Marco Barisione :

The generation of pickle files in load_grammar in lib2to3/pgen2/driver.py is 
racy as other processes may end up reading a half-written pickle file.

This is reproducible with the command line tool, but it's easier to reproduce 
by importing lib2to3. You just need different processes importing lib2to3 at 
the same time to make this happen, see the attached reproducer.

I tried with Python 3.9 for completeness and, while it happens there as well, 
it seems to be less frequent ony my computer than when using Python 3.6 (2% 
failure rate instead of 50% failure rate).

--
components: 2to3 (2.x to 3.x conversion tool)
files: pool.py
messages: 372760
nosy: barisione
priority: normal
severity: normal
status: open
title: lib2to3 generation of pickle files is racy
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49284/pool.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39573] [C API] Make PyObject an opaque structure in the limited C API



Change by William Pickard :


--
nosy: +WildCard65
nosy_count: 9.0 -> 10.0
pull_requests: +20410
pull_request: https://github.com/python/cpython/pull/21262

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41066] Update documentation for Pathlib


Change by Srinivas  Reddy Thatiparthy(శ్రీనివాస్ రెడ్డి తాటిపర్తి) 
:


--
keywords: +patch
pull_requests: +20409
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21261

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41158] IDLE: rewrite the code for handling file encoding



miss-islington  added the comment:


New changeset fe0175f5b50967aae90cb759a0d2d016fb745584 by Miss Islington (bot) 
in branch '3.9':
bpo-41158: IDLE: rewrite the code for handling file encoding (GH-21215)
https://github.com/python/cpython/commit/fe0175f5b50967aae90cb759a0d2d016fb745584


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41181] [macOS] Build macOS installer with LTO and PGO optimizations



STINNER Victor  added the comment:

> We cannot depend on PGO and LTO for it unless we start building the installer 
> on 10.15.

Clang 6.0 doesn't support LTO and PGO? Would you mind to elaborate?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41184] Reconciling IDLE's Comment Out Region / Uncomment Region with PEP 8 guidelines for commenting

2020-07-01 Thread Anthropologist



New submission from Anthropologist :

IDLE's Comment Out Region formatting tool currently adds two octothorpe 
characters followed by no spaces.  The Uncomment Region tool removes up to 2 
octothorpes but does not remove spaces. 

The Python Style Guide (PEP 8) specifies that: "An inline comment is a comment 
on the same line as a statement. Inline comments should be separated by at 
least two spaces from the statement. They should start with a # and a single 
space."

I propose reconciling these conflicting approaches to commenting out code, 
either by changing IDLE's behavior (i.e., instead of ##, Comment Out Region 
should insert # followed by a space, per PEP 8), or by amending the Python 
Style Guide to provide clear instructions about commenting out code, which is 
arguably a different use of comments than a single-line comment for the sake of 
explaining code. 

If the resolution involves changing IDLE's behavior, the Uncomment Region 
feature should also be changed to remove the space character after the 
octothorpe. As it currently stands, this feature fails to uncomment code that 
has been manually commented out following the guidelines in PEP 8.

--
assignee: docs@python
components: Documentation, IDLE
messages: 372757
nosy: anthropologist, docs@python, terry.reedy
priority: normal
severity: normal
status: open
title: Reconciling IDLE's Comment Out Region / Uncomment Region with PEP 8 
guidelines for commenting
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41158] IDLE: rewrite the code for handling file encoding



Change by miss-islington :


--
pull_requests: +20408
pull_request: https://github.com/python/cpython/pull/21260

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41158] IDLE: rewrite the code for handling file encoding



Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 2.0 -> 3.0
pull_requests: +20407
pull_request: https://github.com/python/cpython/pull/21259

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41183] Workaround or fix for SSL "EE_KEY_TOO_SMALL" test failures



Change by Christian Heimes :


--
keywords: +patch
pull_requests: +20405
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/21258

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34542] [TLS] Update test certs to future proof settings



Change by Christian Heimes :


--
pull_requests: +20406
pull_request: https://github.com/python/cpython/pull/21258

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41177] ConvertingList and ConvertingTuple lack iterators and ConvertingDict lacks items()

2020-07-01 Thread Brett Hannigan



Change by Brett Hannigan :


--
versions:  -Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41183] Workaround or fix for SSL "EE_KEY_TOO_SMALL" test failures



Christian Heimes  added the comment:

I'll look into it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41183] Workaround or fix for SSL "EE_KEY_TOO_SMALL" test failures

2020-07-01 Thread Larry Hastings



New submission from Larry Hastings :

I'm testing 3.5.10rc1 on a freshly installed Linux (Pop!_OS 20.04), and I'm 
getting a lot of these test failures:

ssl.SSLError: [SSL: EE_KEY_TOO_SMALL] ee key too small (_ssl.c:2951)

Apparently the 2048 keys used in the tests are considered "too small" with 
brand-new builds of the SSL library.

Christian: you upgraded the test suite keys to 3072 bits back in 2018 (issue 
#34542), but didn't backport this as far as 3.5 because it was in 
security-fixes-only mode.  I experimented with taking your patch to 3.6 and 
applying it to 3.5, but 80% of the patches didn't apply cleanly.  Could you 
either backport this upgrade to 3.5 (I'll happily accept the PR), or advise me 
on how to otherwise mitigate the problem?  I don't really want to turn off all 
those tests.  Thanks!

--
assignee: christian.heimes
components: Tests
messages: 372755
nosy: christian.heimes, larry
priority: high
severity: normal
stage: needs patch
status: open
title: Workaround or fix for SSL "EE_KEY_TOO_SMALL" test failures
type: crash
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41179] find_library on macOS Big Sur

2020-07-01 Thread Sumanth Ratna



Sumanth Ratna  added the comment:

I see; thanks for letting me know! I saw Issue41100 but I had assumed that it 
was only for build issues :)

Should I close this issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41169] socket.inet_pton raised when pass an IPv6 address like "[::]" to it

2020-07-01 Thread Wator Sead



Wator Sead  added the comment:

Can you try bind "::".

My ask is "Can make a consistent behave via constraints it?".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41170] Use strnlen instead of strlen when the size i known.

2020-07-01 Thread Larry Hastings



Larry Hastings  added the comment:

strnlen() isn't standard C, but an exciting new function strnlen_s() is, as of 
C11.

https://en.cppreference.com/w/c/string/byte/strlen

(At this rate, we should be able to code CPython using that standard in about 
2030.)

But!  I found a 2005 thread on /. talking about strnlen in MSVC.  So maybe it's 
there.  Though Microsoft has this funny habit of putting an underscore in front 
of C library functions that aren't standard, so maybe it's _strnlen().

--
nosy: +larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41169] socket.inet_pton raised when pass an IPv6 address like "[::]" to it



Christian Heimes  added the comment:

The behavior of sockets depends on platform and implementation details of OS 
and libc. Binding to ('[::]', 888) does not work for me on Linux. It might work 
on other problems. The majority of functions and methods in the socket module 
are thin wrappers around low-level OS features. 

>>> import socket
>>> import sys
>>> sys.version
'3.8.3 (default, May 29 2020, 00:00:00) \n[GCC 10.1.1 20200507 (Red Hat 
10.1.1-1)]'
>>> sys.platform
'linux'
>>> s = socket.socket(socket.AF_INET6)
>>> s.bind(('[::]', 888))
Traceback (most recent call last):
  File "", line 1, in 
socket.gaierror: [Errno -2] Name or service not known

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41169] socket.inet_pton raised when pass an IPv6 address like "[::]" to it

2020-07-01 Thread Wator Sead



Wator Sead  added the comment:

[::] can be bound, but the resoult is [::1], you must use this address to 
connect. Excuse me, are you a developer of the Python?

>>> import socket
>>> ls = socket.socket(socket.AF_INET6)
>>> cs = socket.socket(socket.AF_INET6)
>>> ls.bind(('[::]', 888))  # no raise
>>> ls.listen(1)
>>> cs.connect(('[::1]', 888))  # no raise

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41176] revise Tkinter mainloop dispatching flag behavior

2020-07-01 Thread Richard Sheridan



Richard Sheridan  added the comment:

Removing `WaitForMainloop` would surely break some existing programs, but
that's why I suggested deprecation instead of just removing it suddenly. We
could issue a RuntimeWarning if `WaitForMainloop` actually waits and tell
the client to take responsibility for the race condition they created.
(They may have no idea! What if their delay unexpectedly increases to 1.2
seconds?) Whether or not waiting gets deprecated, it would make sense to
make the sleep behavior configurable instead of hardcoded. I'll include
something along those lines in my PR.

On Wed, Jul 1, 2020 at 6:15 AM E. Paine  wrote:

>
> E. Paine  added the comment:
>
> I agree it would be helpful to expose an explicit way of telling if the
> mainloop was running but am not sure about removing `WaitForMainloop` as it
> could very easily break existing programs.
>
> If a program executes a tkinter method in a thread before the mainloop is
> executed, the method will wait because of the call to `WaitForMainloop`. In
> the example script this is done deliberately to demonstrate the behaviour
> but could be done accidentally if the main thread has to do something else
> before the mainloop (and after the thread has been created).
>
> I think the changes (whatever is concluded we should do) would be
> considered an 'enhancement', which would not be backported to 3.9 and
> before (I believe 'behaviour' is generally used for logic errors).
>
> I am very willing to help review a PR, however the people you really need
> to convince are Serhiy and/or Guilherme (I have added them to the nosy).
>
> --
> nosy: +epaine, gpolo, serhiy.storchaka
> versions:  -Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9
> Added file: https://bugs.python.org/file49283/waitmainloop.py
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41153] [easy Doc] "PyPreConfig_InitIsolatedConfig" and "PyPreConfig_InitPythonConfig" are given the opposite's documentation.



Change by William Pickard :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41106] os.scandir() Windows bug dir_entry.stat() not works on file during writing.

2020-07-01 Thread Cezary Wagner



Cezary Wagner  added the comment:

As far as I know os.stat() resets d.stat() maybe should be added some option to 
d.stat() to force update(). d.stat(nt_force_update=True).

I am not sure if os.path.getmtime() can reset d.stat().

os.stat() is 2x times slower than os.path.getmtime() and os.path.getmtime is 
16x slower than d.stat(). MAJOR PROBLEM is PERFORMANCE of os.stat() since for 
directories with 1000 files it takes big number of seconds to read all stats - 
something wrong is here I think since Windows Explorer is doing it very fast.

So I can not use os.stat() ONLY and it complicates code since I need to use 
os.stat() after d.stat() if files is OLDER THAN because if I use os.stat() the 
most program time will be these calls.

Do you know which code makes such reset of d.stat()?

If there is not possible optimization of there is need DOCUMENTATION update 
because it is really hard to understand why it is not working under windows 
some REMARKS can help me and others.

I have still believe that some optimization is possible for Windows.

Maybe it can be force to read stat by os.scandir(force_scan_stat=True) so all 
directory entries will be have cached stats before d.stat() is called. It can 
be faster I think since less calls from Python and probably better Windows API 
for it and same for Linux.

I will study C code later if it is possible or write some snippet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41181] [macOS] Build macOS installer with LTO and PGO optimizations

2020-07-01 Thread Łukasz Langa


Łukasz Langa  added the comment:

The installer is built on Mac OS X 10.9 so that it is forward compatible with 
all OS X and macOS versions. We cannot depend on PGO and LTO for it unless we 
start building the installer on 10.15. We cannot do this currently as those 
installers would not work with older macOS and OS X versions.

Since the Mac is switching to Apple Silicon, the plan is to start building a 
separate macOS 11+ installer. *That* could use PGO and LTO.

--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41182] DefaultSelector fails to detect selector on VMware ESXi

2020-07-01 Thread Abhijeet Kasurde



Change by Abhijeet Kasurde :


--
keywords: +patch
pull_requests: +20403
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21257

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41182] DefaultSelector fails to detect selector on VMware ESXi

2020-07-01 Thread Abhijeet Kasurde



New submission from Abhijeet Kasurde :

When DefaultSelector is used on VMware ESXi, it fails with 

>>> import selectors
>>> selector = selectors.DefaultSelector()
Traceback (most recent call last):
  File "", line 1, in 
  File 
"/build/mts/release/bora-4887370/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/selectors.py",
 line 390, in __init__
OSError: [Errno 38] Function not implemented

After debugging, I found that it is using Selector which is not implemented for 
ESXi kernel. 

Change DefaultSelector mechanism to use 'select' implementation to choose 
default selector.

--
components: Library (Lib)
messages: 372746
nosy: akasurde
priority: normal
severity: normal
status: open
title: DefaultSelector fails to detect selector on VMware ESXi
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41181] [macOS] Build macOS installer with LTO and PGO optimizations



Change by STINNER Victor :


--
components: +macOS
nosy: +inada.naoki, ned.deily, rhettinger, ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41181] [macOS] Build macOS installer with LTO and PGO optimizations



Change by STINNER Victor :


--
keywords: +patch
pull_requests: +20402
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21256

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36839] Support the buffer protocol in code objects

2020-07-01 Thread Stefan Krah



Stefan Krah  added the comment:

For clarification, the incident I referred to is entirely unrelated to *this* 
issue:  Of course Dino Viehland has been rational, friendly and competent.

I wanted to point out that people might have had a formative experience 
*elsewhere*.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41181] [macOS] Build macOS installer with LTO and PGO optimizations



STINNER Victor  added the comment:

The performance issue was noticed by Raymond Hettinger who ran a microbenchmark 
on tuplegetter_descr_get(), comparison between Python 3.8 and Python 3.9:

https://mail.python.org/archives/list/python-...@python.org/message/Q3YHYIKNUQH34FDEJRSLUP2MTYELFWY3/

INADA-san confirms that the performance regression was introduced by the commit 
45ec5b99aefa54552947049086e87ec01bc2fc9a (bpo-40170) which changes 
PyType_HasFeature() implementation to always call PyType_GetFlags() as a 
function rather than reading directly the PyTypeObject.tp_flags member.

https://mail.python.org/archives/list/python-...@python.org/message/FOKJXG2SYMXCHYPGUZWVYMHLDR42BYFB/


On Fedora 32, there is no performance difference because binaries are built 
with GCC using LTO and PGO: the PyType_GetFlags() function call is inlined by 
GCC 10.


I built Python on macOS with clang 11.0.3 on macOS 10.15.4, and I confirm that 
LTO+PGO allows to inline the PyType_GetFlags() function call in 
tuplegetter_descr_get().

Using "./configure && make":
---
$ lldb ./python.exe
(lldb) disassemble --name tuplegetter_descr_get
(...)
python.exe[0x1001c46ad] <+29>:  callq  0x10009c720   ; 
PyType_GetFlags at typeobject.c:2338
python.exe[0x1001c46b2] <+34>:  testl  $0x400, %eax  ; imm = 
0x400 
(...)
---

Using "./configure --with-lto --enable-optimizations && make":
---
$ lldb ./python.exe
(lldb) disassemble --name tuplegetter_descr_get
(...)
python.exe[0x1002a9542] <+18>:  movq   0x10(%rbx), %rdx
python.exe[0x1002a9546] <+22>:  movq   0x8(%rsi), %rax
python.exe[0x1002a954a] <+26>:  testb  $0x4, 0xab(%rax)
python.exe[0x1002a9551] <+33>:  je 0x1002a956f   ; <+63>
(...)
---

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41181] [macOS] Build macOS installer with LTO and PGO optimizations



New submission from STINNER Victor :

Link Time Optimization (LTO) and Profile-Guided Optimization (PGO) have a major 
impact on Python performance: they make Python between 10% and 30% faster 
(coarse estimation).

Currently, macOS installers distributed on python.org are built with Clang 6.0 
without LTO or PGO. I propose to enable LTO and PGO to make these binaries 
faster.

IMO we should build all new Python macOS installers with these optimizations.

Attached PR adds the flags.


Python 3.9.0b3 binary:

$ python3.9
Python 3.9.0b3 (v3.9.0b3:b484871ba7, Jun  9 2020, 16:05:25) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

configure options:

>>> import sysconfig; print(sysconfig.get_config_var('CONFIG_ARGS'))
'-C' '--enable-framework' '--enable-universalsdk=/' 
'--with-universal-archs=intel-64' '--with-computed-gotos' '--without-ensurepip' 
'--with-tcltk-includes=-I/tmp/_py/libraries/usr/local/include' 
'--with-tcltk-libs=-ltcl8.6 -ltk8.6' 'LDFLAGS=-g' 'CFLAGS=-g' 'CC=gcc'

Compiler flags:

>>> sysconfig.get_config_var('PY_CFLAGS') + 
>>> sysconfig.get_config_var('PY_CFLAGS_NODIST')
'-Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic 
-DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -g-std=c99 -Wextra 
-Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers 
-Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  
-I/Users/sysadmin/build/v3.9.0b3/Include/internal'

Linker flags:

>>> sysconfig.get_config_var('PY_LDFLAGS') + 
>>> sysconfig.get_config_var('PY_LDFLAGS_NODIST')
'-arch x86_64 -g'

--
components: Build
messages: 372743
nosy: vstinner
priority: normal
severity: normal
status: open
title: [macOS] Build macOS installer with LTO and PGO optimizations
type: performance
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37495] socket.inet_aton parsing issue on some libc versions

2020-07-01 Thread disconnect3d



disconnect3d  added the comment:

Regarding the exception case: seems like OSError, since that's what originally 
was done and we don't want to break users of this code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41176] revise Tkinter mainloop dispatching flag behavior

2020-07-01 Thread E. Paine



E. Paine  added the comment:

I agree it would be helpful to expose an explicit way of telling if the 
mainloop was running but am not sure about removing `WaitForMainloop` as it 
could very easily break existing programs.

If a program executes a tkinter method in a thread before the mainloop is 
executed, the method will wait because of the call to `WaitForMainloop`. In the 
example script this is done deliberately to demonstrate the behaviour but could 
be done accidentally if the main thread has to do something else before the 
mainloop (and after the thread has been created).

I think the changes (whatever is concluded we should do) would be considered an 
'enhancement', which would not be backported to 3.9 and before (I believe 
'behaviour' is generally used for logic errors).

I am very willing to help review a PR, however the people you really need to 
convince are Serhiy and/or Guilherme (I have added them to the nosy).

--
nosy: +epaine, gpolo, serhiy.storchaka
versions:  -Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49283/waitmainloop.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37495] socket.inet_aton parsing issue on some libc versions

2020-07-01 Thread disconnect3d



disconnect3d  added the comment:

Its a while since this has been reported. I think inet_aton_pton.py is fine, 
though, a commit adding it should explain why we do it this way.

@vstinner can you push this patch further or do you want me to do it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36839] Support the buffer protocol in code objects