[issue36540] PEP 570: Python Positional-Only Parameters

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36535] Windows build failure when use the code from the GitHub master branch

[Manjusaka]

Change by Manjusaka :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36535] Windows build failure when use the code from the GitHub master branch

[Manjusaka]

Manjusaka  added the comment:

Done,thx

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36521] Consider removing docstrings from co_consts in code objects

[Inada Naoki]

Inada Naoki  added the comment:

There is idea about reading docstring lazily, when func.__doc__ is accessed.

I don't think the idea can be implemented by 3.8.  But if we change code object 
now, I want new API can be used to implement this idea.

One breaking change is better than two.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar more closely match Python

[Tim Hatch]

Change by Tim Hatch :


--
pull_requests: +12627

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar more closely match Python

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +12626
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar more closely match Python

[Tim Hatch]

New submission from Tim Hatch :

The grammar in lib2to3 is out of date and can't parse `:=` nor `f(**not x)` 
from running on real code.  I've done a cursory `diff -uw Grammar/Grammar 
Lib/lib2to3/grammar.txt`, and would like to fix lib2to3 so we can merge into 
both fissix and blib2to3, to avoid further divergence of the forks.

I'm unsure if I need a separate bug per pull request, but need at least one to 
get started.

--
components: 2to3 (2.x to 3.x conversion tool)
messages: 339522
nosy: georg.brandl, lukasz.langa, thatch
priority: normal
severity: normal
status: open
title: Make lib2to3 grammar more closely match Python
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36540] PEP 570: Python Positional-Only Parameters

[Eric N. Vander Weele]

Change by Eric N. Vander Weele :


--
nosy: +ericvw

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36540] PEP 570: Python Positional-Only Parameters

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +12625
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36540] PEP 570: Python Positional-Only Parameters

[Pablo Galindo Salgado]

New submission from Pablo Galindo Salgado :

This issue will serve to track development and PRs for the implementation of 
PEP 570: Python Positional-Only Parameters.

--
assignee: pablogsal
components: Interpreter Core
messages: 339521
nosy: pablogsal
priority: normal
severity: normal
status: open
title: PEP 570: Python Positional-Only Parameters
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32280] Expose `_PyRuntime` through a section name

[Eric Snow]

Change by Eric Snow :


--
versions: +Python 3.8 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36539] Distutils VC 6.0 Errors When Using mingw-w64 GCC

[Dan Yeaw]

New submission from Dan Yeaw :

I am using the mingw-w64-x86_64-python3 in MSYS2 on Windows to package a 
PyGObject app. When I try to pip install the app, I am getting errors about the 
VC 6.0 isn't supported.

It looks like setuptools is trying to patch distutils msvc. The msvc9compiler 
module in distutils uses a get_build_version function to get the version of 
MSVC on the system. This version of Python 3 is compiled with GCC 8.3.0 so the 
function doesn't find the "MSC v." prefix and returns version 6.

Here is the full error:

$ pip install -e .
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... error
  Complete output from command C:/tools/msys64/mingw64/bin/python3.exe 
C:/tools/msys64/mingw64/lib/python3.7/site-packages\pip install 
--ignore-installed --no-user --prefix 
C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-bfwge_92/normal 
--no-warn-script-location --no-binary :none: --only-binary :none: -i 
https://pypi.org/simple -- gaphas>=1.0.0,<2.0.0 PyGObject>=3.30,<4.0 
pycairo>=1.18,<2.0 zope.component>=4.5,<5.0:
  Collecting gaphas<2.0.0,>=1.0.0
Using cached 
https://files.pythonhosted.org/packages/68/1d/4c8501535889538fe2144b5b8836fa2b3296e06d4a3d9f7e4e7e8cc1e90f/gaphas-1.0.0-py2.py3-none-any.whl
  Collecting PyGObject<4.0,>=3.30
Using cached 
https://files.pythonhosted.org/packages/0b/fd/56ac6898afc5c7f5718026103bd8f0b44714b6f79ac20d7eb8990c9a7eab/PyGObject-3.32.0.tar.gz
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
Complete output from command C:/tools/msys64/mingw64/bin/python3.exe 
C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py 
get_requires_for_build_wheel C:/Users/dyeaw/AppData/Local/Temp/tmp8lre6w7i:
Traceback (most recent call last):
  File 
"C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", 
line 207, in 
main()
  File 
"C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", 
line 197, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
  File 
"C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", 
line 48, in get_requires_for_build_wheel
backend = _build_backend()
  File 
"C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", 
line 34, in _build_backend
obj = import_module(mod_path)
  File "C:/tools/msys64/mingw64/lib/python3.7\importlib\__init__.py", line 
127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "", line 1006, in _gcd_import
  File "", line 983, in _find_and_load
  File "", line 953, in _find_and_load_unlocked
  File "", line 219, in 
_call_with_frames_removed
  File "", line 1006, in _gcd_import
  File "", line 983, in _find_and_load
  File "", line 967, in _find_and_load_unlocked
  File "", line 677, in _load_unlocked
  File "", line 728, in exec_module
  File "", line 219, in 
_call_with_frames_removed
  File 
"C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-5pamh2nb/overlay/lib/python3.7/site-packages\setuptools\__init__.py",
 line 228, in 
monkey.patch_all()
  File 
"C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-5pamh2nb/overlay/lib/python3.7/site-packages\setuptools\monkey.py",
 line 101, in patch_all
patch_for_msvc_specialized_compiler()
  File 
"C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-5pamh2nb/overlay/lib/python3.7/site-packages\setuptools\monkey.py",
 line 164, in patch_for_msvc_specialized_compiler
patch_func(*msvc9('find_vcvarsall'))
  File 
"C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-5pamh2nb/overlay/lib/python3.7/site-packages\setuptools\monkey.py",
 line 151, in patch_params
mod = import_module(mod_name)
  File "C:/tools/msys64/mingw64/lib/python3.7\importlib\__init__.py", line 
127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "", line 1006, in _gcd_import
  File "", line 983, in _find_and_load
  File "", line 967, in _find_and_load_unlocked
  File "", line 677, in _load_unlocked
  File "", line 728, in exec_module
  File "", line 219, in 
_call_with_frames_removed
  File "C:/tools/msys64/mingw64/lib/python3.7\distutils\msvc9compiler.py", 
line 296, in 
raise DistutilsPlatformError("VC %0.1f is not supported by this module" 
% VERSION)
distutils.errors.DistutilsPlatformError: VC 6.0 is not supported by this 
module

Others are getting this error as well: 
https://stackoverflow.com/questions/52166914/msys2-mingw64-pip-vc-6-0-is-not-supported-by-this-module

One way to fix may be to not raise the error if it occurs:
VERSION = get_build_version()
if VERSION < 8.0:
-raise 

[issue35983] tp_dealloc trashcan shouldn't be called for subclasses

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

I realized that there is a nasty interaction between the trashcan and __del__: 
if you're very close to the trashcan limit and you're calling __del__, then 
objects that should have been deallocated in __del__ (in particular, an object 
involving self) might instead end up in the trashcan. So self might be 
resurrected when it shouldn't be.

This is causing test failures in the 2.7 backport due to a different 
implementation of __del__.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36538] _thread.interrupt_main() no longer interrupts Lock.wait

[Gregory P. Smith]

New submission from Gregory P. Smith :

In Python 2.7 our threading implementation was so poor that a thread join 
ultimately called our lock wait implementation that busy looped polling and 
sleeping to check for a lock acquisition success.  calling 
thread.interrupt_main() which is just PyErr_SetInterrupt() C API in disguise 
successfully broke out of that lock wait loop.

In Python 3 with our drastically improved threading implementation, a lock wait 
is a pthreads sem_timedwait() or sem_trywait() API call, blocking within the 
pthreads library or OS kernel.  PyErr_SetInterrupt() obviously has no effect on 
that.  Only an actual signal arriving can interrupt that.

Thus instead of code using _thread.interrupt_main() - in 2and3 compatible 
applications, six.moves._thread.interrupt_main() - they should instead write:

 os.kill(os.getpid(), signal.SIGINT)

Given that _thread is a private module making _thread.interrupt_main() a 
private API, do we need to keep it?  If we do, we should at least document this 
behavior and recommend actually sending the signal.  It is less capable of 
actually interrupting the main thread from some common blocking operations 
today.  Sending the signal seems like it would always be better.

--
assignee: docs@python
components: Documentation, Extension Modules, Library (Lib)
messages: 339518
nosy: docs@python, gregory.p.smith
priority: normal
severity: normal
stage: needs patch
status: open
title: _thread.interrupt_main() no longer interrupts Lock.wait
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36495] Out-of-bounds array reads in Python/ast.c

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36529] Python from WindowsStore: can't install package using "-m pip"

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36537] except statement block incorrectly assumes end of scope(?).

[SilentGhost]

SilentGhost  added the comment:

Can confirm also on 3.6, but this seems to be something particular to 
set_trace, when pdb is entered via post_mortem, the variable is available in 
locals().

--
nosy: +SilentGhost, barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36523] Add docstring to io.IOBase.writelines

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I assume that this can be backported even though it touches .c rather than .rst.

--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36523] Add docstring to io.IOBase.writelines

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Verified.  
>>> import io
>>> io.IOBase.writelines.__doc__
>>>

--
nosy: +terry.reedy
title: missing docs for IOBase writelines -> Add docstring to 
io.IOBase.writelines

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36521] Consider removing docstrings from co_consts in code objects

[Terry J. Reedy]

Terry J. Reedy  added the comment:

So we have the same issue with f.__name__ and f.__code__.co_name becoming 
unsynchronized.

FWIW, I would prefer that the code docstring be co_doc, rather than hidden in 
co_constants, so that 'name' and 'doc' follow the same pattern.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36532] Example of logging.formatter with new str.format style

[Fred L. Drake, Jr.]

Fred L. Drake, Jr.  added the comment:

Good catch, Vinay!  Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36535] Windows build failure when use the code from the GitHub master branch

[Steve Dower]

Steve Dower  added the comment:

We added a new dependency.

Run a build using PCbuild/build.bat and it will download it for you. Then 
building through VS should work again.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36488] os.sendfile() on BSD, macOS don't return bytes sent on EINTR

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

sendfile() on BSD/OSX is complicated by the headers/trailers args. You'll have 
to take that into account in the retry logic, adding unnecessary complexity. 
Since sendfile() may already return fewer bytes than requested (e.g. 
non-blocking sockets or big files) it's just easier to return the bytes sent 
thus far (if any). I can work on a patch once I find some time.

> Wasn't the point of PEP475 that all EINTR returns would be explicitly handled 
> by retrying rather than forcing the user to handle it?

>From PEP475: <<[...] to relieve application code from the burden of doing so>>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36537] except statement block incorrectly assumes end of scope(?).

[Saim Raza]

New submission from Saim Raza :

If pdb.set_trace() is the last statement in the first code snippet, variable 
'err' is (wrongly?) excluded from locals(). Adding any code after the 
pdb.set_trace() statement makes 'err' available in locals.

In [2]: try:
   ...: raise ValueError("I am ValueError")
   ...: except ValueError as err:
   ...: print("err" in locals())
   ...: import pdb; pdb.set_trace()
   ...:
True
--Return--
> (5)()->None
-> import pdb; pdb.set_trace()
(Pdb) print("err" in locals())
False  <-- BUG??
(Pdb) c

In [3]: try:
   ...: raise ValueError("I am ValueError")
   ...: except ValueError as err:
   ...: print("err" in locals())
   ...: import pdb; pdb.set_trace()
   ...: import os # Dummy code - makes variable 'err' available inside the 
debugger.
   ...:
True
> (6)()->None
-> import os # Dummy code - makes variable err available inside debugger
(Pdb) print("err" in locals())
True

In [4]: sys.version_info
Out[4]: sys.version_info(major=3, minor=7, micro=3, releaselevel='final', 
serial=0)

FTR, the variable 'err' is available in both cases in case of Python 2.7. Also, 
this happens with ipdb as well.

Please note that I am aware that I need to assign the variable 'err' to some 
variable inside the except block to access it outside the except block. 
However, the pdb statement is still inside the except block in both cases.

--
components: Library (Lib)
messages: 339510
nosy: Saim Raza
priority: normal
severity: normal
status: open
title: except statement block incorrectly assumes end of scope(?).
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36502] str.isspace() for U+00A0 and U+202F differs from document

[Terry J. Reedy]

Change by Terry J. Reedy :


--
title: The behavior of str.isspace() for U+00A0 and U+202F is different from 
what is documented -> str.isspace() for U+00A0 and U+202F differs from document
versions:  -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36490] Modernize function signature in Archiving section of shutil doc

[Terry J. Reedy]

Change by Terry J. Reedy :


--
stage:  -> needs patch
title: Modernize function signature format in Archiving section of shutil doc 
-> Modernize function signature  in Archiving section of shutil doc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36488] os.sendfile() on BSD, macOS don't return bytes sent on EINTR

[Terry J. Reedy]

Change by Terry J. Reedy :


--
title: os.sendfile() on BSD and macOS does not return bytes sent on EINTR -> 
os.sendfile() on BSD, macOS don't return bytes sent on EINTR
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36486] Bugs and inconsistencies in unicodedata

[Terry J. Reedy]

Change by Terry J. Reedy :


--
stage:  -> needs patch
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36484] Can't reorder TLS 1.3 ciphersuites

[Terry J. Reedy]

Change by Terry J. Reedy :


--
type:  -> enhancement
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36487] Make C-API docs clear about what the "main" interpreter is

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:

Thanks for this feedback. Let me see how I can incorporate this in the PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36487] Make C-API docs clear about what the "main" interpreter is

[Eric Snow]

Eric Snow  added the comment:

I should have added something like this earlier, but here are key points to 
consider covering:

* "main" interpreter is the original, created when the runtime initializes
* historically almost always the only Python interpreter in a process
* this is changing (more projects using subinterpreters, PEP 554)
* in the "python" executable, subinterpreters available via extension 
modules (but uncommon)
* it has extra responsiblities:
* signal handling, in main thread
* "pending calls", in main thread
* not necessarily something to publicize :)
* running in the "main" thread
* can be taken over by a sub-interpreter, but not likely
* execution *during* runtime initialization
* usually (always?) the active interpreter during runtime finalization
* others?
* no strong link between interpreters (e.g. main <--> sub <--> sub)
* no record of what interpreter was active (if any) when a subinterpreter 
was created
* no "parent" interpreter
* thread states may share thread ID
* a bunch of stuff in the C-API and in the runtime still assumes that the main 
interpreter is running in the current OS thread
* other stuff?


Not all of that necessarily belongs there in the docs, but a lot of it does. :)

--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

[Hugh Redelmeier]

Change by Hugh Redelmeier :


--
nosy: +hugh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6721] Locks in the standard library should be sanitized on fork

[Hugh Redelmeier]

Change by Hugh Redelmeier :


--
nosy: +hugh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36525] Deprecate instancemethod

[Christian Heimes]

Christian Heimes  added the comment:

After some discussion we came to the conclusion that the API is still useful 
for some people. 
https://mail.python.org/pipermail/python-dev/2019-April/156986.html

--
components: +Interpreter Core
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

[Ned Deily]

Change by Ned Deily :


--
Removed message: https://bugs.python.org/msg339506

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

[Ebizz Infotech]

Ebizz Infotech  added the comment:

The best-recommended method is hiring php web development services.

https://www.ebizzinfotech.com/php-web-development/

--
nosy: +ebizzinfotech

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

[cagney]

cagney  added the comment:

On Fri, 5 Apr 2019 at 05:02, Gregory P. Smith  wrote:
>
>
> Gregory P. Smith  added the comment:
>
> A stdlib alternative to this whole mess would be to avoid acquiring the 
> logging locks before fork() as we currently do and just blindly re-initialize 
> all of them afterwards under the assumption that they "can't" be protecting 
> anything in a newly forked child process.  Handlers that need specific 
> resource synchronization around fork would then be required to deal with 
> their own os.register_at_fork() calls.  (ex: to avoid multiple processes 
> writing to the same file or fd at once)

FYI, below is a simpler, but related, test.
_at_fork_acquire_release_weakset doesn't seem to be locked:

Ignoring exception from logging atfork  release
method: cannot release un-acquired lock

Exception ignored in: 
Traceback (most recent call last):
  File "/home/python/v3.7.3/lib/python3.7/logging/__init__.py", line
269, in _after_at_fork_weak_calls
_at_fork_weak_calls('release')
  File "/home/python/v3.7.3/lib/python3.7/logging/__init__.py", line
254, in _at_fork_weak_calls
for instance in _at_fork_acquire_release_weakset:
  File "/home/python/v3.7.3/lib/python3.7/_weakrefset.py", line 60, in __iter__
for itemref in self.data:
RuntimeError: Set changed size during iteration

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/python/v3.7.3/lib/python3.7/threading.py", line 917, in
_bootstrap_inner
self.run()
  File "/home/python/v3.7.3/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
  File "./btc.py", line 11, in lockie
h = logging.Handler()
  File "/home/python/v3.7.3/lib/python3.7/logging/__init__.py", line
824, in __init__
self.createLock()
  File "/home/python/v3.7.3/lib/python3.7/logging/__init__.py", line
847, in createLock
_register_at_fork_acquire_release(self)
  File "/home/python/v3.7.3/lib/python3.7/logging/__init__.py", line
250, in _register_at_fork_acquire_release
_at_fork_acquire_release_weakset.add(instance)
  File "/home/python/v3.7.3/lib/python3.7/_weakrefset.py", line 83, in add
self._commit_removals()
  File "/home/python/v3.7.3/lib/python3.7/_weakrefset.py", line 56, in
_commit_removals
discard(l.pop())
IndexError: pop from empty list



import logging
import os
import sys
import threading

def lockie():
while True:
# this adds handle to _at_fork_acquire_release_weakset
#sys.stdout.write(".")
#sys.stdout.flush()
h = logging.Handler()

threading.Thread(target=lockie).start()

for n in range(0,10):
if n % 100 == 0:
sys.stdout.write("%d" % n)
sys.stdout.flush()
pid = os.fork()
if pid != 0:
os.wait()
else:
os._exit(0)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36532] Example of logging.formatter with new str.format style

[Vinay Sajip]

Vinay Sajip  added the comment:

But there is an example in the cookbook already:

https://docs.python.org/3/howto/logging-cookbook.html#use-of-alternative-formatting-styles

It's right there in the first code sample in that section.

You might want to amend your change to point to this section from the one where 
you want to make the change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

[cagney]

cagney  added the comment:

On Fri, 5 Apr 2019 at 04:15, Gregory P. Smith  wrote:
>
>
> New submission from Gregory P. Smith :
>
> I'm spawning a dicussion buried in the way too long thread of 
> https://bugs.python.org/issue6721 over here into its own specific issue to 
> treat as a 3.7 release blocker for a rollback or repair decision before 3.7.4.
>
> https://github.com/python/cpython/commit/3b699932e5ac3e76031bbb6d700fbea07492641d
>
> I believe that was released in 3.7.1 is leading to a behavior regression for 
> an application (the Fedora installer's libreswan kvmrunner?).  Full details 
> can be found in the messages of the other issue starting with:
>   https://bugs.python.org/issue6721#msg329474

Two separate applications have tripped up on this:

- Fedora's installer aka anaconda

I don't know any further details.

- libreswan's test framework aka kvmrunner

kvmrunner uses pexpect
pexpect uses ptyprocess
ptyprocess uses pty.fork() + os.exec*(); but, hey, who knew!  I didn't
until this week.
this seems to be a long standing concern:
https://github.com/pexpect/ptyprocess/issues/43

> TL;DR - logging.Handler instances each have their own threading.Rlock.
> libreswan implements at least one logging.Handler subclass.  That subclass's 
> custom emit() implementation directly calls potentially many other 
> sub-handlers emit() methods.  Some of those emit() methods (such as 
> logging.StreamHandler) call flush() which acquires the handler's lock.

# Implement log-level inversion.
#
# Ref: https://docs.python.org/2/howto/logging.html#logging-flow
#
# By default, a parent (root) logger, regardless of its log-level,
# will log all the records logged by a child.  For instance, if a
# child logger is logging at DEBUG-level, then the parent will log
# (display on the console) those DEBUG-level records even when it has
# been configured to log only INFO-level records.  This is because the
# default log-level ("Logger enabled for level of call?") check is
# only applied once at record-creation.
#
# This code allows DEBUG-level logging to a file, while simultaneously
# (the inversion) restricting console log records to just INFO-level
# say.

The logging.Handler's lock ensures that the two sub-streams are kept
in ordered.  It prevents threads interleaving their writes.

> So they've got a dependency between these two locks, the first's must be 
> acquired before the second.
>
> But the logging module APIs have no concept of sub-handlers and lock ordering.
>
> I see many flaws with the libreswan code's design (I'm already ignoring the 
> futility of threading + fork) but this still caused a behavior regression in 
> the stable 3.7 release.

Always first understand the problem.

> (more comments coming as followups to avoid a wall of text with too many 
> topics)
>
> --
> assignee: gregory.p.smith
> components: Library (Lib)
> keywords: 3.7regression
> messages: 339472
> nosy: cagney, gregory.p.smith, ned.deily, vstinner
> priority: release blocker
> severity: normal
> status: open
> title: logging regression with threading + fork are mixed in 3.7.1rc2 
> (deadlock potential)
> type: behavior
> versions: Python 3.7
>
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18748] libgcc_s.so.1 must be installed for pthread_cancel to work

[Zack Weinberg]

Zack Weinberg  added the comment:

I have observed this problem in a production application using Python 3.5 and 
3.6 (system-packaged interpreters from Ubuntu) and I would like to mention that 
this is an effective workaround:

```
import ctypes
libgcc_s = ctypes.CDLL("libgcc_s.so.1")
```

provided you do it before creating any threads, and the variable `libgcc_s` 
remains live until after all threads are terminated.

--
nosy: +zwol

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35983] tp_dealloc trashcan shouldn't be called for subclasses

[Jeroen Demeyer]

Change by Jeroen Demeyer :


--
pull_requests: +12624

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33533] Provide an async-generator version of as_completed

[Roman Evstifeev]

Change by Roman Evstifeev :


--
nosy: +Roman.Evstifeev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36534] tarfile: handling Windows (path) illegal characters in archive member names

[Eryk Sun]

Eryk Sun  added the comment:

_sanitize_windows_name() fails to translate the reserved control characters 
(0x01-0x1F) and backslash in names. 

What I've seen done in some cases (e.g. Unix network shares mapped to SMB) is 
to translate names using the private use area block, e.g. 0xF001 - 0xF07F. 
Windows has no problem with characters in this range in a filename. (Displaying 
these characters sensibly is another matter.) For Windows 10, this is 
especially useful since the Linux subsystem automatically translates this PUA 
block back to ASCII when accessing a Windows volume via drvfs. For example:

C:\Temp\pua>python -q
>>> import sys
>>> sys.platform
'win32'
>>> name = ''.join(map(chr, range(0xf001, 0xf080)))
>>> _ = open(name, 'w')
>>> ^Z

C:\Temp\pua>bash -c "python3 -q"
>>> import os, sys
>>> sys.platform
'linux'
>>> os.listdir()
['\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f
  \x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f
   !"#$%&\'()*+,-./0123456789:;<=>?
  @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_
  `abcdefghijklmnopqrstuvwxyz{|}~\x7f']

Also, while _sanitize_windows_name() handles trailing dots, for some reason it 
overlooks trailing spaces. It also doesn't handle reserved DOS device names. 
The reserved names include NUL, CON, CONIN$, CONOUT$, AUX, PRN, COM[1-9], 
LPT[1-9], and these names plus zero or more spaces and possibly a dot or colon 
and any subsequent characters. For example:

>>> os.path._getfullpathname('con')
'.\\con'
>>> os.path._getfullpathname('con  ')
'.\\con'
>>> os.path._getfullpathname('con:')
'.\\con'
>>> os.path._getfullpathname('con :')
'.\\con'
>>> os.path._getfullpathname('con : spam')
'.\\con'
>>> os.path._getfullpathname('con . eggs')
'.\\con'

It's not a reserved device name if the first character after zero or more 
spaces is not a dot or colon. For example:

>>> os.path._getfullpathname('con spam')
'C:\\con spam'

We can create filenames with reserved device names or trailing spaces and dots 
by using a \\?\ prefixed path (i.e. a non-normalized device path). However, 
most programs don't use \\?\ paths, so it's probably better to translate these 
names.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36044] PROFILE_TASK for PGO build is not a good workload

[Inada Naoki]

Change by Inada Naoki :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35551] Encoding and alias issues

[Inada Naoki]

Change by Inada Naoki :


--
assignee:  -> lemburg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36536] is there a python implementation of the cpython commandline interpretor?

[Larry Hastings]

Larry Hastings  added the comment:

The Python bug tracker is not your personal resource for answering programming 
questions.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36536] is there a python implementation of the cpython commandline interpretor?

[wis]

New submission from wis :

this algorithm: 
https://github.com/python/cpython/blob/34ef64fe5947bd7e1b075c785fc1125c4e600cd4/Python/coreconfig.c#L1644

I need a python library that does that to fix this answer
https://stackoverflow.com/a/55413882/4178053
I need to get the correct path of the script from the commandline of the python 
process.

I searched for libraries or implementations and couldn't find one, can someone 
help me? I can't read C.

--
components: Argument Clinic
messages: 339499
nosy: larry, wis
priority: normal
severity: normal
status: open
title: is there a python implementation of the cpython commandline interpretor?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

[Inada Naoki]

Change by Inada Naoki :


--
keywords: +patch
pull_requests: +12623
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

[Inada Naoki]

Inada Naoki  added the comment:

Additionally, _safe_read calls fp.read() multiple times to handle EINTR.
But EINTR is handled by socket module now (PEP 475).

Now the function can be very simple.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36256] parser module fails on legal input

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

3.6 only accepts security fixes at this point.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

[Inada Naoki]

Inada Naoki  added the comment:

issue1296004 is too old (512MB RAM machine!) and I can not confirm it.

But I think it was caused by inefficient realloc() in CRT.

See 
https://github.com/python/cpython/blob/6c52d76db8b1cb836c136bd6a1044e85bfe8241e/Lib/socket.py#L298-L303

_fileobject called socket.recv with remaining size.
Typically, socket can't return MBs at once.  So it cause:

1. Large (at most `amt`, some MBs) string (bytes) are allocated. (malloc)
2. recv is called.
3. _PyString_Resize() (realloc) is called with smaller bytes (typically ~128KiB)
4. amt -= received
5. if amt == 0: exit; goto 1.

This might stress malloc and realloc in CRT.  It caused fragmentation and 
MemoryError.

---

For now, almost everything is rewritten.

In case of _pyio, BufferedIOReader calls RawIOBase.read().  It copies from 
bytearray to bytes.  So call only malloc and free.  Stress for realloc will be 
reduced.

https://github.com/python/cpython/blob/50866e9ed3e4e0ebb60c20c3483a8df424c02722/Lib/_pyio.py#L586-L591

In case of C _io module, it is more efficient.  It allocate PyBytes once and 
calls SocketIO.read_into directly.  No temporary bytes objects are created.

https://github.com/python/cpython/blob/50866e9ed3e4e0ebb60c20c3483a8df424c02722/Modules/_io/bufferedio.c#L1632
https://github.com/python/cpython/blob/50866e9ed3e4e0ebb60c20c3483a8df424c02722/Modules/_io/bufferedio.c#L1658
https://github.com/python/cpython/blob/50866e9ed3e4e0ebb60c20c3483a8df424c02722/Modules/_io/bufferedio.c#L1470

--
type:  -> performance
versions: +Python 3.8 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14841] os.get_terminal_size() should check stdin as a fallback

[daniel hahler]

daniel hahler  added the comment:

Created a PR at https://github.com/python/cpython/pull/12697.

It prefers stdin and stderr over stdout.

I think stdin is most likely connected to a terminal, and wondered why ncurses 
uses stderr/stdout first, and only then stdin.
stdin handling was added there in 
https://github.com/mirror/ncurses/commit/aa70bf3c#diff-10ad6ea20599ac9258757354665b80cbR1295,
 and it looks just like an oversight maybe - it is also not that critical in C 
I guess.

--
nosy: +blueyed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14841] os.get_terminal_size() should check stdin as a fallback

[daniel hahler]

Change by daniel hahler :


--
keywords: +patch
pull_requests: +12622
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36535] Windows build failure when use the code from the GitHub master branch

[Manjusaka]

New submission from Manjusaka :

I use Visual Studio 2017 to build the source code from the master branch. But 
it failed. The output message shows that I lose my ffi.h.

I find that the developer had removed the libffi_module directory under 
cpython/modules/_ctypes  since 32119e10b792ad7ee4e5f951a2d89ddbaf111cc5

I guess that maybe that's the problem why I get failure

--
components: Windows
messages: 339494
nosy: Manjusaka, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows build failure when use the code from the GitHub master branch
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

[Inada Naoki]

Change by Inada Naoki :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36256] parser module fails on legal input

[A. Skrobov]

A. Skrobov  added the comment:

Is it intentional that the fix is not backported to 3.6 as well?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25451] tkinter: PhotoImage transparency methods

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34090] Python function call optimization: avoid temporary tuple to pass **kwargs

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

This might be solvable using PEP 580 by using METH_VARARGS instead of 
METH_FASTCALL for such functions. This would still require a temporary tuple 
for the positional args but no additional dict would need to be allocated.

--
nosy: +jdemeyer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25451] tkinter: PhotoImage transparency methods

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 50866e9ed3e4e0ebb60c20c3483a8df424c02722 by Serhiy Storchaka 
(Zackery Spytz) in branch 'master':
bpo-25451: Add transparency methods to tkinter.PhotoImage. (GH-10406)
https://github.com/python/cpython/commit/50866e9ed3e4e0ebb60c20c3483a8df424c02722


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35459] Use PyDict_GetItemWithError() instead of PyDict_GetItem()

[Inada Naoki]

Inada Naoki  added the comment:

Serhiy, can this issue be closed?

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36534] tarfile: handling Windows (path) illegal characters in archive member names

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
components: +Windows
nosy: +lars.gustaebel, paul.moore, steve.dower, tim.golden, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29209] Remove old-deprecated ElementTree features

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

This should be closed.

--
nosy: +jdemeyer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29202] Improve dict iteration

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29202] Improve dict iteration

[Inada Naoki]

Inada Naoki  added the comment:


New changeset f66e336f455b5a6bb0ca857d61c43be410d0df13 by Inada Naoki (Cheryl 
Sabella) in branch 'master':
bpo-29202: improve dict iteration (GH-11900)
https://github.com/python/cpython/commit/f66e336f455b5a6bb0ca857d61c43be410d0df13


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14017] Make it easy to create a new TextIOWrapper based on an existing

[Inada Naoki]

Inada Naoki  added the comment:

TextIOWrapper now has reconfigure() method.
Can this issue be closed?

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36534] tarfile: handling Windows (path) illegal characters in archive member names

[Cristi Fati]

New submission from Cristi Fati :

Although tar is a Nix based (and mostly used) format, it gains popularity on 
Win too.

As tarfile is running on Win, I think it should handle (work around) path 
incompatibilities, as zipfile (`ZipFile._sanitize_windows_name`) does.

Applies to all branches.

More details on [Tarfile/Zipfile extractall() changing filename of some 
files](https://stackoverflow.com/questions/55340013/tarfile-zipfile-extractall-changing-filename-of-some-files/55348443#55348443).

Regarding the current zipfile handling: it also can be improved (as it has a 
small bug), for example if the archive contains 2 files ("file:" and "file_") 
it won't work as expected. But this is a rare corner case.

I didn't prepare a patch, since I did so for another issue 
(https://bugs.python.org/issue36247 - which I consider an ugly one),  
 and it wasn't well received, also it was rejected (for different reasons). If 
this issue gets the green light from whomever is in charge, I'll be happy to 
provide one.

--
components: Library (Lib)
messages: 339486
nosy: CristiFati
priority: normal
severity: normal
status: open
title: tarfile: handling Windows (path) illegal characters in archive member 
names
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36496] Local variables can be used uninitialized in _PyPreConfig_Read()

[STINNER Victor]

STINNER Victor  added the comment:

Fixed by commit 6a8c3139ae9ada89d4a95985ec7cf8bb7d03bc01.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36496] Local variables can be used uninitialized in _PyPreConfig_Read()

[STINNER Victor]

STINNER Victor  added the comment:

Thanks for your bug report and fix Brad Larsen ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36301] Add _Py_PreInitialize() function

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 6a8c3139ae9ada89d4a95985ec7cf8bb7d03bc01 by Victor Stinner in 
branch 'master':
bpo-36301: Fix _PyPreConfig_Read() compiler warning (GH-12695)
https://github.com/python/cpython/commit/6a8c3139ae9ada89d4a95985ec7cf8bb7d03bc01


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36404] Document PendingDeprecationWarning is not so useful.

[miss-islington]

miss-islington  added the comment:


New changeset 86fbe0287dd774022fd2b6c2dcbfbb5573a0b874 by Miss Islington (bot) 
in branch '3.7':
bpo-36404: recommend DeprecationWarning over PendingDeprecationWarning 
(GH-12505)
https://github.com/python/cpython/commit/86fbe0287dd774022fd2b6c2dcbfbb5573a0b874


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Now to back up:  Why was this application using fork() in a threaded 
application at all?  That is a fundamental flaw.  Should we be doing work to 
support things that so far merely _happen_ to work on such broken designs?

Another alternative for the DebugHandler implementation in whatever code 
implemented it is for it to de-register itself from the logging library's 
private WeakSet of handler locks to acquire at fork time from its own 
constructor.  We don't have a public API for this so the workaround doing that 
on 3.7.1 - 3.7.3 would look like:

   def __init__(self, ...):
   super().__init__(...)
   if hasattr(logging, '_at_fork_acquire_release_weakset'):
   logging._at_fork_acquire_release_weakset.discard(self)

This means it'd still have the bug the code already had on all prior to 
versions of Python before this logging library "acquire the locks before fork" 
fix went in: Namely if the forked child tries to log it could deadlock due to 
forking while the inherited logging handler lock held.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

[Gregory P. Smith]

Gregory P. Smith  added the comment:

A stdlib alternative to this whole mess would be to avoid acquiring the logging 
locks before fork() as we currently do and just blindly re-initialize all of 
them afterwards under the assumption that they "can't" be protecting anything 
in a newly forked child process.  Handlers that need specific resource 
synchronization around fork would then be required to deal with their own 
os.register_at_fork() calls.  (ex: to avoid multiple processes writing to the 
same file or fd at once)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36404] Document PendingDeprecationWarning is not so useful.

[miss-islington]

Change by miss-islington :


--
pull_requests: +12621

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36404] Document PendingDeprecationWarning is not so useful.

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36404] Document PendingDeprecationWarning is not so useful.

[Inada Naoki]

Inada Naoki  added the comment:

After discussion on ML, I just recommend DeprecationWarning
over PendingDeprecationWarning in this time.
PendingDeprecationWarning is not deprecated for now.

--
title: Document PendingDeprecationWarning as deprecated -> Document 
PendingDeprecationWarning is not so useful.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36404] Document PendingDeprecationWarning as deprecated

[Inada Naoki]

New submission from Inada Naoki :


New changeset 176d26364bb67801fa522f52f20cbe44420d6942 by Inada Naoki in branch 
'master':
bpo-36404: recommend DeprecationWarning over PendingDeprecationWarning 
(GH-12505)
https://github.com/python/cpython/commit/176d26364bb67801fa522f52f20cbe44420d6942


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34396] Certain methods that heap allocated subtypes inherit suffer a 50-80% performance penalty

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

OK, makes sense. Also super() calls I guess: you can write 
super().__getitem__(x) but not super()[x] (although the latter *could* be 
implemented if we wanted to).

I see two ways of fixing this:

1. Make wrapper descriptors faster, removing the need for the METH_COEXIST hack 
in the first place.

2. Deal better with METH_COEXIST methods when inheriting.

Since both of these solutions are closely related to the topic of PEP 580/PEP 
590, I suggest to try to fix this after either is implemented.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36301] Add _Py_PreInitialize() function

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12620

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34589] Py_Initialize() and Py_Main() should not enable C locale coercion

[STINNER Victor]

STINNER Victor  added the comment:

I fixed the issue in Python 3.8 with bpo-36443:

commit d929f1838a8fba881ff0148b7fc31f6265703e3d
Author: Victor Stinner 
Date:   Wed Mar 27 18:28:46 2019 +0100

bpo-36443: Disable C locale coercion and UTF-8 Mode by default (GH-12589)

I'm not sure if it would be accceptable to the behavior in Python 3.7.x, 
especially without PEP 587 which targets Python 3.8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Within the logging module we could go beyond using a WeakSet and maintain an 
ordering.  But we'd need to allow a way for Handler subclasses to indicate what 
order matters; that'd require a new API (not suitable for 3.7)

An ordering itself may be insufficient as code can implement arbitrary Handler 
code with cyclic graphs of handlers calling directly into other handlers such 
that no possible order could ever satisfy them all.

This pulls us back into "stop having multiple locks" for any code that creates 
relationships between handlers.  Which is good advice for Handler authors and 
something we should document if it isn't already.

Side note: We should also document that Handler.emit() as an API MUST be called 
with the Handler's lock acquired via Handler.acquire() first.  That is the 
undocumented contracted way logging.Logger.handle()'s code always calls into 
handler emit()  The libreswan code calling other handler's emit() methods 
without doing that smells wrong.  But that has no relation to their current 
problem: Even if it did, it'd still deadlock, just sooner rather than within 
the other handler's emit()->flush()->acquire() call chain.

Logging library users are better off never directly calling another handler.  
Put message onto queues with a dedicated thread processing those into the 
relevant handlers.  Only block waiting for those queued items to have been 
processed _after_ releasing your own lock if you want to maintain the 
synchronicity of logging API calls returning only after the message has been 
handled.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26470] Make OpenSSL module compatible with OpenSSL 1.1.0

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12619

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

[Gregory P. Smith]

Gregory P. Smith  added the comment:

That custom DebugHandler's emit() implementation that calls into one or more 
sub-handlers suggests that libreswan _might_ be able to fix it in the custom 
DebugHandler by implementing custom acquire() and release() methods... BUT that 
is a fundamentally flawed problem: It requires acquiring multiple locks in a 
single atomic operation OR guaranteeing that they will always be acquired in a 
single specific order.

Given the logging module's fork-time code maintains no order, even implementing 
a custom acquire() and release() method in your custom DebugHandler would not 
be sufficient as sub-handler locks could be acquired first during fork.  also, 
that would be assuming your sub-handlers are not also used directly elsewhere 
within all possible coexisting logger configurations.

An implementable _gross workaround_ that libreswan or anything else in this 
boat could implement satisfying that constraint would be to force all 
potentially related handlers to share the same single RLock instance.  :(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6721] Locks in the standard library should be sanitized on fork

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Thanks for the debugging details!  I've filed 
https://bugs.python.org/issue36533 to specifically track this potential 
regression in the 3.7 stable branch.  lets carry on there where the discussion 
thread isn't too long for bug tracker sanity.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

[Gregory P. Smith]

New submission from Gregory P. Smith :

I'm spawning a dicussion buried in the way too long thread of 
https://bugs.python.org/issue6721 over here into its own specific issue to 
treat as a 3.7 release blocker for a rollback or repair decision before 3.7.4.

https://github.com/python/cpython/commit/3b699932e5ac3e76031bbb6d700fbea07492641d

I believe that was released in 3.7.1 is leading to a behavior regression for an 
application (the Fedora installer's libreswan kvmrunner?).  Full details can be 
found in the messages of the other issue starting with:
  https://bugs.python.org/issue6721#msg329474

TL;DR - logging.Handler instances each have their own threading.Rlock. 
libreswan implements at least one logging.Handler subclass.  That subclass's 
custom emit() implementation directly calls potentially many other sub-handlers 
emit() methods.  Some of those emit() methods (such as logging.StreamHandler) 
call flush() which acquires the handler's lock.

So they've got a dependency between these two locks, the first's must be 
acquired before the second.

But the logging module APIs have no concept of sub-handlers and lock ordering.

I see many flaws with the libreswan code's design (I'm already ignoring the 
futility of threading + fork) but this still caused a behavior regression in 
the stable 3.7 release.

(more comments coming as followups to avoid a wall of text with too many topics)

--
assignee: gregory.p.smith
components: Library (Lib)
keywords: 3.7regression
messages: 339472
nosy: cagney, gregory.p.smith, ned.deily, vstinner
priority: release blocker
severity: normal
status: open
title: logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock 
potential)
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36532] Example of logging.formatter with new str.format style

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +vinay.sajip
versions:  -Python 3.5, Python 3.6, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com