[issue35866] concurrent.futures deadlock

[Gregory P. Smith]

Gregory P. Smith  added the comment:

> "the single-threaded ProcessPoolExecutor test program"

I doubt it is single threaded, the .submit() method appears to spawn a thread 
internally.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36277] pdb's recursive debug command is not listed in the docs

[Dave Nguyen]

Change by Dave Nguyen :


--
keywords: +patch
pull_requests: +12796
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36654] Add example to tokenize.tokenize

[Windson Yang]

New submission from Windson Yang :

> The tokenize() generator requires one argument, readline, which must be a 
> callable object which provides the same interface as the io.IOBase.readline() 
> method of file objects. Each call to the function should return one line of 
> input as bytes.

Add an example like this should be easier to understand:

# example.py
def foo:
pass

# tokenize_example.py
import tokenize
f = open('example.py', 'rb')
token_gen = tokenize.tokenize(f.readline)

for token in token_gen:
# Something like this
# TokenInfo(type=1 (NAME), string='class', start=(1, 0), end=(1, 5), 
line='class Foo:\n')
# TokenInfo(type=1 (NAME), string='Foo', start=(1, 6), end=(1, 9), 
line='class Foo:\n')
# TokenInfo(type=53 (OP), string=':', start=(1, 9), end=(1, 10), 
line='class Foo:\n')
print(token)

--
assignee: docs@python
components: Documentation
messages: 340467
nosy: Windson Yang, docs@python
priority: normal
severity: normal
status: open
title: Add example to tokenize.tokenize
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36646] os.listdir() got permission error in Python3.6 but it's fine in Python2.7

[Steve Dower]

Steve Dower  added the comment:

This is weird.

Are you able to tell us anything about how L: is mounted or what it's pointing 
at? Do you know if it's SMB or similar, or if there are symlinks or similar?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14817] pkgutil.extend_path has no tests

[Windson Yang]

Windson Yang  added the comment:

I added some tests in the PR. Actually, there are some tests for extend_path 
already (see 
https://github.com/python/cpython/blob/master/Lib/test/test_pkgutil.py#L235). 
However, I didn't test every line of the code in the extend_path function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14817] pkgutil.extend_path has no tests

[Windson Yang]

Change by Windson Yang :


--
keywords: +patch
pull_requests: +12795
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

So:

#1 we've a bug: the single-threaded ProcessPoolExecutor test program should 
work 100% reliably - it does not

#2 we've a cause: ProcessPoolExecutor is implemented internally using an 
unfortunate combination of fork and threads, this is causing the deadlock

#3 we've got a workaround - something like:
   ProcessPoolExecutor(multiprocessing.get_context('spawn'))
but I'm guessing, the documentation is scant.

As for a fix, maybe:
- have ProcessPoolExecutor use 'spawn' by default; this way things always work
- have ProcessPoolExecutor properly synchronized its threads before 
"spawning"/"forking"/... so that "single-threaded" code works
- document that combining ProcessPoolExecutor's "fork" option and user threads 
isn't a good idea

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36646] os.listdir() got permission error in Python3.6 but it's fine in Python2.7

[Ryan]

Ryan  added the comment:

PS D:\workspace> python
Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.listdir(u'L:\\Temp')
[u'manager.txt', u'alps-trunk-m0.tk.log', u'2015102914.LOG', u'log', 
u'alps-trunk-m0.tk-k55mv1_64_om_c2k6m_eng-201511121
5.LOG', u'logof', u'1', u'dsp_1_ltg_n.bin', u'dsp_1_lwg_n.bin', 
u'modem_1_ltg_n.img', u'modem_1_lwg_n.img', u'kernel_sym
bols', u'k55v1_64_om_c2k6m_clang.log', u'k55v1_64_om_c2k6m_android.log', 
u'k50v1_64_om_c2k6m_android.log', u'collectBuil
dLog.sh', u'UtilitySpotlight', u'k55v1_bsp_android.log', u'specialowner.pm', 
u'DBbuild_GIT.pl', u'daily_out-k37mv1_basic
', u'mtkall_new.txt', u'Visa', u'insert_err_command_file.txt', u'test', 
u'DB_X_remake.list', u'aosp_img.txt', u'Jola', u
'04272', u'alps_dailybuild.ini', u'Luca', u'android-security-bulletin', 
u'michael', u'2018073115.LOG', u'LOG~1', u'mtk-d
b.pl', u'MCD_k79v1_64_android.log', u'build-full_k79v1_64.ninja']
>>> exit()
PS D:\workspace>
PS D:\workspace>
PS D:\workspace> python3
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.listdir(u'L:\\Temp')
Traceback (most recent call last):
  File "", line 1, in 
PermissionError: [WinError 5] Access denied.: 'L:\\Temp'
>>> exit()
PS D:\workspace>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36071] Add support for Windows ARM32 in ctypes/libffi

[Steve Dower]

Change by Steve Dower :


--
assignee:  -> steve.dower
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36638] typeperf.exe is not in all skus of Windows SKUs

[Steve Dower]

Change by Steve Dower :


--
assignee:  -> steve.dower
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36071] Add support for Windows ARM32 in ctypes/libffi

[Steve Dower]

New submission from Steve Dower :


New changeset 11efd79076559cc6e4034bb36db73e5e4293f02d by Steve Dower (Paul 
Monson) in branch 'master':
bpo-36071 Add support for Windows ARM32 in ctypes/libffi (GH-12059)
https://github.com/python/cpython/commit/11efd79076559cc6e4034bb36db73e5e4293f02d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36638] typeperf.exe is not in all skus of Windows SKUs

[Steve Dower]

Steve Dower  added the comment:


New changeset 264a0b40b030fc0ff919b8294df91bdaac853bfb by Steve Dower (Paul 
Monson) in branch 'master':
bpo-36638: Fix WindowsLoadTracker exception on some Windows versions (GH-12849)
https://github.com/python/cpython/commit/264a0b40b030fc0ff919b8294df91bdaac853bfb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36466] Adding a way to strip annotations from compiled bytecode

[cary]

cary  added the comment:

Thanks for the feedback! I wasn't aware that modules depended on this during 
runtime.

Abandoning this :)

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[Gregory P. Smith]

Gregory P. Smith  added the comment:

concurrent.futures.ProcessPoolExecutor uses both multiprocessing and threading. 
 multiprocessing defaults to using os.fork().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36648] MAP_SHARED isn't proper for anonymous mappings for VxWorks

[LihuaZhao]

Change by LihuaZhao :


--
pull_requests:  -12788

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36653] Dictionary Key is without ' ' quotes

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Hi PushkarVaity, and welcome!

I hope that Matthew's suggestion fixes your code for you, but please remember 
that this is a bug tracker for bugs in the Python language and standard 
library, not a help desk.

As a beginner, 99.9% of the times you think that you have found a bug in 
Python, you haven't, it will be a bug in your own code. There are many forums 
where you can ask for help with your code, such as the tutor mailing list 

https://mail.python.org/mailman/listinfo/tutor

Stackoverflow, Reddit's /rlearnpython, and more. You should check with other, 
more experienced programmers before reporting things as bugs.

Thank you.

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36653] Dictionary Key is without ' ' quotes

[Matthew Barnett]

Matthew Barnett  added the comment:

That should be:

def __repr__(self):
return repr(self.name)

Not a bug.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

@gregory.p.smith, I'm puzzled by your references to POSIX and/or os.fork().

The code in question looks like:

import concurrent.futures
import sys

def f():
import ctypes

while True:
with concurrent.futures.ProcessPoolExecutor() as executor:
ftr = executor.submit(f)
ftr.result()

which, to me, looks like pure Python.

Are you saying that this code can't work on GNU/Linux systems.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35959] math.prod(range(10)) caues segfault

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Hi Pablo,

Was this something you still wanted to clean up or can this be closed?  Thanks!

--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32913] Improve regular expression HOWTO

[Brett Cannon]

Brett Cannon  added the comment:


New changeset a6de52c74d831e45ee0ff105196da8a58b9e43cd by Brett Cannon (josh) 
in branch 'master':
bpo-32913: Added re.Match.groupdict example to regex HOWTO (GH-5821)
https://github.com/python/cpython/commit/a6de52c74d831e45ee0ff105196da8a58b9e43cd


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33039] int() and math.trunc don't accept objects that only define __index__

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Rémi,

Are you still working on the patch for this?  Thanks!

--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36653] Dictionary Key is without ' ' quotes

[PushkarVaity]

New submission from PushkarVaity :

I am using Python 3.7 with anaconda install.
I am trying to write out a dictionary with similar key's.

for ex: proc_dict = {'add': ('/home/file.tcl', 'args'), 'add': 
('/home/file2.tcl', 'args'), 'sub': ('/home/file2.tcl', 'args')}

To do this, I am using the following class definition and functions:
class ProcOne(object):
def __init__(self, name):
self.name = name

def __repr__(self):
return self.name

I am writing out the dictionary in the following way:
proc_dict[ProcOne(proc_name)] = (full_file, proc_args)

Now, the dictionary key as shown in the example at top is of string type. 
proc_name is the variable holding this string.

The values are tuples. Both elements in the tuple are strings.

When the dictionary is finally written out, the format is as below:
proc_dict = {add: ('/home/file.tcl', 'args'), add: ('/home/file2.tcl', 'args'), 
sub: ('/home/file2.tcl', 'args')} 

Please note the difference from the first example.
The key values don't have a ' ' quote in spite of being a string variable type. 

Since the string quotes are missing, it is very difficult to do post processing 
on this dictionary key.

I am a student and I though that this is an issue because now I am not able to 
compare the key value with a normal string because of the missing quotes. The 
in or not in checking operations do not evaluate to true/false because of the 
missing quotes.

Please let me know if this has never been reported before as I am just a novice 
programmer and would be a big boost to my morale :-) 
Also, please let me know if this issue was already known or wasn't an issue at 
all.

--
components: Regular Expressions
messages: 340452
nosy: PushkarVaity, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Dictionary Key is without ' ' quotes
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36277] pdb's recursive debug command is not listed in the docs

[Cheryl Sabella]

Change by Cheryl Sabella :


--
keywords: +easy
stage:  -> needs patch
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

We're discussing vanilla Python, for instance v3.7.0 is:

  git clone .../cpython
  cd cpython
  git checkout v3.7.0
  ./configure --prefix=/home/python/v3.7.0
  make -j && make -j install

(my 3.6.x wasn't vanilla, but I clearly stated that)

Like I also mentioned, loading down the machine also helps.  Try something like 
running #cores*2 of the script in parallel?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[miss-islington]

miss-islington  added the comment:


New changeset 0d4f16d283fe3b8a183775ac7ac193988d971ad5 by Miss Islington (bot) 
in branch '3.7':
bpo-36649: Remove trailing spaces for registry keys when installed via the 
Store (GH-12865)
https://github.com/python/cpython/commit/0d4f16d283fe3b8a183775ac7ac193988d971ad5


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[miss-islington]

Change by miss-islington :


--
pull_requests: +12794

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Steve Dower]

Steve Dower  added the comment:


New changeset 4c3efd9cd07194b5db2a60ae5951134cda8b69db by Steve Dower in branch 
'master':
bpo-36649: Remove trailing spaces for registry keys when installed via the 
Store (GH-12865)
https://github.com/python/cpython/commit/4c3efd9cd07194b5db2a60ae5951134cda8b69db


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36635] Add _testinternalcapi module

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 5c75f37d473140f0e0b7d9bf3a8c08343447ded1 by Victor Stinner in 
branch 'master':
bpo-36635: Change pyport.h for Py_BUILD_CORE_MODULE define (GH-12853)
https://github.com/python/cpython/commit/5c75f37d473140f0e0b7d9bf3a8c08343447ded1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36645] re.sub() library entry does not adequately document surprising change in behavior between versions

[mollison]

mollison  added the comment:

@brett.cannon: Yes, I will submit a PR. I have the code ready to go already. I 
just submitted the Python contributor agreement. I think it will take the 
system a day or two to register that. Then, I will submit the PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] On Unix, shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

STINNER Victor  added the comment:

> I suspect that the former is implementation accident. I can't imagine why
would anyone want this behavior.
>
> NB, POSIX says that when PATH is unset or empty, the path search is
implementation-defined.

Not thank you POSIX for being clueless. Let's say that Python is
opiniatied, as when we decided that file descriptors must be created
non-inheritable by default even if "it goes against POSIX".

If you want to look if the current directory, you now have to ask for it
kindly and explicitly ;-)

IHMO Using CS_PATH rather than hardcoded os.defpath is also a major step
forward ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36652] Non-embedded zip distribution

[wheelerlaw]

New submission from wheelerlaw :

Pretty straight forward request. It would be nice if there was an installation 
method where I can just unzip a Python distribution rather than running an 
installer. 

Specifically this is for getting Python to run in Wine. Right now, Python for 
Windows runs fine under Wine, but the installer doesn't, so a manual process of 
running the installer on a Windows machine and then copying the installed 
resources to a Linux machine with Wine installed. A zip distribution would 
solve this, since I could just unzip it and run it under Wine.

--
components: Installation
messages: 340445
nosy: wheelerlaw
priority: normal
severity: normal
status: open
title: Non-embedded zip distribution
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18564] Integer overflow in the socket function parsing a Bluetooth address

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Seconded Viktor's question.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10417] [2.7] unittest triggers UnicodeEncodeError with non-ASCII character in the docstring of the test function

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I mean that '\xe4'.encode(encoding, 'backslashreplace') will fail.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27679] set_bitfields() unused in _ctypes_test

[Joannah Nanjekye]

Change by Joannah Nanjekye :


--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36651] Asyncio Event Loop documentation inconsistency (call_later and call_at methods)

[Enrico Carbognani]

Change by Enrico Carbognani :


--
keywords: +patch
pull_requests: +12793
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[Gregory P. Smith]

Gregory P. Smith  added the comment:

I am unable to get cf-deadlock.py to hang on my own builds of pure CPython 
3.7.2+ d7cb2034bb or 3.6.8+ be77fb7a6e (versions i had in a local git clone).

which specific python builds are seeing the hang using?  Which specific 
platform/distro version?  "3.7.2" isn't enough, if you are using a distro 
supplied interpreter please try and reproduce this with a build from the 
CPython tree itself.  distros always apply their own patches to their own 
interpreters.

...

Do realize that while working on this it is fundamentally *impossible* per 
POSIX for os.fork() to be safely used at the Python level in a process also 
using pthreads.  That this _ever_ appeared to work is a pure accident of 
implementations of underlying libc, malloc, system libraries, and kernel 
behaviors.  POSIX considers it undefined behavior.  Nothing done in CPython can 
avoid that.  Any "fix" for these kinds of issues is merely working around the 
inevitable which will re-occur.

concurrent.futures.ProcessPoolExecutor uses multiprocessing for its process 
management.  As of 3.7 ProcessPoolExecutor accepts a mp_context parameter to 
specify the multiprocessing start method.  Alternatively the default appears
to be controllable as a global setting 
https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods.

Use the 'spawn' start method and the problem should go away as it'll no longer 
be misusing os.fork().  You _might_ be able to get the 'forkserver' start 
method to work, but only reliably if you make sure the forkserver is spawned 
_before_ any threads in the process (such as ProcessPoolExecutor's own queue 
management thread - which appears to be spawned upon the first call to 
.submit()).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21710] --install-base option ignored?

[Joannah Nanjekye]

Change by Joannah Nanjekye :


--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Steve Dower]

Steve Dower  added the comment:

Great, thanks.

I have a fix in PR, but I'm going to do a "real" build to check. My machine is 
running a beta build, unfortunately, so I can't validate it against the older 
version. Might ping you for some help with that.

--
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Jason R. Coombs]

Jason R. Coombs  added the comment:

Hi Josh. Thanks for the tip on types.MethodType. I've updated the code to use 
that and the behavior seems to be the same, MethodType does seem a more 
appropriate way to create a bound method.

Regarding the referenced tickets, I suspect they're not pertinent, as the 
behavior did work in Python 3.7.1 (confirmed) and 3.7.2 (highly likely), so the 
regression appears to be in the changes for 3.7.3 
(https://docs.python.org/3.7/whatsnew/changelog.html#python-3-7-3-final).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Josh Rosenberg]

Josh Rosenberg  added the comment:

Hmm... Although I can't seem to reproduce on 3.7.2 (ob.calls is 1), so assuming 
it is really happening in 3.7.3, it wouldn't be either of those issues (which 
were fully in place long before 3.7.2 I believe).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Josh Rosenberg]

Josh Rosenberg  added the comment:

It seems highly likely this is related to #26110 which optimizes method calls 
by avoiding the creation of bound methods in certain circumstances, and/or the 
follow-up #29263.

Side-note:

bound_method = functools.partial(method, self)

is not how you create real bound methods. The correct approach (that makes a 
bound method identical to what the interpreter makes when necessary) for your 
use case is:

bound_method = types.MethodType(method, self)

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[Rudolph Froger]

Rudolph Froger  added the comment:

Thanks all for the fixes!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23382] Maybe can not shutdown ThreadPoolExecutor when call the method of shutdown

[Josh Rosenberg]

Josh Rosenberg  added the comment:

Correct me if I'm wrong, but this isn't actually an issue for CPython, right? 
The GIL ensures that when a thread writes to _shutdown, nothing else is reading 
it until the GIL is released and acquired by a new thread (which synchronizes 
_shutdown).

It might conceivably be a problem on non-CPython interpreters if they have no 
other form of inter-thread synchronization involved; that said, the locking 
involved in the self.work_queue.get(block=True) call likely synchronizes by 
side-effect even there.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Hugues Valois]

Hugues Valois  added the comment:

C:\Users\huvalo>ver

Microsoft Windows [Version 10.0.17763.437]

--
versions:  -Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36651] Asyncio Event Loop documentation inconsistency (call_later and call_at methods)

[Enrico Carbognani]

New submission from Enrico Carbognani :

In the documentation for the call_later and the call_at methods there is a note 
which says that the delay cannot be longer than a day, but both methods have a 
note saying that this limitation was removed in Python 3.8.

--
assignee: docs@python
components: Documentation
files: documenation_incosistency.png
messages: 340434
nosy: Enrico Carbognani, docs@python
priority: normal
severity: normal
status: open
title: Asyncio Event Loop documentation inconsistency (call_later and call_at 
methods)
versions: Python 3.8
Added file: https://bugs.python.org/file48273/documenation_incosistency.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Jason R. Coombs]

Jason R. Coombs  added the comment:

I've put together this full reproducer script:

```
import functools


def method_cache(method):
def wrapper(self, *args, **kwargs):
# it's the first call, replace the method with a cached, bound 
method
bound_method = functools.partial(method, self)
cached_method = functools.lru_cache()(bound_method)
setattr(self, method.__name__, cached_method)
return cached_method(*args, **kwargs)
return wrapper


class MyClass:
calls = 0

@method_cache
def call_me_maybe(self, number):
self.calls += 1
return number


ob = MyClass()
ob.call_me_maybe(0)
ob.call_me_maybe(0)
assert ob.calls == 1
```

That code fails on Python 3.7.3. If I bypass the `from _functools import 
_lru_cache_wrapper` in functools, the test no longer fails, so the issue seems 
to be only with the C implementation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +12792
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

script to capture stack backtrace at time of fork, last backtrace printed will 
be for hang

--
Added file: https://bugs.python.org/file48272/gdb.sh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

run ProcessPoolExecutor with one fixed child (over ride default of #cores)

--
Added file: https://bugs.python.org/file48271/cf-deadlock-1.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Jason R. Coombs]

Change by Jason R. Coombs :


--
components: +Library (Lib)
type:  -> behavior
versions: +Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

Here's a possible stack taken during the fork():

Thread 1 "python3" hit Breakpoint 1, 0x77124734 in fork () from 
/lib64/libc.so.6

Thread 1814 (Thread 0x7fffe69d5700 (LWP 23574)):
#0  0x77bc24e5 in __pthread_mutex_unlock_usercnt () from 
/lib64/libpthread.so.0
#1  0x771928e3 in dl_iterate_phdr () from /lib64/libc.so.6
#2  0x7fffe5fcfe55 in _Unwind_Find_FDE () from /lib64/libgcc_s.so.1
#3  0x7fffe5fcc403 in uw_frame_state_for () from /lib64/libgcc_s.so.1
#4  0x7fffe5fcd90f in _Unwind_ForcedUnwind_Phase2 () from 
/lib64/libgcc_s.so.1
#5  0x7fffe5fcdf30 in _Unwind_ForcedUnwind () from /lib64/libgcc_s.so.1
#6  0x77bc7712 in __pthread_unwind () from /lib64/libpthread.so.0
#7  0x77bbf7e7 in pthread_exit () from /lib64/libpthread.so.0
#8  0x0051b2fc in PyThread_exit_thread () at Python/thread_pthread.h:238
#9  0x0055ed16 in t_bootstrap (boot_raw=0x7fffe8da0e40) at 
./Modules/_threadmodule.c:1021
#10 0x77bbe594 in start_thread () from /lib64/libpthread.so.0
#11 0x77157e5f in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x77fca080 (LWP 20524)):
#0  0x77124734 in fork () from /lib64/libc.so.6
#1  0x00532c8a in os_fork_impl (module=) at 
./Modules/posixmodule.c:5423
#2  os_fork (module=, _unused_ignored=) at 
./Modules/clinic/posixmodule.c.h:1913

where, in my source code, dl_iterate_phdr() starts with something like:

  /* Make sure nobody modifies the list of loaded objects.  */
  __rtld_lock_lock_recursive (GL(dl_load_write_lock));

i.e., when the fork occures, the non-fork thread has acquired 
dl_load_write_lock - the same lock that the child will later try to acquire 
(and hang)

no clue as to what that thread is doing though; other than it looks like it is 
trying to generate a backtrace?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Jason R. Coombs]

New submission from Jason R. Coombs :

In [this ticket](https://github.com/jaraco/jaraco.functools/issues/12), I 
learned that 
[jaraco.functools.method_cache](https://github.com/jaraco/jaraco.functools/blob/6b32ee0dfd3e7c88f99e88cd87c35fa9b76f261f/jaraco/functools.py#L109-L180)
 no longer works on Python 3.7.3.

A distilled version of what's not working is this example:

```
>>> import jaraco.functools
>>> class MyClass:
...   calls = 0
...   @jaraco.functools.method_cache
...   def call_me_maybe(self, val):
... self.calls += 1
... return val
... 
>>> a = MyClass()
>>> a.call_me_maybe(0)
0
>>> a.call_me_maybe(0)
0
>>> a.calls
2
```

The second call to the cached function is missing the cache even though the 
parameters to the function are the same.


```
>>> a.call_me_maybe

>>> a.call_me_maybe.cache_info()
CacheInfo(hits=0, misses=2, maxsize=128, currsize=2)
```

Here's a further distilled example not relying on any code from 
jaraco.functools:

```
>>> def method_cache(method):
... def wrapper(self, *args, **kwargs):
... # it's the first call, replace the method with a cached, bound 
method
... bound_method = functools.partial(method, self)
... cached_method = functools.lru_cache()(bound_method)
... setattr(self, method.__name__, cached_method)
... return cached_method(*args, **kwargs)
... return wrapper
... 
>>> import functools
>>> class MyClass:
...   calls = 0
...   @method_cache
...   def call_me_maybe(self, val):
... self.calls += 1
... return val
... 
>>> a = MyClass()
>>> a.call_me_maybe(0)
0
>>> a.call_me_maybe(0)
0
>>> a.calls
2
```

I was not able to replicate the issue with a simple lru_cache on a partial 
object:

```
>>> def func(a, b):
...   global calls
...   calls += 1
... 
>>> import functools
>>> cached = functools.lru_cache()(functools.partial(func, 'a'))
>>> calls = 0
>>> cached(0)
>>> cached(0)
>>> calls
1
```

Suggesting that there's some interaction with the instance attribute and the 
caching functionality.

I suspect the issue arose as a result of changes in issue35780.

--
assignee: rhettinger
keywords: 3.7regression
messages: 340429
nosy: jaraco, rhettinger
priority: normal
severity: normal
status: open
title: Cached method implementation no longer works on Python 3.7.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

[Nick Davies]

Nick Davies  added the comment:

My preference would actually be number 3 because:

 1: I agree that this isn't really a safe option because it could slow things 
down (possibly a lot)
 2: I haven't found this to be rare in my situation but I am not sure how 
common my setup is. We have a threaded server with a mix of sync and asyncio so 
we use run in a bunch of places inside threads. Any time the server gets busy 
any task creation that occurs during the return of run crashes. My two main 
reservations about this approach are:
- There is a potentially unbounded number of times that this could need to 
retry.
- Also this is covering up a thread unsafe operation and we are pretty 
lucky based on the current implementation that it explodes in a consistent and 
sane way that we can catch and retry.
 3: Loop is already expected to be hashable in 3.7 as far as I can tell 
(https://github.com/python/cpython/blob/3.7/Lib/asyncio/tasks.py#L822) so other 
than the slightly higher complexity this feels like the cleanest solution.


> The fix can be applied to 3.7 and 3.8 only, sorry. Python 3.6 is in security 
> mode now.

Thats fine, you can work around the issue using asyncio.set_task_factory to 
something that tracks the tasks per loop with some care.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Steve Dower]

Steve Dower  added the comment:

Can you get me the *exact* Windows version number you're using?

I added those to work around a bug where keys were being incorrectly trimmed, 
and that format seemed to evaluate to nothing correctly, but perhaps that bug 
has been fixed in the OS?

--
assignee:  -> steve.dower
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Hugues Valois]

New submission from Hugues Valois :

When reading registry values under HKCU\SOFTWARE\Python\PythonCore\3.7 that 
were written by the Windows Store app install, all file and folder paths are 
incorrect.

Notice the extra [ ] as well as the missing backslash before python.exe and 
pythonw.exe

Paths read from registry are:
```
C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0[
]

C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0python.exe[
]

C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0pythonw.exe[
]

```

Paths on disk are:
```
C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0

C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0\python.exe

C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0\pythonw.exe
```

--
components: Installation, Windows
messages: 340426
nosy: Hugues Valois, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows Store app install registry keys have incorrect paths
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

[Andrew Svetlov]

Andrew Svetlov  added the comment:

The fix can be applied to 3.7 and 3.8 only, sorry.
Python 3.6 is in security mode now.

--
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Thanks for the report!

I see 3 ways to fix the bug:
1. Guard _all_tasks with threading.Lock. It hurts performance significantly.
2. Retry list(_all_tasks) call in a loop if RuntimeError was raised. A chance 
of collision is very low, the strategy is good enough
3. Change _all_tasks from weak set of tasks to WeakDict[AbstractEventLoop, 
WeakSet[Task]]. This realization eliminates the collision it is a little 
complicated. Plus loop should be hashable now (perhaps ok but I'm not sure if 
all third-party loops support it).

Thus I'm inclining to bullet 2.
THoughts?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36645] re.sub() library entry does not adequately document surprising change in behavior between versions

[Brett Cannon]

Brett Cannon  added the comment:

@mollison: would you like to open a PR w/ how you would expect it to be 
formatted?

--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] On Unix, shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[Jakub Wilk]

Jakub Wilk  added the comment:

When PATH is empty string:
* zsh and FreeBSD which look for binaries in cwd.
* debianutils and GNU which always fail.

I suspect that the former is implementation accident. I can't imagine why would 
anyone want this behavior.

NB, POSIX says that when PATH is unset or empty, the path search is 
implementation-defined.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

[Tim Hatch]

Change by Tim Hatch :


--
nosy: +thatch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18564] Integer overflow in the socket function parsing a Bluetooth address

[STINNER Victor]

Change by STINNER Victor :


--
title: Integer overflow in socketmodule -> Integer overflow in the socket 
function parsing a Bluetooth address
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18564] Integer overflow in socketmodule

[STINNER Victor]

STINNER Victor  added the comment:

> In Modules/socketmodule.c , the bluetooth address supplied is vulnerable to 
> integer overflow.

Attached PR 12864 modifies the following code:


  unsigned int b0, b1, b2, b3, b4, b5;
  char ch;
  int n;
  n = sscanf(name, "%X:%X:%X:%X:%X:%X%c", , , , , , , );

Can someone please elaborate how this code can trigger an integer overflow? 
What is the consequence of an integer overflow? Does Python crash?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18564] Integer overflow in socketmodule

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +12791
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] On Unix, shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

STINNER Victor  added the comment:

I modified posixpath.defpath, shutil.which() and 
distutils.spawn.find_executable() in 3.7 and master (future Python 3.8) 
branches. I close the issue. Thanks everybody for the review and helping me to 
collect info about corner cases!

I chose to also change Python 3.7. IMHO there is a low risk of breaking 
applications: I expect that few users run Python with no PATH environment 
variable *and* expect that Python looks for programs in the current directory. 
But it enhances the security a little bit.

For Python 2.7... well, I don't think that this issue is important enough to 
justify a backport. I prefer to do nothing rather than having to deal with 
unhappy users complaining that Python 2.7 changed broke their application in a 
minor 2.7.x release :-) Even if, again, the risk of regression is very low.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
title: shutil.which() and subprocess no longer look for the executable in the 
current directory if PATH environment variable is not set -> On Unix, 
shutil.which() and subprocess no longer look for the executable in the current 
directory if PATH environment variable is not set

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

STINNER Victor  added the comment:

I'm still confused by distutils.spawn.find_executable() function which *always* 
first look the current directory. I don't know the rationale for this behavior, 
so I made the conservation choice of keeping it.

If someone wants to change distutils.spawn.find_executable(), please open a 
separated issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

STINNER Victor  added the comment:

Gregory:
> I'm not arguing against this change, just trying to figure out where it came 
> from in the first place.  We should fix the value on all OSes.

In the meanwhile, I reverted the ntpath change. I'm not sure that it's ok to 
change the Windows case.

shutil.which() *always* starts by checking if the searched program is the 
current directory:

if sys.platform == "win32":
# The current directory takes precedence on Windows.
...
if curdir not in path:
path.insert(0, curdir)

If someone cares about changing Windows, please open a separated issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36639] Provide list.rindex()

[林自均]

林自均  added the comment:

Hi @rhettinger ,

Thank you for the reply. May I ask what is the known, strong use cases for 
str.rindex()?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

Change by STINNER Victor :


--
title: shutil.which() and subprocess no longer looks the executable in the 
current directory if PATH environment variable is not set -> shutil.which() and 
subprocess no longer look for the executable in the current directory if PATH 
environment variable is not set

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] shutil.which() and subprocess no longer looks the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

Change by STINNER Victor :


--
title: Remove current directory from posixpath.defpath to enhance security -> 
shutil.which() and subprocess no longer looks the executable in the current 
directory if PATH environment variable is not set

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 394b991e41a2a4ce3afc8e6fde44de46e73bbb34 by Victor Stinner in 
branch '3.7':
[3.7] bpo-35755: shutil.which() uses os.confstr("CS_PATH") (GH-12862)
https://github.com/python/cpython/commit/394b991e41a2a4ce3afc8e6fde44de46e73bbb34


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[STINNER Victor]

STINNER Victor  added the comment:

Thanks Rudolph Froger for the bug report: the issue is now fixed in 3.7 and 
master (future Python 3.8) branches. Sorry for the delay.

--

Alexey Izbyshev wrote PR 5773 to also use fstat() on Linux.

I chose to merge my PR 12852 which is more conservative: it keeps dup() on 
Linux. I'm not sure why exactly, but I recall that the author of the function, 
Antoine Pitrou, wanted to use dup() on Linux.

I'm not convinced by the O_PATH issue on Linux (described above), so I merged 
my conservative change instead.

Later, we can still move to fstat() on Linux as well if someone comes with a 
more concrete example against dup().

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[STINNER Victor]

STINNER Victor  added the comment:

In short, Python 2.7 doesn't seem to be affected by fstat/dup issues.

Python 2.7 doesn't check if file descriptors 0, 1 and 2 at startup. Python 2 
uses PyFile_FromFile() to create sys.stdin, sys.stdout and sys.stderr which 
create a "file" object. The function calls fstat(fd) but it ignores the error: 
fstat() is only used to fail if the fd is a directory.

Python 2.7 doesn't have the is_valid_fd() function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b87a8073db73f9ffa96104e00c624052e34b11c7 by Victor Stinner (Miss 
Islington (bot)) in branch '3.7':
bpo-32849: Fix is_valid_fd() on FreeBSD (GH-12852) (GH-12863)
https://github.com/python/cpython/commit/b87a8073db73f9ffa96104e00c624052e34b11c7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 3092d6b2630e4d2bd200fbc3231c27a7cba4d6b2 by Victor Stinner in 
branch 'master':
bpo-32849: Fix is_valid_fd() on FreeBSD (GH-12852)
https://github.com/python/cpython/commit/3092d6b2630e4d2bd200fbc3231c27a7cba4d6b2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[miss-islington]

Change by miss-islington :


--
pull_requests: +12790

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22454] Adding the opposite function of shlex.split()

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31094] asyncio: get list of connected clients

[LihuaZhao]

Change by LihuaZhao :


--
pull_requests: +12789

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36648] MAP_SHARED isn't proper for anonymous mappings for VxWorks

[LihuaZhao]

Change by LihuaZhao :


--
keywords: +patch
pull_requests: +12788
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36648] MAP_SHARED isn't proper for anonymous mappings for VxWorks

[LihuaZhao]

Change by LihuaZhao :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36648] MAP_SHARED isn't proper for anonymous mappings for VxWorks

[LihuaZhao]

Change by LihuaZhao :


--
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36648] MAP_SHARED isn't proper for anonymous mappings for VxWorks

[LihuaZhao]

New submission from LihuaZhao :

anonymous mappings is not part of the POSIX standard, python user just need to 
specified -1 as fd value when do anonymous map, for example:

m = mmap.mmap(-1, 100)

then python adapter module(mmapmodule.c) try to specify MAP_SHARED or 
MAP_PRIVATE based on operate system requirement, Linux require MAP_SHARED, 
VxWorks require MAP_PRIVATE, this different should be hidden by this module, 
and python user won't be affected.

Currently, mmap is only adapted for the system which use MAP_SHARED when do 
anonymous map, VxWorks need be supported.

https://en.wikipedia.org/wiki/Mmap

--
components: Library (Lib)
messages: 340411
nosy: lzhao
priority: normal
pull_requests: 12787
severity: normal
status: open
title: MAP_SHARED isn't proper for anonymous mappings for VxWorks
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12786

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 197f0447e3bcfa4f529fedab09966d7e3d283979 by Victor Stinner in 
branch 'master':
bpo-35755: Don't say "to mimick Unix which command behavior" (GH-12861)
https://github.com/python/cpython/commit/197f0447e3bcfa4f529fedab09966d7e3d283979


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31904] Python should support VxWorks RTOS

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 693c104ae74feea11f0b51176dc91ecd153230c0 by Victor Stinner (Lihua 
Zhao) in branch 'master':
bpo-31904: Port test_resource to VxWorks (GH-12719)
https://github.com/python/cpython/commit/693c104ae74feea11f0b51176dc91ecd153230c0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

[STINNER Victor]

STINNER Victor  added the comment:

> urllib3 now vendors a copy of the rfc3986 library:
> https://pypi.org/project/rfc3986/

There are multiple Python projects to validate URI:

* https://github.com/python-hyper/rfc3986/ -> https://pypi.org/project/rfc3986/
* https://github.com/dgerber/rfc3987 -> https://pypi.org/project/rfc3987/ (the 
name is confusing: the library implements the RFC 3986, not the RFC 3987)
* https://github.com/tkem/uritools/ -> https://pypi.org/project/uritools/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

[STINNER Victor]

STINNER Victor  added the comment:

"wave Hi! I've noticed that CVE-2019-11236 has been assigned to the CRLF 
injection issue described here. It seems that the library has been patched in 
GitHub, but no new release has been made to pypi. (...)"

This urllib3 change:
https://github.com/urllib3/urllib3/commit/0aa3e24fcd75f1bb59ab159e9f8adb44055b2271

urllib3 now vendors a copy of the rfc3986 library:

https://pypi.org/project/rfc3986/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31904] Python should support VxWorks RTOS

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 2954550818e5c23a082e6279eb326168230ebf04 by Victor Stinner (Lihua 
Zhao) in branch 'master':
bpo-31904:  Port test_cmd_line to VxWorks (#12648)
https://github.com/python/cpython/commit/2954550818e5c23a082e6279eb326168230ebf04


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

[STINNER Victor]

STINNER Victor  added the comment:

It seems like a change has been pushed into urllib3 to fix this issue, but that 
there is an issue with international URLs and that maybe RFC 3986 should be 
updated.

RFC 3986: "Uniform Resource Identifier (URI): Generic Syntax" (January 2005)
https://www.ietf.org/rfc/rfc3986.txt

"Without #1531 or IRI support in rfc3986 releasing master in it's current state 
will break backwards compatibility with international URLs."

https://github.com/urllib3/urllib3/issues/1553#issuecomment-474046652

=> where 1531 means https://github.com/urllib3/urllib3/pull/1531

"wave Hi! I've noticed that CVE-2019-11236 has been assigned to the CRLF 
injection issue described here. It seems that the library has been patched in 
GitHub, but no new release has been made to pypi. Will a new release containing 
the fix be made to pypi soon? Based on @theacodes comment it seems like a 
release was going to be made, but I also see her status has her perhaps 
unavailable. Is someone else perhaps able to cut a new release into pypi?"

https://github.com/urllib3/urllib3/issues/1553#issuecomment-484113222

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

STINNER Victor  added the comment:

Anyway, I wrote PR 12861 to remove "to mimick Unix which command behavior".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12785

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

STINNER Victor  added the comment:

> My point is that "which" implementations have different behavior

I don't understand this point. Your example is consistent with what I saw on my 
Fedora 29 and the Python implementation that I just merged. Would you mind to 
elaborate which corner case is handled differently and describe how?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[Jakub Wilk]

Jakub Wilk  added the comment:

(Note that in msg333835 another implementation, presumably GNU which, was 
tested.)

My point is that "which" implementations have different behavior, so justifying 
anything with "which" compatibility is weird at best. You can't be compatible 
with all them.

Also telling users that you "mimick Unix which command behavior" is unhelpful, 
because vast majority of users have no idea how it behaves in this corner case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 2c4c02f8a876fcf084575dcaf857a0236c81261a by Victor Stinner in 
branch 'master':
bpo-35755: Remove current directory from posixpath.defpath (GH-11586)
https://github.com/python/cpython/commit/2c4c02f8a876fcf084575dcaf857a0236c81261a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

STINNER Victor  added the comment:

CS_PATH value:

* Fedora 29: "/usr/bin"
* Ubuntu 16.04: "/bin:/usr/bin"

It seems like the current directory is usually not part of the CS_PATH value.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 228a3c99bdb2d02771bead66a0beabafad3a90d3 by Victor Stinner in 
branch 'master':
bpo-35755: shutil.which() uses os.confstr("CS_PATH") (GH-12858)
https://github.com/python/cpython/commit/228a3c99bdb2d02771bead66a0beabafad3a90d3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36647] TextTestRunner doesn't honour "buffer" argument

[José Luis Segura Lucas]

New submission from José Luis Segura Lucas :

When using "buffer = True" in a TextTestRunner, the test result behaviour 
doesn't change at all.

This is because TextTestRunner.stream is initialised using a decorator 
(_WritelnDecorator). When "buffer" is passed, the TestResult base class will 
try to redirect the stdout and stderr to 2 different io.StringIO objects. As 
the TextTestRunner.stream is initialised before that "redirection", all the 
"self.stream.write" calls will end using the original stream (stderr by 
default), and resulting in not buffering at all.

--
components: Tests
messages: 340398
nosy: José Luis Segura Lucas
priority: normal
severity: normal
status: open
title: TextTestRunner doesn't honour "buffer" argument
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36646] os.listdir() got permission error in Python3.6 but it's fine in Python2.7

[Eryk Sun]

Eryk Sun  added the comment:

Did you try os.listdir(u'L:\\Temp') in Python 2?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36305] Inconsistent behavior of pathlib.WindowsPath with drive paths

[Eryk Sun]

Eryk Sun  added the comment:

> In contrast, in the case of CreateProcessW, both Path('./exec') and 
> Path('exec') are the *correct* paths to the executable. The ./ is not 
> necessary in that case to display the path correctly, but just to 
> interact correctly with CreateProcessW's interface.

It's semantic information, just for different reasons than the "./c:a" case. In 
this case, it's about what a path means in a search context. I think specifying 
a filesystem path for a search disposition is as valid a use case as is 
specifying a path for an open or create disposition. 

In Windows, the qualified path r".\exec" is resolved relative to just the 
working directory. Classically, for an unqualified name such as "exec", the 
working directory is checked after the application directory. Starting with 
Windows Vista, CreateProcessW and the CMD shell won't check the working 
directory at all for unqualified "exec" if NoDefaultCurrentDirectoryInExePath 
is set in the environment, not unless a "." entry is explicitly added to PATH.

The behavior of CreateProcessW also differs for r".\spam\exec" vs r"spam\exec". 
Without explicitly including the leading ".", the system searches every 
directory in the executable search path (i.e. the application directory, 
working directory, system directories, and PATH). This differs from Unix, in 
which any path with a slash in it is always resolved relative to the working 
directory. Getting this behavior in Windows requires using the qualified path 
r".\spam\exec".

We may want either behavior. I think if a path is specified explicitly with a 
leading "." component, or joined from one, the "." component should be 
preserved, just as we already preserve a leading ".." component. 

This is a separate issue, as I suggested in the PR comment. It was just 
supposed to be a related issue that you might be interested in. I didn't intend 
to conflate it with this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36273] test_thread leaks a core dump on PPC64 AIX 3.x

[Michael Felt]

Michael Felt  added the comment:

On 17/04/2019 14:02, STINNER Victor wrote:
> STINNER Victor  added the comment:
>
> The same bug is back:
> https://buildbot.python.org/all/#/builders/10/builds/2443
>
> Warning -- files was modified by test_threading
>   Before: []
>   After:  ['core']
>
> --

What can I do to "copy" the core dump before normal testing removes it?

I have had a similar message regarding the multiprocessing tests, but in
those cases I expect it is the "client" thread complaining about a
"server" thread crash.

>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com