[issue39081] pathlib '/' operator does not resolve Enums with str mixin as expected

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

As per the fspath PEP https://www.python.org/dev/peps/pep-0519/#c-api the below 
precedence is set. So for the enum inheriting from str the object itself is 
returned and MyEnum.RED.__str__ is used returning MyEnum.RED. You can remove 
inheritance from str and implement __fspath__ for your enum class to be used as 
per fspath protocol.

> If the object is str or bytes, then allow it to pass through with
  an incremented refcount. If the object defines __fspath__(), then
  return the result of that method. All other types raise a TypeError.

# bpo39081.py

import os
import pathlib
import enum

class MyEnum(enum.Enum):
RED = 'red'

def __fspath__(self):
return self.name

print(os.fspath(MyEnum.RED))
print(pathlib.Path.home() / MyEnum.RED)

$ python3.8 bpo39081.py
RED
/Users/kasingar/RED

--
nosy: +ethan.furman, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39028] ENH: Fix performance issue in keyword extraction

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39028] ENH: Fix performance issue in keyword extraction

[Inada Naoki]

Inada Naoki  added the comment:


New changeset 75bb07e92baa7267a61056d03d7e6b475588e793 by Inada Naoki 
(Sebastian Berg) in branch 'master':
bpo-39028: Performance enhancement in keyword extraction (GH-17576)
https://github.com/python/cpython/commit/75bb07e92baa7267a61056d03d7e6b475588e793


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32954] Lazy Literal String Interpolation (PEP-498-based fl-strings)

[Mateusz Bysiek]

Change by Mateusz Bysiek :


--
nosy: +mbdevpl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39075] types.SimpleNamespace should preserve attribute ordering (?)

[Raymond Hettinger]

Raymond Hettinger  added the comment:

+1 for dropping the sort.  Also, the repr should be made to round-trip with 
eval().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39082] AsyncMock is unable to correctly patch static or class methods

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

There seems to be a difference in obtaining the attribute out of the __dict__ 
and getattr. patch uses __dict__ [0] to access the attribute to be patched and 
falls back to getattr. There is a difference in detecting normal attribute 
access to be a coroutine function versus the one obtained from __dict__ . We 
can perhaps make _is_async_obj [1] to see if the passed obj is a 
classmethod/staticmethod and to use __func__ to detect a coroutine function.

As a workaround for now you can pass AsyncMock explicitly to patch to return an 
AsyncMock. There was an open issue (issue36092) about patch and 
staticmethods/classmethods but not sure this is related to this.

Retagging it to remove asyncio. Thanks for the report!

# bpo39082.py

from unittest.mock import patch
import inspect

class Helper:

@classmethod
async def async_class_method(cls):
pass

@staticmethod
async def async_static_method(*args, **kwargs):
pass


print("Patching async static method")
async_patcher = patch("__main__.Helper.async_class_method")
print(f"{Helper.async_class_method = }")
print(f"{Helper.__dict__['async_class_method'] = }")
print(f"{inspect.iscoroutinefunction(Helper.async_class_method) = }")
print(f"{inspect.iscoroutinefunction(Helper.__dict__['async_class_method']) = 
}")
print(f"{inspect.iscoroutinefunction(getattr(Helper, 'async_class_method')) = 
}")

mock_ = async_patcher.start()
print(mock_)

print("\nPatching async static method")
async_patcher = patch("__main__.Helper.async_static_method")
print(f"{Helper.async_static_method = }")
print(f"{Helper.__dict__['async_static_method'] = }")
print(f"{inspect.iscoroutinefunction(Helper.async_static_method) = }")
print(f"{inspect.iscoroutinefunction(Helper.__dict__['async_static_method']) = 
}")
print(f"{inspect.iscoroutinefunction(getattr(Helper, 'async_static_method')) = 
}")

mock_ = async_patcher.start()
print(mock_)


$ python3.8 bpo39082.py
Patching async static method
Helper.async_class_method = >
Helper.__dict__['async_class_method'] = 
inspect.iscoroutinefunction(Helper.async_class_method) = True
inspect.iscoroutinefunction(Helper.__dict__['async_class_method']) = False
inspect.iscoroutinefunction(getattr(Helper, 'async_class_method')) = True


Patching async static method
Helper.async_static_method = 
Helper.__dict__['async_static_method'] = 
inspect.iscoroutinefunction(Helper.async_static_method) = True
inspect.iscoroutinefunction(Helper.__dict__['async_static_method']) = False
inspect.iscoroutinefunction(getattr(Helper, 'async_static_method')) = True



Detect __func__ to be used for iscoroutinefunction

diff --git Lib/unittest/mock.py Lib/unittest/mock.py
index cd5a2aeb60..572468ca8d 100644
--- Lib/unittest/mock.py
+++ Lib/unittest/mock.py
@@ -48,6 +48,8 @@ _safe_super = super
 def _is_async_obj(obj):
 if _is_instance_mock(obj) and not isinstance(obj, AsyncMock):
 return False
+if hasattr(obj, '__func__'):
+obj = getattr(obj, '__func__')
 return asyncio.iscoroutinefunction(obj) or inspect.isawaitable(obj)

[0] 
https://github.com/python/cpython/blob/50d4f12958bf806a4e1a1021d70cfd5d448c5cba/Lib/unittest/mock.py#L1387
[1] 
https://github.com/python/cpython/blob/50d4f12958bf806a4e1a1021d70cfd5d448c5cba/Lib/unittest/mock.py#L48

--
components: +Library (Lib) -Tests, asyncio

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39076] Use types.SimpleNamespace for argparse.Namespace

[Raymond Hettinger]

Raymond Hettinger  added the comment:

There are some sticking points:

* types.SimpleNamespace() sorts attributes, so this would get in the way of 
issue #39058.

* argparse.Namespace() supports a __contains__() method that isn't offered by 
types.SimpleNamespace():

>>> 'abc' in Namespace(abc=10)
True

* Argparse is sensitive to start-up time so we mostly want to avoid adding new 
dependencies.  Start-up time recently got worse when the re module added a 
number of dependencies.

* The __repr__ for SimpleNamespace() doesn't round-trip and isn't what we have 
now with Namespace.  That potentially breaks doctests and diverges from 
published examples:

>>> Namespace(abc=10)
Namespace(abc=10)
>>> SimpleNamespace(abc=10)
namespace(abc=10)

* Ironically, the class name "Namespace" is simpler than "SimpleNamespace" ;-)

* Much of the code in argparse.Namespace() inherits from _AttributeHolder, so 
switching to types.SimpleNamespace() doesn't really save us much code.

Are there any upsides to switching?  Attribute lookup is almost equally fast 
using either approach, so there is no speed benefit:

$ python3.8 -m timeit -r 11 -s 'from argparse import Namespace as NS' -s 
'args=NS(abc=10)' 'args.abc'
1000 loops, best of 11: 32.7 nsec per loop

$ python3.8 -m timeit -r 11 -s 'from types import SimpleNamespace as NS' -s 
'args=NS(abc=10)' 'args.abc'
1000 loops, best of 11: 32.4 nsec per loop

--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39085] Improve docs for await expression

[Kyle Stanley]

New submission from Kyle Stanley :

For context, I decided to open this issue after receiving a substantial volume 
of very similar questions and misconceptions from users of asyncio and trio 
about what `await` does, mostly within a dedicated "async" topical help chat 
(in the "Python Discord" community). For the most part, the brief explanation 
provided in the language reference docs 
(https://docs.python.org/3/reference/expressions.html#await-expression) did not 
help to clear up their understanding.

Also, speaking from personal experience, I did not have a clear understanding 
of what `await` actually did until I gained some experience working with 
asyncio. When I read the language reference definition for the await expression 
for the first time, it did not make much sense to me either.

As a result, I think the documentation for the `await` expression could be made 
significantly more clear. To users that are already familiar with asynchronous 
programming it likely makes more sense, but I don't think it's as helpful as it 
could be for those who are trying to fundamentally understand how `await` works 
(without having prior experience):

"Suspend the execution of coroutine on an awaitable object. Can only be used 
inside a coroutine function."

(https://docs.python.org/3/reference/expressions.html#await-expression)

(Also, note that there's a typo in the current version, "of coroutine" should 
probably be "of a coroutine")

While this explanation is technically accurate, it also looks to be the 
_shortest_ one out of all of the defined expressions on the page. To me, this 
doesn't seem right considering that the await expression is not the easiest one 
to learn or understand.

The vast majority of the questions and misunderstandings on `await` that I've 
seen typically fall under some variation of one of the following:

1) What exactly is being suspended?
2) When is it resumed/unsuspended?
3) How is it useful?

>From what I can tell, (1) is unclear to them is partly because the awaitable 
>object that is after the `await` can be a coroutine object. It's not at all 
>uncommon to see "await some_coro()".

I think this would be much more clear if it were to instead be something along 
the lines of one the following (changes indicated with *):

1) "Suspend the execution of *the current coroutine function* on an awaitable 
object. Can only be used inside a coroutine function."

Where "the current coroutine function" is the coroutine function that contains 
the await expression. I think this would help to clear up the first question, 
"What exactly is being suspended?".

2) "Suspend the execution of *the current coroutine function* on an awaitable 
object. *The coroutine function is resumed when the awaitable object is 
completed and returns its result*. Can only be used inside a coroutine 
function."

This would likely help to clear up "When is it resumed/unsuspended?".

Optimally, this definition could also include some form of example code like 
several of the other expressions have. It's not particularly easy to use a 
demonstrable example without using an async library (such as asyncio), but 
using a specific async library would not make sense to have in this location of 
the docs because the language reference is supposed to be as implementation 
agnostic as possible.

However, I think a very brief visual example with some explanation could still 
be useful for explaining the basics of how await works:

3) ```
async def coro():
# before await
await some_awaitable
# after await

When the coroutine function `coro()` is executed, it will behave roughly the 
same as any subroutine function in the "before await" section. However, upon 
reaching `await some_awaitable`, the execution of `coro()` will be suspended on 
`some_awaitable`, preventing the execution of anything in the "after await" 
section until `some_awaitable` is completed. This process is repeated with 
successive await expressions. Also, multiple coroutines can be suspended at the 
same time.

Suspension can be used to indicate that other coroutines can be executed in the 
meantime. This can be used to write asynchronous and concurrent programs 
without the usage of callbacks.
```

Including the brief example and explanation would likely help to further clear 
up all three of the questions.

The present version has a high degree of technical accuracy, but I don't think 
its as helpful as it could be for furthering the understanding of users or 
providing an introduction to the await expression. I'm sure that there will 
still be some questions regarding `await` even if any of these changes are 
made, but it would at least provide a good place to link to for an informative 
explanation of `await` that's entirely agnostic from any specific 
implementation.

I'm entirely open to any alternative suggestions, or making a change that's 
some combination or variation of the above three ideas. Alternatively, if there 
are determined

[issue39082] AsyncMock is unable to correctly patch static or class methods

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Using patch as a function just returns a patch object. You need to call start 
method on the patch object to return a Mock.

--
nosy: +lisroach, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39040] Wrong attachement filename when mail mime header was too long

[Abhilash Raj]

Abhilash Raj  added the comment:

Sure, fixed as per your comments in the PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39079] help() modifies the string module

[Zachary Ware]

Zachary Ware  added the comment:

Since this has been this way for 22 years without a previous report that I can 
find, +1 to closing as "won't fix"/"not a bug" and so doing.  However, I'm also 
adding the 2.7 release manager to make sure he knows about it and can reopen if 
he wants.

--
nosy: +benjamin.peterson, zach.ware
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39084] string.letters is flipped after setlocale is called

[Zachary Ware]

Change by Zachary Ware :


--
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed
superseder:  -> help() modifies the string module

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39084] string.letters is flipped after setlocale is called

[Eric V. Smith]

Eric V. Smith  added the comment:

This is a duplicate of issue 39079.

I recommend that we don't "fix" this. It is only an issue with 2.7, and hasn't 
been a problem for years.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39080] Inconsistency with Starred Expression line/col info

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39083] Dictionary get(key, default-expression) not short circuit behavior

[Eric V. Smith]

Eric V. Smith  added the comment:

.get() is just a regular function call. And like all python functions, all of 
the arguments are evaluated before the function is called. There is no 
mechanism in python to delay the evaluation of a arguments.

You might want to look at collections.defaultdict. You can supply a factory 
function, so that the call is delayed until a missing key is found.

--
nosy: +eric.smith
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39084] string.letters is flipped after setlocale is called

[Manish]

Change by Manish :


--
keywords: +patch
pull_requests: +17117
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17650

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39084] string.letters is flipped after setlocale is called

[Manish]

New submission from Manish :

Steps to reproduce:

>>> import string
>>> string.letters
'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
>>> help(string)
..
>>> string.letters
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'


The help(string) can also be replaced with locale.setlocale(locale.LC_CTYPE, 
"en_US.UTF-8")

What's happening here is that any call to setlocale() (which help() calls 
internally) recomputes string.letters. The recomputation flips the order in the 
current implementation.

--
messages: 358604
nosy: Manishearth
priority: normal
severity: normal
status: open
title: string.letters is flipped after setlocale is called
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39080] Inconsistency with Starred Expression line/col info

[miss-islington]

miss-islington  added the comment:


New changeset b1f204471092678dd89117e608fa041a9589d14c by Miss Islington (bot) 
(Pablo Galindo) in branch '3.8':
[3.8] bpo-39080: Starred Expression's column offset fix when inside a CALL 
(GH-17645) (GH-17649)
https://github.com/python/cpython/commit/b1f204471092678dd89117e608fa041a9589d14c


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39083] Dictionary get(key, default-expression) not short circuit behavior

[Martin Meo]

New submission from Martin Meo :

"""
Unexpected behavior report

Dictionary get(key, default-expression) not short circuit behavior

MacOS 10.14.6
sys.version_info(major=3, minor=6, micro=5, releaselevel='final', serial=0)

BACKGROUND
A python dictionary is a data structure that associates a set of keys with a 
set of values.
Accessing a non-existent key produces a KeyError.
Dictionaries have a get() method.
get(key[, default-expression])
Return the value for key if key is in the dictionary, else default-expression.
If default-expression is not given, it defaults to None, so that this method 
never raises a KeyError.

EXPECTED BEHAVIOR
get() would only evaluate default-expression if it has to, when key is not 
found.  It would have short-circuit behavior like boolean operators.

ACTUAL BEHAVIOR
The default-expression DOES get evaluated even when the key IS found in the 
dictionary. And if default-expression is a function call, the function DOES get 
called.

"""

denominations = {0:'zero', 1:'one', 2:'two', 3:'three', 4:'four'}

def foo(n):
print('FOO CALLED. n =', n)
return str(n*10)

words = []
words.append(denominations[1])
words.append(denominations[4])
words.append(denominations.get(1))
words.append(denominations.get(4))
words.append(denominations.get(1, 'ERROR-A'))
words.append(denominations.get(4, 'ERROR-B'))

words.append(denominations.get(22, 'ERROR-1'))
words.append(denominations.get(88, 'ERROR-2'))

words.append(denominations.get(1, foo(1)))
words.append(denominations.get(4, foo(4)))

print(words)



def isItZero(n):
print('ISITZERO CALLED.  n=', n)
return False

a = (True or isItZero(9))   # (True or x) is always True so x is not evaluated

--
messages: 358602
nosy: martinmeo
priority: normal
severity: normal
status: open
title: Dictionary get(key, default-expression) not short circuit behavior
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39082] AsyncMock is unable to correctly patch static or class methods

[Aniket Panse]

New submission from Aniket Panse :

Currently, patch is unable to correctly patch coroutinefunctions decorated with 
`@staticmethod` or `@classmethod`.

Example:

```
[*] aniketpanse [~/git/cpython] -> ./python 


±[master]
Python 3.9.0a1+ (heads/master:50d4f12958, Dec 17 2019, 16:31:30) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Helper:
... @classmethod
... async def async_class_method(cls):
... pass
... 
>>> from unittest.mock import patch
>>> patch("Helper.async_class_method")

```

This should ideally return an `AsyncMock()`.

--
components: Tests, asyncio
messages: 358601
nosy: asvetlov, czardoz, yselivanov
priority: normal
severity: normal
status: open
title: AsyncMock is unable to correctly patch static or class methods
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39080] Inconsistency with Starred Expression line/col info

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +17116
pull_request: https://github.com/python/cpython/pull/17649

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39080] Inconsistency with Starred Expression line/col info

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 50d4f12958bf806a4e1a1021d70cfd5d448c5cba by Pablo Galindo 
(Lysandros Nikolaou) in branch 'master':
bpo-39080: Starred Expression's column offset fix when inside a CALL (GH-17645)
https://github.com/python/cpython/commit/50d4f12958bf806a4e1a1021d70cfd5d448c5cba


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Eric V. Smith]

Eric V. Smith  added the comment:

I agree that a quick glance in the rear view mirror shows that the design isn't 
awesome. But I just don't see how we can change it at this point. It is what it 
is.

And it's no surprise that int and float have the same behavior: they share the 
same code in Python/formatter_unicode.h. Which means that even if we did want 
to change one but not the other, we'd have to duplicate a lot of code and live 
with the maintenance hassle forever.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39081] pathlib '/' operator does not resolve Enums with str mixin as expected

[Andrew Ni]

New submission from Andrew Ni :

import os
import pathlib
import enum

class MyEnum(str, enum.Enum):
RED = 'red'

# this resolves to: '/Users/niandrew/MyEnum.RED'
# EXPECTED: '/Users/niandrew/red'
str(pathlib.Path.home() / MyEnum.RED)

# this resolves to: '/Users/niandrew/red'
os.path.join(pathlib.Path.home(), MyEnum.RED)

--
components: Library (Lib)
messages: 358598
nosy: andrewni
priority: normal
severity: normal
status: open
title: pathlib '/' operator does not resolve Enums with str mixin as expected
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39074] Threading memory leak in _shutdown_locks for non-daemon threads

[Adam]

Adam  added the comment:

Looks like this issue might be a duplicate of https://bugs.python.org/issue37788

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39074] Threading memory leak in _shutdown_locks for non-daemon threads

[Adam]

Change by Adam :


--
type: resource usage -> security

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39079] help() modifies the string module

[Zectbumo]

Zectbumo  added the comment:

@eric.smith
It actually caused a problem and then turned into a curiosity. At the beginning 
I wanted to remind myself about the string module so I did a help(string). Then 
right after that I wrote a program that generated filenames from 
string.letters. I closed the python interpreter down and then reopened it and 
regenerated filenames again using string.letters. To my surprise I found the 
second iteration to come out with completely different results messing up my 
work. Once I found that help() was causing this problem and it became a 
curiosity because you would think help() would not modify anything. Obviously 
this problem is easy to work around in my case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Michael Amrhein]

Michael Amrhein  added the comment:

>
> ... Has anyone checked what C does?
>

#include 
int main() {
int i = -12345;
double f = -12345.0;
printf("%-020d\n", i);
printf("%020d\n", i);
printf("%20d\n", i);
printf("%-020f\n", f);
printf("%020f\n", f);
printf("%20f\n", f);
return 0;
}

Output:

-12345 
-0012345
  -12345
-12345.00  
-00012345.00
   -12345.00

https://en.cppreference.com/w/c/io/fprintf:

Each conversion specification has the following format:

  introductory % character 

  (optional) one or more flags that modify the behavior of the conversion: 

-: the result of the conversion is left-justified within the field (by 
default it is right-justified)
+: the sign of signed conversions is always prepended to the result of the 
conversion (by default the result is preceded by minus only when it is negative)
space: if the result of a signed conversion does not start with a sign 
character, or is empty, space is prepended to the result. It is ignored if + 
flag is present.
# : alternative form of the conversion is performed. See the table below 
for exact effects otherwise the behavior is undefined.
0 : for integer and floating point number conversions, leading zeros are 
used to pad the field instead of space characters. For integer numbers it is 
ignored if the precision is explicitly specified. For other conversions using 
this flag results in undefined behavior. It is ignored if - flag is present. 

Last sentence means that zero-padding is only done when the output is 
right-aligned. I can't find an equivalent for Pythons '=' align option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39079] help() modifies the string module

[Mark Dickinson]

Mark Dickinson  added the comment:

There's no reasonable way I can see to fix this. The reassignment of those 
string attributes is clearly intentional (it's even documented) and there's 
probably code somewhere that relies on it. I think the best we can do is close 
as "won't fix" though "not a bug" may be more accurate: the code is clearly 
behaving as intended and designed, even if the design is questionable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38546] test_concurrent_futures: reap_children() warnings on RHEL7 and RHEL8 buildbots

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +17115
pull_request: https://github.com/python/cpython/pull/17647

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13601] sys.stderr should be line-buffered when stderr is not a TTY

[Jendrik Seipp]

Jendrik Seipp  added the comment:

I took the liberty of increasing the target version. It would be great if 
someone could review my patch for this issue at 
https://github.com/python/cpython/pull/17646 .

--
nosy: +jendrik
versions: +Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36500] Add "regen-*" equivalent projects for Windows builds

[Steve Dower]

Steve Dower  added the comment:


New changeset a9d0a6a1b932752873e04714c5dda707729078e4 by Steve Dower in branch 
'master':
bpo-36500: Simplify PCbuild/build.bat and prevent path separator changing in 
comments (GH-17644)
https://github.com/python/cpython/commit/a9d0a6a1b932752873e04714c5dda707729078e4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36500] Add "regen-*" equivalent projects for Windows builds

[Steve Dower]

Change by Steve Dower :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.9 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13601] sys.stderr should be line-buffered when stderr is not a TTY

[Jendrik Seipp]

Change by Jendrik Seipp :


--
keywords: +patch
pull_requests: +17114
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/17646

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39058] argparse should preserve argument ordering in Namespace

[paul j3]

paul j3  added the comment:

This patch changes the super class, _AttributeHolder.  ArgumentParser and 
Actions also inherit from this, though they have their own _get_kwargs methods, 
and so aren't affected by the sort and its removal.

I just had occasion on stackoverflow to discuss the order in which attributes 
are added.  A poster wanted to preserve the sys.argv order.

https://stackoverflow.com/questions/58904423/find-the-order-of-arguments-in-argparse-python3/58905067#58905067

Most attributes are added as defaults at the start of parsing - via a loop 
through parser._actions.  Predefining the namespace, SUPPRESS defaults, 
parser.set_defaults may alter this default order.

Anyways removing the sort makes sense, and the proposed change phrase "in the 
order attributes were added" is sufficiently general.

--
nosy: +paul.j3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Mark Dickinson]

Mark Dickinson  added the comment:

[Eric]

> Is that the "0" here: [...]

Yes, sorry; that could have been clearer.

> In particular, what if you specify a different alignment type with the 
> pre-width 0?

Right, that's the critical question here. For floats and ints, an 
explicitly-specified alignment type overrides the implicit "=". But Decimal 
raises. The Decimal behaviour seems more reasonable, but the float and int 
behaviours are more historically baked-in, and riskier to change.

And then there's a parallel question with the fill character: should an 
explicitly-given fill override the "0"-fill character that's implicit in that 
"[0]"? int and float say "yes". Decimal says "ValueError".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39079] help() modifies the string module

[Mark Dickinson]

Mark Dickinson  added the comment:

For completeness, here's the path leading to the reassignment (after hacking in 
a raise to the appropriate place in _localemodule.c). It's a side-effect of 
calling `locale.getpreferredencoding`.

>>> help(int)
Traceback (most recent call last):
  File "", line 1, in 
  File "/Users/mdickinson/GitHub/python/cpython/Lib/site.py", line 445, in 
__call__
import pydoc
  File "/Users/mdickinson/GitHub/python/cpython/Lib/pydoc.py", line 202, in 

_encoding = locale.getpreferredencoding()
  File "/Users/mdickinson/GitHub/python/cpython/Lib/locale.py", line 616, in 
getpreferredencoding
setlocale(LC_CTYPE, "")
  File "/Users/mdickinson/GitHub/python/cpython/Lib/locale.py", line 581, in 
setlocale
return _setlocale(category, locale)
ValueError: about to reassign string.letters


So a minimal reproducer is something like:

Python 2.7.17 (default, Oct 20 2019, 14:46:50) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import string, locale
>>> string.letters
'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
>>> locale.getpreferredencoding()
'UTF-8'
>>> string.letters
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39079] help() modifies the string module

[Eric V. Smith]

Eric V. Smith  added the comment:

@Zectbumo: is this causing you any practical problem? Or is it just a curiosity?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Eric V. Smith]

Eric V. Smith  added the comment:

I'm not sure what you mean by the "pre-width 0". Is that the "0" here:

format_spec ::=  
[[fill]align][sign][#][0][width][grouping_option][.precision][type]

?

And now that I write out the question, I'm sure that's what you mean.

PEP 3101 says "If the width field is preceded by a zero ('0') character, this 
enables zero-padding. This is equivalent to an alignment type of '=' and a fill 
character of '0'.". I don't see any other discussion of it in the PEP. In 
particular, what if you specify a different alignment type with the pre-width 0?

I believe this all originated in C's printf, via PEP 3101. Has anyone checked 
what C does?

But in any event, I don't think we can change the int formatting, in 
particular. There's no doubt someone who's relying on every little quirk. I'm 
less concerned about float and decimal, although we'd still need to be very 
careful.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39080] Inconsistency with Starred Expression line/col info

[Lysandros Nikolaou]

Change by Lysandros Nikolaou :


--
keywords: +patch
pull_requests: +17113
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17645

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36500] Add "regen-*" equivalent projects for Windows builds

[Steve Dower]

Steve Dower  added the comment:

Adding just a couple of minor touch-ups in PR 17644, but otherwise thanks 
Anthony!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39025] Windows Python Launcher does not update PATH to Scripts directory

[Bluebird]

Bluebird  added the comment:

Ok, I'll close my request if you do not think of any value in it. 

Py is such an elegant solution, I wish something as simple would also address 
this problem.

Virtual environement do have the downside of not coming directly with Python. 
In my work environment, extra Python packages are strongly discouraged and my 
colleagues are not proficient enough in the Python ecosystem to use venv.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39076] Use types.SimpleNamespace for argparse.Namespace

[paul j3]

Change by paul j3 :


--
nosy: +paul.j3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36500] Add "regen-*" equivalent projects for Windows builds

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +17112
pull_request: https://github.com/python/cpython/pull/17644

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39080] Inconsistency with Starred Expression line/col info

[Lysandros Nikolaou]

New submission from Lysandros Nikolaou :

When a starred expression like *[0, 1] is parsed, the following AST gets 
generated:

Module(
body=[
Expr(
value=Starred(
value=List(
elts=[
Constant(
value=0,
kind=None,
lineno=1,
col_offset=2,
end_lineno=1,
end_col_offset=3,
),
Constant(
value=1,
kind=None,
lineno=1,
col_offset=5,
end_lineno=1,
end_col_offset=6,
),
],
ctx=Load(),
lineno=1,
col_offset=1,
end_lineno=1,
end_col_offset=7,
),
ctx=Load(),
lineno=1,
col_offset=0,
end_lineno=1,
end_col_offset=7,
),
lineno=1,
col_offset=0,
end_lineno=1,
end_col_offset=7,
)
],
type_ignores=[],
)


But, when a starred expression is an argument to a function call then the 
line/col info are wrong (end_col_offset is always equal to col_offset + 1):

Module(
body=[
Expr(
value=Call(
func=Name(
id="f", ctx=Load(), lineno=1, col_offset=0, end_lineno=1, 
end_col_offset=1
),
args=[
Starred(
value=List(
elts=[
Constant(
value=0,
kind=None,
lineno=1,
col_offset=4,
end_lineno=1,
end_col_offset=5,
),
Constant(
value=1,
kind=None,
lineno=1,
col_offset=7,
end_lineno=1,
end_col_offset=8,
),
],
ctx=Load(),
lineno=1,
col_offset=3,
end_lineno=1,
end_col_offset=9,
),
ctx=Load(),
lineno=1,
col_offset=2,
end_lineno=1,
end_col_offset=9,
)
],
keywords=[],
lineno=1,
col_offset=0,
end_lineno=1,
end_col_offset=10,
),
lineno=1,
col_offset=0,
end_lineno=1,
end_col_offset=10,
)
],
type_ignores=[],
)

--
components: Interpreter Core
messages: 358584
nosy: lys.nikolaou
priority: normal
severity: normal
status: open
title: Inconsistency with Starred Expression line/col info
type: behavior
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Mark Dickinson]

Mark Dickinson  added the comment:

> Regarding cases 3-7 I'd like to suggest a slightly different resolution:

Hmm, yes. I was concentrating on the Decimal results, but I agree that these 
int/float results are disturbing:

>>> format(12345, "<020")
'12345000'
>>> format(12345.0, "<020")
'12345.00'
>>> format(12345, "^020")
'00012345'

I'm fine with an explicit *fill* character of zero producing misleading 
results; the user just gets what they ask for in that case. (And the filling 
could be happening in generic code that isn't aware of the numeric context any 
more, so it could be tricky to change.)

But having the pre-width 0 be interpreted this way is questionable. Eric: 
thoughts?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Michael Amrhein]

Michael Amrhein  added the comment:

Mark, to answer your question regarding the two implementations of Decimals:
No, in the cases I stated above there are no differences in their behaviour.

In order to dig a bit deeper, I wrote a little test:

d = cdec.Decimal("1234567890.1234")
p = pydec.Decimal("1234567890.1234")
for fill in ('', ' ', '_'):
for align in ('', '<', '>', '^', '='):
for sign in ('', '-', '+', ' '):
for zeropad in ('', '0'):
for min_width in ('', '25'):
for thousands_sep in ('', ','):
for precision in ('', '.3', '.5'):
for ftype in ('', 'e', 'f', 'g'): 
fmt = 
f"{fill}{align}{sign}{zeropad}{min_width}{thousands_sep}{precision}{ftype}"
try:
df = format(d, fmt)
except ValueError:
df = ""
try:
pf = format(p, fmt)
except ValueError:
pf = ""
if df != pf:
print(fmt, df, pf)

It did not reveal any differences. The two implementations are equivalent 
regarding the tested combinations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39079] help() modifies the string module

[Mark Dickinson]

Mark Dickinson  added the comment:

> But given that there's only 2 weeks of support left for 2.7, I don't see this 
> getting changed.

Agreed: I can't reproduce on Python 3, and it looks as though the offending 
code is gone from the codebase in Python 3, so this is pretty much just a 
historical oddity at this point.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36500] Add "regen-*" equivalent projects for Windows builds

[Steve Dower]

Steve Dower  added the comment:


New changeset 9e36589d49c1d6b06c0239fa69e8274d7e89e375 by Steve Dower (Anthony 
Shaw) in branch 'master':
bpo-36500: Add --regen option to PCbuild/build.bat so Windows users can regen 
grammar, opcodes, tokens and symbols (GH-12654)
https://github.com/python/cpython/commit/9e36589d49c1d6b06c0239fa69e8274d7e89e375


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39079] help() modifies the string module

[Mark Dickinson]

Mark Dickinson  added the comment:

Here's where `string.letters` is reassigned: 
https://github.com/python/cpython/blob/5f2c1345a79f205c680ed6e0a6ed44199546d79e/Modules/_localemodule.c#L136-L147

That code looks like it's exercised whenever setlocale is (or possibly just 
when LC_CTYPE is changed). I haven't yet figured out which part of the help 
machinery causes that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39079] help() modifies the string module

[Eric V. Smith]

Eric V. Smith  added the comment:

With 2.7.16 on cygwin, I get:

>>> import string
>>> string.letters
'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
>>> help(int)
>>> string.letters
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'

But given that there's only 2 weeks of support left for 2.7, I don't see this 
getting changed.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Michael Amrhein]

Michael Amrhein  added the comment:

Mark, I mostly agree with your classifications / proposals.
Regarding cases 3-7 I'd like to suggest a slightly different resolution:
Following my interpretation of the spec ("use zeropad *only* if no align is 
given"), "<020", ">020", "^020" and "=020" would be treated equivalent to 
"<20", ">20", "^20" and "=20":

format(12345, "<020") -> '-12345  ', not '-1234500'
format(12345, ">020") -> '  -12345', not '00-12345'
format(12345, "^020") -> '   -12345   ', not '000-12345000'
format(12345, "=020") -> '-  12345', not '-0012345'

For '<', '>' and '^' I can't imagine any code depending on the current 
behaviour of int and float, so this change is unlikely to break anything. 
For '=' it might be reasonable to make an exception (and state it in the doc), 
so that "020", "=020", "0=020" and "0=20" are treated as equivalent.
For Decimal this would mean to loosen the behaviour, as you proposed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39079] help() modifies the string module

[Mark Dickinson]

Mark Dickinson  added the comment:

I *can* reproduce, on macOS 10.14.6, Python 2.7.17 (from macports).

Python 2.7.17 (default, Oct 20 2019, 14:46:50) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import string
>>> string.letters
'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
>>> help(int)  # brings up pager; page through to the end

>>> string.letters
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39078] __function.__defaults__ breaks for __init__ of dataclasses with default factory

[Eric V. Smith]

Eric V. Smith  added the comment:

I guess I could make the default value something like _CALLABLE(dict), but I'm 
not sure that would do you any good. But at least you could tell from the repr 
what it is.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39079] help() modifies the string module

[Steven D'Aprano]

Steven D'Aprano  added the comment:

I am unable to reproduce this.

Can you double check that this occurs in the plain Python interpreter or IDLE, 
and provide a minimum reproducible example? (No third-party IDEs or 
interpreters, such as Jupyter.)

These guidelines may help:

https://en.wikipedia.org/wiki/Minimal_working_example

http://sscce.org/

https://stackoverflow.com/help/minimal-reproducible-example

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39040] Wrong attachement filename when mail mime header was too long

[R. David Murray]

R. David Murray  added the comment:

One more tweak to the test and we'll be good to go.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39073] email incorrect handling of crlf in Address objects.

[R. David Murray]

R. David Murray  added the comment:

Hmm.  Yes, \r\n should be disallowed in the arguments to Address.  I thought it 
already was, so that's a bug.  That bug produces the other apparent bug as 
well: because the X: was treated as a separate line, the previous header did 
not need double quotes so they are no longer added.

So there's no 3.8 specific bug here, but there is a bug.

--
title: email regression in 3.8: folding -> email incorrect handling of crlf in 
Address objects.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39071] email.parser.BytesParser - parse and parsebytes work not equivalent

[R. David Murray]

R. David Murray  added the comment:

All of which isn't to discount that you might have a found a bug, by the way, 
if you want to investigate further :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39071] email.parser.BytesParser - parse and parsebytes work not equivalent

[R. David Murray]

R. David Murray  added the comment:

The problem is that you are starting with different inputs.  unicode strings 
and bytes are different things, and so parsing them can produce different 
results.  The fact of that matter is that email messages are defined to be 
bytes, so parsing a unicode string pretending it is an email message is just 
asking for errors anyway.  The string parsing methods are really only provided 
for backward compatibility and historical reasons.

I thought this was clear from the existing documentation, but clearly it isn't 
:)  I'll review a suggested doc change, but the thing to explain is not that 
parse and parsebytes might produce different results, but that parsing email 
from strings is not a good idea and will likely produce unexpected results for 
anything except the simplest non-mime messages.

Note: the reason you got different checksums might have had to do with line 
ends, depending on how you calculated the checksums.  You should also consider 
using get_content and not get_payload.  get_payload has a weird legacy API that 
doesn't always do what you think it will, and that might be another source of 
checksum issues.  But really, parsing a unicode representation of a mime 
message is just likely to be buggy.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39070] Uninstalling 3.8.0 fails but it says it succeeds..

[Steve Dower]

Steve Dower  added the comment:

Hi tuijatuulia

In your %TEMP% directory there should be a few sets of log files for Python. 
Would you be able to zip them up and attach them to this issue? That will help 
us figure out what hasn't worked for you.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39078] __function.__defaults__ breaks for __init__ of dataclasses with default factory

[Eric V. Smith]

Eric V. Smith  added the comment:

The problem is that __init__ has to have a sentinel to know whether or not to 
call the default value. If the default were just "dict", should it call it, or 
is the passed in value really dict, in which case it doesn't get called?

dataclass()
class Test:
a: int
b: Dict[Any, Any] = field(default_factory=dict)

The generated __init__ looks like:

def __init__(self, a, b=_HAS_DEFAULT_FACTORY):
   self.a = a
   self.b = dict() if b is _HAS_DEFAULT_FACTORY else b

If it were:

def __init__(self, a, b=dict):
   self.a = a

Then what would the assignment to self.b look like? What if you instantiated an 
object as Test(0, dict)? You wouldn't want dict to get called. You need to 
differentiate between Test(0, dict) and Test(0). The former does not call b, 
but the latter does call b.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Mark Dickinson]

Mark Dickinson  added the comment:

Thanks for the report. I think most of this is a documentation issue: we either 
need to make clear that the formatting documentation applies only to the float 
type and that Decimal follows its own rules (usually for good reason, namely 
that it's required to follow Mike Cowlishaw's General Decimal Arithmetic 
Specification), or adjust the main string formatting documentation to make sure 
it covers the Decimal type as well as float.

Michael: thank you for including both the _pydecimal and decimal results here. 
Just to double check, I'm not seeing any differences between just those two 
here (other than the exact exception messages); are you seeing differences just 
between those two types that you think shouldn't exist?

In more detail:

Cases 1-2: these look like a documentation issue. The decimal behaviour is 
definitely desirable here: when possible, we want the string representation to 
preserve the information about the decimal exponent.

Case 3: This looks like a bug in the float formatting to me: the "0" prefix for 
the width implies "=" for the alignment, which conflicts with the explicit ">". 
But making something that previously worked a ValueError is hazardous; perhaps 
we can accept that in this case the explicitly-given alignment overrides the 
implicit "=", and modify decimal accordingly.

Case 4: This is a weaker case than case 3, where the implicit alignment matches 
the explicitly given one; I think we should fix decimal to accept this, since 
there's no ambiguity.

Case 5: Same reasoning as case 4: let's fix decimal.

Case 6: Like case 3: there's a conflict between the implicit fill of "0" and 
the explicitly given fill of " ". Again, it seems reasonable to let the 
explicit win over the implicit here.

Case 7: like case 3 and case 6 combined; if we fix those, we might as well also 
fix this one for consistency, even though at that point the "0" prefix for the 
width is doing nothing at all.

Cases 8-10: the space in case 8 isn't being interpreted as a fill character 
here; it's being interpreted as the sign character. I don't think there's 
anything to fix for these cases.

Cases 11-12: I don't think there's anything to be fixed here: yes, padding on 
the right with zeros creates misleading results, but I don't think it's 
Python's job to prevent the user from doing that.

Case 13: This is a doc issue; that

--
nosy: +facundobatista, rhettinger, skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Mark Dickinson]

Change by Mark Dickinson :


--
Removed message: https://bugs.python.org/msg358567

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39071] email.parser.BytesParser - parse and parsebytes work not equivalent

[Manfred Kaiser]

Manfred Kaiser  added the comment:

I think, the best way is to fix the documentation. The reason is, when a 
developer rely to the behavior of the function but the behavior is changed, a 
program may work incorrect.

Just think about forensic stuff. If a hash value will be created with the 
"parsebytes" method and the behavior will be changed to match the behavior of 
the "parse" method, the the evidence can not be validated with the latest 
python versions.

We could add a comment to the documentation. For example "parsebytes parses the 
mail in a different way than parse, which may produce slightly different 
messages. If you rely on the same behavior for file and byte like objects you 
can use the parse method with BytesIO"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Mark Dickinson]

Mark Dickinson  added the comment:

Thanks for the report. I think most of this is a documentation issue: we either 
need to make clear that the formatting documentation applies only to the float 
type and that Decimal follows its own rules (usually for good reason, namely 
that it's required to follow Mike Cowlishaw's General Decimal Arithmetic 
Specification), or adjust the main string formatting documentation to make sure 
it covers the Decimal type as well as float.

Michael: thank you for including both the _pydecimal and decimal results here. 
Just to double check, I'm not seeing any differences between just those two 
here (other than the exact exception messages); are you seeing differences just 
between those two types that you think shouldn't exist?

In more detail:

Cases 1-2: these look like a documentation issue. The decimal behaviour is 
definitely desirable here: when possible, we want the string representation to 
preserve the information about the decimal exponent.

Case 3: This looks like a bug in the float formatting to me: the "0" prefix for 
the width implies "=" for the alignment, which conflicts with the explicit ">". 
But making something that previously worked a ValueError is hazardous; perhaps 
we can accept that in this case the explicitly-given alignment overrides the 
implicit "=", and modify decimal accordingly.

Case 4: This is a weaker case than case 3, where the implicit alignment matches 
the explicitly given one; I think we should fix decimal to accept this, since 
there's no ambiguity.

Case 5: Same reasoning as case 4: let's fix decimal.

Case 6: Like case 3: there's a conflict between the implicit fill of "0" and 
the explicitly given fill of " ". Again, it seems reasonable to let the 
explicit win over the implicit here.

Case 7: like case 3 and case 6 combined; if we fix those, we might as well also 
fix this one for consistency, even though at that point the "0" prefix for the 
width is doing nothing at all.

Cases 8-10: the space in case 8 isn't being interpreted as a fill character 
here; it's being interpreted as the sign character. I don't think there's 
anything to fix for these cases.

Cases 11-12: I don't think there's anything to be fixed here: yes, padding on 
the right with zeros creates misleading results, but I don't think it's 
Python's job to prevent the user from doing that.

Case 13: This is a doc issue; without a precision, the Decimal output again 
tries to preserve the exponent information, while also ensuring that the value 
is printed in a form that doesn't use the exponent.


So cases 3-7 look like the only ones where we should consider changing the 
behaviour; the issue 17247 that you pointed to proposed tightening the 
behaviour for float to match Decimal, but I think it would be just as 
reasonable to loosen the Decimal behaviour to match float.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39079] help() modifies the string module

[Alfred Morgan]

New submission from Alfred Morgan :

import string
a = string.letters
help(int)
b = string.letters
a == b # False

--
components: Library (Lib)
messages: 358564
nosy: Zectbumo
priority: normal
severity: normal
status: open
title: help() modifies the string module
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Mark Dickinson]

Change by Mark Dickinson :


--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Eric V. Smith]

Change by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39075] types.SimpleNamespace should preserve attribute ordering (?)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

We usually do not iterate all attributes, and dir() always sort attribute names.

--
nosy: +rhettinger, serhiy.storchaka, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39078] __function.__defaults__ breaks for __init__ of dataclasses with default factory

[Eric V. Smith]

Change by Eric V. Smith :


--
assignee:  -> eric.smith
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39078] __function.__defaults__ breaks for __init__ of dataclasses with default factory

[Veith Röthlingshöfer]

New submission from Veith Röthlingshöfer :

When creating a dataclass with a default that is a field with a default 
factory, the factory is not correctly resolved in cls.__init__.__defaults__. It 
evaluates to the __repr__ of dataclasses._HAS_DEFAULT_FACTORY_CLASS, which is 
"".

The expected behavior would be to have a value of whatever the default factory 
produces as a default.

This causes issues for example when using 
inspect.BoundParameters.apply_defaults() on the __init__ of such a dataclass.

Code to reproduce:

```
from dataclasses import dataclass, field
from typing import Any, Dict


@dataclass()
class Test:
a: int
b: Dict[Any, Any] = field(default_factory=dict)

print(Test.__init__.__defaults__)  # 
```

The affected packages are on a high-level dataclasses, on a lower level the 
issue is in the builtin __function.__defaults__.

--
components: C API
messages: 358562
nosy: RunOrVeith
priority: normal
severity: normal
status: open
title: __function.__defaults__ breaks for __init__ of dataclasses with default 
factory
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39077] Numeric formatting inconsistent between int, float and Decimal

[Michael Amrhein]

New submission from Michael Amrhein :

The __format__ methods of int, float and Decimal (C and Python implementation) 
do not interpret the Format Specification Mini-Language in the same way:

>>> import decimal as cdec
... cdec.__file__
...
'/usr/lib64/python3.6/decimal.py'
>>> import _pydecimal as pydec
... pydec.__file__
...
'/usr/lib64/python3.6/_pydecimal.py'

>>> i = -1234567890
... f = float(i)
... d = cdec.Decimal(i)
... p = pydec.Decimal(i)
...
>>> # Case 1: no fill, no align, no zeropad
... fmt = "28,"
>>> format(i, fmt)
'  -1,234,567,890'
>>> format(f, fmt)
'-1,234,567,890.0'
>>> format(d, fmt)
'  -1,234,567,890'
>>> format(p, fmt)
'  -1,234,567,890'

>>> # Case 2: no fill, no align, but zeropad
... fmt = "028,"
>>> format(i, fmt)
'-000,000,000,001,234,567,890'
>>> format(f, fmt)
'-0,000,000,001,234,567,890.0'
>>> format(d, fmt)
'-000,000,000,001,234,567,890'
>>> format(p, fmt)
'-000,000,000,001,234,567,890'

>>> # Case 3: no fill, but align '>' + zeropad
... fmt = ">028,"
>>> format(i, fmt)
'00-1,234,567,890'
>>> format(f, fmt)
'-1,234,567,890.0'
>>> format(d, fmt)
ValueError: invalid format string
>>> format(p, fmt)
ValueError: Alignment conflicts with '0' in format specifier: >028,

>>> # Case 4: no fill, but align '=' + zeropad
... fmt = "=028,"
>>> format(i, fmt)
'-000,000,000,001,234,567,890'
>>> format(f, fmt)
'-0,000,000,001,234,567,890.0'
>>> format(d, fmt)
ValueError: invalid format string
>>> format(p, fmt)
ValueError: Alignment conflicts with '0' in format specifier: =028,

>>> # Case 5: fill '0', align '=' + zeropad
... fmt = "0=028,"
>>> format(i, fmt)
'-000,000,000,001,234,567,890'
>>> format(f, fmt)
'-0,000,000,001,234,567,890.0'
>>> format(d, fmt)
ValueError: invalid format string
>>> format(p, fmt)
ValueError: Fill character conflicts with '0' in format specifier: 0=028,

>>> # Case 6: fill ' ', align '=' + zeropad
... fmt = " =028,"
>>> format(i, fmt)
'-  1,234,567,890'
>>> format(f, fmt)
'-1,234,567,890.0'
>>> format(d, fmt)
ValueError: invalid format string
>>> format(p, fmt)
ValueError: Fill character conflicts with '0' in format specifier:  =028,

>>> # Case 7: fill ' ', align '>' + zeropad
... fmt = " >028,"
>>> format(i, fmt)
'  -1,234,567,890'
>>> format(f, fmt)
'-1,234,567,890.0'
>>> format(d, fmt)
ValueError: invalid format string
>>> format(p, fmt)
ValueError: Fill character conflicts with '0' in format specifier:  >028,

>>> # Case 8: fill ' ', no align, but zeropad
... fmt = " 028,"
>>> format(i, fmt)
'-000,000,000,001,234,567,890'
>>> format(f, fmt)
'-0,000,000,001,234,567,890.0'
>>> format(d, fmt)
'-000,000,000,001,234,567,890'
>>> format(p, fmt)
'-000,000,000,001,234,567,890'

>>> # Case 9: fill '_', no align, but zeropad
... fmt = "_028,"
>>> format(i, fmt)
ValueError: Invalid format specifier
>>> format(f, fmt)
ValueError: Invalid format specifier
>>> format(d, fmt)
ValueError: invalid format string
>>> format(p, fmt)
ValueError: Invalid format specifier: _028,

>>> # Case 10: fill '_', no align, no zeropad
... fmt = "_28,"
>>> format(i, fmt)
ValueError: Invalid format specifier
>>> format(f, fmt)
ValueError: Invalid format specifier
>>> format(d, fmt)
ValueError: Invalid format string
>>> format(p, fmt)
ValueError: Invalid format specifier: _28,

>>> # Case 11: fill '0', align '>', no zeropad
... fmt = "0>28,"
>>> format(i, fmt)
'00-1,234,567,890'
>>> format(f, fmt)
'-1,234,567,890.0'
>>> format(d, fmt)
'00-1,234,567,890'
>>> format(p, fmt)
'00-1,234,567,890'

>>> # Case 12: fill '0', align '<', no zeropad
... fmt = "0<28,"
>>> format(i, fmt)
'-1,234,567,89000'
>>> format(f, fmt)
'-1,234,567,890.0'
>>> format(d, fmt)
'-1,234,567,89000'
>>> format(p, fmt)
'-1,234,567,89000'

>>> # Case 13: fixed-point notation w/o precision
... fmt = "f"
>>> format(f, fmt)
'-1234567890.00'
>>> format(d, fmt)
'-1234567890'
>>> format(p, fmt)
'-1234567890'

Case 1 & 2:
For a format string not giving a type ("None") the spec says: "Similar to 'g', 
except that fixed-point notation, when used, has at least one digit past the 
decimal point." float does follow this rule, Decimal does not.
While this may be regarded as reasonable, it should be noted in the doc. 

Cases 3 to 7:
Both implementations of Decimal do not allow to combine align and zeropad, 
while int and float do. When also fill is given, int and float ignore zeropad, 
but use '0' instead of ' ' (default), if not. 
(For an exception see the following case.)
The spec says: "When no explicit alignment is given, preceding the width field 
by a zero ('0') character enables sign-aware zero-padding for numeric types. 
This is equivalent to a fill character of '0' with an alignment type of '='." 
That does not explicitly give a rule for align + zeropad together, but IMHO it 
suggests to use zeropad *only* if no

[issue38546] test_concurrent_futures: reap_children() warnings on RHEL7 and RHEL8 buildbots

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9707e8e22d80ca97bf7a9812816701cecde6d226 by Victor Stinner in 
branch 'master':
bpo-38546: multiprocessing tests stop the resource tracker (GH-17641)
https://github.com/python/cpython/commit/9707e8e22d80ca97bf7a9812816701cecde6d226


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38904] "signal only works in main thread" in main thread

[Richard Warfield]

Richard Warfield  added the comment:

I think so, yes.

On Wed, Dec 18, 2019 at 1:10 AM Eric Snow  wrote:

>
> Eric Snow  added the comment:
>
> So resolving issue39042 would be enough, particularly if we backported
> the change to 3.8?
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38904] "signal only works in main thread" in main thread

[Eric Snow]

Eric Snow  added the comment:

So resolving issue39042 would be enough, particularly if we backported
the change to 3.8?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39042] Use the runtime's main thread ID in the threading module.

[Eric Snow]

Eric Snow  added the comment:

Hmm, I wonder if this should be considered a regression in 3.8.  As 
demonstrated in issue38904, the following code changed behavior as of 3.8, 
under certain conditions:


import signal
import threading

def int_handler():
   ...

if threading.current_thread() == threading.main_thread():
signal.signal(signal.SIGINT, int_handler)


Note the specific conditions:

* threading and signal have not been imported yet
* the current thread when the module is *imported* is not the main thread (this 
only affects folks embedding Python)

Also note that the only other help we offer is a "private" function in the 
C-API: _PyOS_IsMainThread().  That would have given the correct answer, but we 
do not expect users to call it, and it doesn't help them from Python code 
anyway.

Also, the same problem existed pre-3.8 if signal and threading were imported in 
different threads before the above script ran.

Finally, I'm (mostly) confident that this would be a backward-compatible fix.

What do you think about this being a 3.8 regression, Łukasz?

--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39075] types.SimpleNamespace should preserve attribute ordering (?)

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39076] Use types.SimpleNamespace for argparse.Namespace

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39058] argparse should preserve argument ordering in Namespace

[Eric Snow]

Eric Snow  added the comment:

FWIW, I've also opened issue #39076 about subclassing types.SimpleNamespace.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39076] Use types.SimpleNamespace for argparse.Namespace

[Eric Snow]

New submission from Eric Snow :

types.SimpleNamespace does pretty much exactly the same thing as 
argparse.Namespace.  We should have the latter subclass the former.  I expect 
the only reason that wasn't done before is because SimpleNamespace is newer.

The only thing argparse.Namespace does differently is it supports the 
contains() builtin.  So the subclass would look like this:

class Namespace(types.SimpleNamespace):
"""..."""
def __contains__(self, key):
return key in self.__dict__

Alternately, we could add __contains__() to SimpleNamespace and then the 
subclass would effectively have an empty body.

--
components: Library (Lib)
messages: 358555
nosy: eric.snow
priority: normal
severity: normal
stage: test needed
status: open
title: Use types.SimpleNamespace for argparse.Namespace
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39033] zipimport raises NameError: name '_boostrap_external' is not defined

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
pull_requests: +17111
pull_request: https://github.com/python/cpython/pull/17642

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39075] types.SimpleNamespace should preserve attribute ordering (?)

[Eric Snow]

Change by Eric Snow :


--
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39058] argparse should preserve argument ordering in Namespace

[Eric Snow]

Eric Snow  added the comment:

> Currently, Namespace() objects sort the attributes in the __repr__.  This is 
> annoying because argument
> order matters and because everywhere else in the module we preserve order 
> (i.e. users see help in the
> order that arguments are added).

Hmm, I was going to suggest switching to types.SimpleNamespace, but
realized we sort that repr too (likely for the same reason).  I've
opened issue #39075 to address that.

In writing up that issue, I considered that a sorted repr can be
useful in some cases.  However, I don't think any of those cases
really apply here.

Anyway, I agree with your conclusion. :)

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39075] types.SimpleNamespace should preserve attribute ordering (?)

[Eric Snow]

New submission from Eric Snow :

types.SimpleNamespace was added in 3.3 (for use in sys.implementation; see PEP 
421), which predates the change to preserving insertion order in dict.  At the 
time we chose to sort the attributes in the repr, both for ease of reading and 
for a consistent output.

The question is, should SimpleNamespace stay as it is (sorted repr) or should 
it show the order in which attributes were added?

On the one hand, alphabetical order can be useful since it makes it easier for 
readers to find attributes, especially when there are many.  However, for other 
cases it is helpful for the repr to show the order in which attributes were 
added.

FWIW, I favor changing the ordering in the repr to insertion-order.  Either is 
relatively trivial to get after the fact (whether "sorted(vars(ns))" or 
"list(vars(ns))"), so I don't think any folks that benefit from alphabetical 
order will be seriously impacted.

--
components: Interpreter Core
messages: 358553
nosy: eric.snow
priority: normal
severity: normal
status: open
title: types.SimpleNamespace should preserve attribute ordering (?)
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39074] Threading memory leak in _shutdown_locks for non-daemon threads

[Adam]

New submission from Adam :

When running 3.7, we noticed a memory leak in threading._shutdown_locks when 
non-deamon threads are started but "join()" or "is_alive()" is never called. 
Here's a test to illustrate the growth:

=
import threading
import time
import tracemalloc

def test_leaking_locks():
tracemalloc.start(10)
snap1 = tracemalloc.take_snapshot()
def print_things():
print('.', end='')

for x in range(500):
t = threading.Thread(target=print_things)
t.start()

time.sleep(5)
print('')
gc.collect()
snap2 = tracemalloc.take_snapshot()
filters = []
for stat in 
snap2.filter_traces(filters).compare_to(snap1.filter_traces(filters), 
'traceback')[:10]:
print("New Bytes: {}\tTotal Bytes {}\tNew blocks: {}\tTotal blocks: {}: 
".format(stat.size_diff, stat.size, stat.count_diff ,stat.count))
for line in stat.traceback.format():
print(line)

=

=
Output in v3.6.8:

New Bytes: 840  Total Bytes 840 New blocks: 1   Total blocks: 1: 
  File "/usr/local/lib/python3.6/threading.py", line 884
self._bootstrap_inner()
New Bytes: 608  Total Bytes 608 New blocks: 4   Total blocks: 4: 
  File "/usr/local/lib/python3.6/tracemalloc.py", line 387
self.traces = _Traces(traces)
  File "/usr/local/lib/python3.6/tracemalloc.py", line 524
return Snapshot(traces, traceback_limit)
  File "/gems/tests/integration/endpoint_connection_test.py", line 856
snap1 = tracemalloc.take_snapshot()
  File "/usr/local/lib/python3.6/site-packages/_pytest/python.py", line 198
testfunction(**testargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 187
res = hook_impl.function(*args)
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 87
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 93
return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 286
return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python3.6/site-packages/_pytest/python.py", line 1459
self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/usr/local/lib/python3.6/site-packages/_pytest/runner.py", line 111
item.runtest()
==
Output in v3.7.4:

New Bytes: 36000Total Bytes 36000   New blocks: 1000Total 
blocks: 1000: 
  File "/usr/local/lib/python3.7/threading.py", line 890
self._bootstrap_inner()
  File "/usr/local/lib/python3.7/threading.py", line 914
self._set_tstate_lock()
  File "/usr/local/lib/python3.7/threading.py", line 904
self._tstate_lock = _set_sentinel()
New Bytes: 32768Total Bytes 32768   New blocks: 1   Total blocks: 
1: 
  File "/usr/local/lib/python3.7/threading.py", line 890
self._bootstrap_inner()
  File "/usr/local/lib/python3.7/threading.py", line 914
self._set_tstate_lock()
  File "/usr/local/lib/python3.7/threading.py", line 909
_shutdown_locks.add(self._tstate_lock)
=

It looks like this commit didn't take into account the tstate_lock cleanup that 
happens in the C code, and it's not removing the _tstate_lock of completed 
threads from the _shutdown_locks once the thread finishes, unless the code 
manually calls "join()" or "is_alive()" on the thread:

https://github.com/python/cpython/commit/468e5fec8a2f534f1685d59da3ca4fad425c38dd

Let me know if I can provide more clarity on this!

--
messages: 358551
nosy: krypticus
priority: normal
severity: normal
status: open
title: Threading memory leak in _shutdown_locks for non-daemon threads
type: resource usage
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] test__xxsubinterpreters fails randomly

[Eric Snow]

Eric Snow  added the comment:

On Fri, Dec 13, 2019 at 8:08 PM Kyle Stanley  wrote:
> Yeah, I named it "_PyInterpreterIsFinalizing" and it's within 
> Include/cpython. Definitely open
> to suggestions on the name though, it's basically just a private getter for 
> interp->finalizing.

For a struct-specific getter we usually end the prefix with an
underscore: _PyInterpreter_IsFinalizing.  Otherwise, that's the same
name I would have used. :)

> Oh, awesome! In that case, I'll do some more rigorous testing before opening 
> the PR then;
> [snip]
> This might be a bit of a time consuming process, but I should have time in 
> the next week
> or so to work on it.

No worries (or hurries).  Just request a review from me when you're
ready.  Thanks again for working on this!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38295] test_relative_path of test_py_compile fails on macOS 10.15 Catalina

[Bo Anderson]

Bo Anderson  added the comment:

> You don't even need a C program to reproduce

Indeed, touch is built upon the POSIX file API (unless Apple modified it). The 
idea for the Apple bug report was to show it happening at a low level and not 
specific to a given tool.

> And the "cd" is optional, I get the same error from my home directory

Yes, /private/tmp is just an example but I'd be cautious saying the same 
happens everywhere. You won't be able to reproduce the issue if your current 
directory is /usr.

/private/tmp, your home directory, etc. are all "firmlinked" to 
/System/Volumes/Data in Catalina. /System/Volumes/Data/private/tmp exists but 
/System/Volumes/Data/tmp doesn't exist. This shouldn't really be an issue as 
the idea of firmlinks is to make the /System/Volumes/Data invisible and thus 
you should be able to relatively go back up to the /System/Volumes/Data and be 
transported back to the root directory, where you can find the /tmp symlink 
(and indeed that works fine with `ls`). Evidently that doesn't seem to work 
properly for file operations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38546] test_concurrent_futures: reap_children() warnings on RHEL7 and RHEL8 buildbots

[STINNER Victor]

STINNER Victor  added the comment:

I can reproduce the issue with the following match file:

test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest.test_no_stale_references
test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest.test_ressources_gced_in_workers
test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest.test_shutdown_race_issue12456

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38546] test_concurrent_futures: reap_children() warnings on RHEL7 and RHEL8 buildbots

[STINNER Victor]

STINNER Victor  added the comment:

Example of failure, with additional debug prints:



0:01:02 load avg: 42.13 [ 38/1] test_concurrent_futures failed (env changed)

  
test_no_stale_references 
(test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 
2.00s reap_children 
  
ok  

  
test_ressources_gced_in_workers 
(test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 
3.86s reap_children 
   
ok  

  
test_shutdown_race_issue12456 
(test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 
2.22s reap_children 
 
ok  

  
test_submit 
(test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 
2.26s reap_children 
   
ok  

  
test_submit_keyword 
(test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 
1.59s reap_children 
   
ok  

  
test_traceback 
(test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 
1.60s reap_children 

ok  

 
MP cleanup: cleanup process 
  
MP cleanup: stop forkserver 
  
MP cleanup: stop res track  

MP cleanup: run finalizers  

reap_children   
  

  
--  
  


Ran 6 tests in 17.210s  


  
OK  
  
reap_children   
  
Warning -- reap_children() reaped child process 4992
 

[issue38546] test_concurrent_futures: reap_children() warnings on RHEL7 and RHEL8 buildbots

[STINNER Victor]

STINNER Victor  added the comment:

I managed to reproduce the issue on the RHEL8 worker using:

* terminal 1: ./python -u -m test --fail-env-changed test_concurrent_futures -v 
-m ProcessPoolSpawnProcessPoolExecutorTest -F -j 15
* terminal 2: ./python -m test -j4 -u all -F -r

I can still reproduce the issue with PR 17641 fix, so it's not enough.

I also applied PR 17640 and 2 extra changes:

diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py
index 1f9c2daa25..dbbf84b3d2 100644
--- a/Lib/multiprocessing/managers.py
+++ b/Lib/multiprocessing/managers.py
@@ -661,7 +661,7 @@ class BaseManager(object):
 except Exception:
 pass
 
-process.join(timeout=1.0)
+process.join(timeout=0.1)
 if process.is_alive():
 util.info('manager still alive')
 if hasattr(process, 'terminate'):
diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
index 215bab8131..110c7f945e 100644
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -2399,6 +2399,7 @@ def reap_children():
 
 # Reap all our dead child processes so we don't leave zombies around.
 # These hog resources and might be causing some of the buildbots to die.
+time.sleep(0.5)
 while True:
 try:
 # Read the exit status of any child process which already completed

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38295] test_relative_path of test_py_compile fails on macOS 10.15 Catalina

[Ronald Oussoren]

Ronald Oussoren  added the comment:

You don't even need a C program to reproduce:

$ cd /private/tmp
$ touch ../../foo.txt
touch: ../../tmp/foo.txt: No such file or directory

And the "cd" is optional, I get the same error from my home directory 
(/Users/ronald). 

All of this on macOS 10.15.2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37421] Some tests leak temporary files

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +17110
pull_request: https://github.com/python/cpython/pull/17641

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39059] Getting incorrect results in rounding procedures

[Mark Dickinson]

Change by Mark Dickinson :


--
nosy:  -mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38546] test_concurrent_futures: reap_children() warnings on RHEL7 and RHEL8 buildbots

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +17109
pull_request: https://github.com/python/cpython/pull/17641

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38546] test_concurrent_futures: reap_children() warnings on RHEL7 and RHEL8 buildbots

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +17108
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17640

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39073] email regression in 3.8: folding

[Jasper Spaans]

Jasper Spaans  added the comment:

As can be seen above, 3.5 wraps the realname in a double quote, but 3.8 fails 
to do so. Note that 3.5 also does not add a whitespace in front of the line 
starting with "X:", so it is also not merged with the previous line when 
parsing.

I guess we'll have to disallow \r and \n in displaynames for now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39073] email regression in 3.8: folding

[Jasper Spaans]

New submission from Jasper Spaans :

big-bob:t spaans$ cat fak.py
import sys

from email.message import EmailMessage
from email.policy import SMTP
from email.headerregistry import Address

msg = EmailMessage(policy=SMTP)

a = Address(display_name='Extra Extra Read All About It This Line Does Not Fit 
In 80 Characters So Should Be Wrapped \r\nX:', 
addr_spec='evil@local')
msg['To'] = a
print(sys.version)
print(msg.as_string())
big-bob:t spaans$ python3.5 fak.py
3.5.2 (default, Jul 16 2019, 13:40:43) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)]
To: "Extra Extra Read All About It This Line Does Not Fit In 80 Characters So 
Should Be Wrapped 
X:" 


big-bob:t spaans$ python3.8 fak.py
3.8.0 (default, Dec 17 2019, 13:32:18) 
[Clang 11.0.0 (clang-1100.0.33.16)]
To: Extra Extra Read All About It This Line Does Not Fit In 80 Characters So
 Should Be Wrapped 
X: 

--
components: email
messages: 358544
nosy: barry, jap, r.david.murray
priority: normal
severity: normal
status: open
title: email regression in 3.8: folding
type: security
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38858] new_interpreter() should reuse more Py_InitializeFromConfig() code

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 630c8df5cf126594f8c1c4579c1888ca80a29d59 by Victor Stinner in 
branch 'master':
bpo-38858: Small integer per interpreter (GH-17315)
https://github.com/python/cpython/commit/630c8df5cf126594f8c1c4579c1888ca80a29d59


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38671] pathlib.Path.resolve(strict=False) returns relative path on Windows if the entry does not exist

[Riccardo Polignieri]

Change by Riccardo Polignieri :


--
nosy: +ricpol

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39059] Getting incorrect results in rounding procedures

[AVicennA]

AVicennA  added the comment:

@mark.dickinson:

You asked some questions, then, I also asked you about solving this problem. It 
seems to me you just love to talk.. I just answered to your questions. If "this 
isn't the right forum for that discussion" as you mean, it concerns you too. 
When you ask it's right, but when I answer it's wrong place to discuss ?! I can 
change this line of my post - "In this case, whatever the range the full right 
result is a return" to " In this case, sometimes right result can be return 
like a round". round function also not work properly, my_round too. Each of 
them must be used in its place. In the end I wanna say that, if you want to 
discuss this further,  please, we can continue conversation in mailing list or 
where you want. Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com