[issue38032] lib2to3 doesn't parse Python 3 identifiers containing non-spacing marks

[Justin Arthur]

Justin Arthur  added the comment:

Not sure if there is consensus on how to fix, but fixing #12731 will fix this 
for most of the cases I've seen complaints about as a side effect.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33533] Provide an async-generator version of as_completed

[Justin Arthur]

Justin Arthur  added the comment:

I've added a new PR, PR 22491. This one has as_completed returning an iterator 
and includes tests for both the old and new style.

I see a trivial amount of latency added in extra call stack over Guido's 
original implementation. Should we decide to deprecate the 
plain-iterator-of-awaitables form, we can possibly regain that latency in 
refactoring this as an asynchronous generator.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +21509
pull_request: https://github.com/python/cpython/pull/22492

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

Mean +- std dev: [master-compute] 317 ns +- 3 ns -> [bpo-41902-compute] 287 ns 
+- 6 ns: 1.11x faster (-10%)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

I found another optimal case and this looks more practical usecase than 
range.index

--
Added file: https://bugs.python.org/file49485/bench_range_compute_item.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33533] Provide an async-generator version of as_completed

[Justin Arthur]

Change by Justin Arthur :


--
nosy: +JustinTArthur
nosy_count: 5.0 -> 6.0
pull_requests: +21508
pull_request: https://github.com/python/cpython/pull/22491

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41907] Regression in IntFlag behaviour in f-string

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +barry, eli.bendersky, ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

> s there reason to believe that range.index is performed often enough that 
> it's worth the extra code and maintenance cost here?

There are at least 3 reasons
1. pointer comparaition optimization quite common usecase in CPython codebase. 
e.x) list.count
2. if you create the range object, step = 1 which is singleton object is quite 
high percentage use case.
   - range(100) -> step = 1
   - range(-100, 100) -> step =1
   - range(-100, 100, 2) -> step != 1
3. fast path code does not cost high difficulty maintainence for this case but 
can bring 12% performance enhancement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[mohamed koubaa]

Change by mohamed koubaa :


--
pull_requests: +21507
pull_request: https://github.com/python/cpython/pull/22490

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[mohamed koubaa]

Change by mohamed koubaa :


--
pull_requests: +21506
pull_request: https://github.com/python/cpython/pull/22489

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41907] Regression in IntFlag behaviour in f-string

[Roger Taylor]

New submission from Roger Taylor :

An IntFlag member before 3.8.6 was converted to an integer in an f-string. 
After 3.8.6, the textual IntFlag class and member name are placed in the 
interpolated f-string instead of the integer.

3.8.3: f"... {X.F} ..." where X.F = 1 << 4 will give "... 16 ..."
3.8.5: Same
3.8.6: f"... {X.F} ..." where X.F = 1 << 4 will give "... X.F ..."

I have reproduced this on Linux using the version compiled and installed by 
pyenv, and on Windows using the 32-bit versions from the downloadable 
standalone installer.

Now I have to locate and go through all my SQL statements and verify that they 
explicitly convert IntFlag values to integer to avoid the errors introduced by 
this regression.

--
components: Interpreter Core
messages: 377793
nosy: rt121212121
priority: normal
severity: normal
status: open
title: Regression in IntFlag behaviour in f-string
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41904] datetime.datetime.today makes no sense and should be removed

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +belopolsky, p-ganssle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41904] datetime.datetime.today makes no sense and should be removed

[Steven D'Aprano]

Steven D'Aprano  added the comment:

On Thu, Oct 01, 2020 at 04:37:28PM +, Damian Yurzola wrote:
> I was inspired to file this bug after reading through a multiplicity 
> of bugs introduced by folks confused by the library's behavior.

Are these public bug reports or private anecdotes?

I'm not saying that the method cannot be deprecated or removed, but it 
needs to be justified. We don't break people's code gratuitiously, at 
least not intentionally. If you make a good enough case for deprecating 
the method, we can do so.

> What do you say about the unnecessarily redefined properties?
> 
> Lines Lib/datetime.py#L1606 to datetime.py#L1620

I don't know. Do all the tests still pass if you take them out?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[raybb]

New submission from raybb :

According to the docs here (https://docs.python.org/3/library/logging.html):

"You don’t need to create specialized Filter classes, or use other classes with 
a filter method: you can use a function (or other callable) as a filter. The 
filtering logic will check to see if the filter object has a filter attribute: 
if it does, it’s assumed to be a Filter and its filter() method is called. 
Otherwise, it’s assumed to be a callable and called with the record as the 
single parameter."


If I use this code:

def noErrorLogs(param):
return 1 if param.levelno < 40 else 0

logconfig_dict = {
'filters': {
'myfilter': {
'()': noErrorLogs,
}
},
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": "DEBUG",
"stream": "ext://sys.stdout",
"filters": ["myfilter"]
}
},
"root": {"level": "DEBUG", "handlers": ["console"]},
"version": 1,
}
dictConfig(logconfig_dict)

I get the error "Unable to configure filter 'myfilter'" because "noErrorLogs() 
missing 1 required positional argument: 'param'"


However, If I use this code:


def noErrorLogs(param):
return 1 if param.levelno < 40 else 0

logconfig_dict = {
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": "DEBUG",
"stream": "ext://sys.stdout",
}
},
"root": {"level": "DEBUG", "handlers": ["console"]},
"version": 1,
}

logging.basicConfig()
dictConfig(logconfig_dict)
l = logging.getLogger()
l.handlers[0].addFilter(noErrorLogs)

Then the filter works correctly.




The bug I am reporting is that when using logging.config.dictConfig you cannot 
pass in a callable that acts a filter. You can only pass in a class with a 
filter method or a function that returns a callable filter.

If this is the expected behavior perhaps the docs should make it clear that 
callables cannot be used with dictConfig.

--
messages: 377791
nosy: raybb
priority: normal
severity: normal
status: open
title: logging.config.dictConfig does not work with callable filters
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41513] High accuracy math.hypot()

[Raymond Hettinger]

Change by Raymond Hettinger :


Added file: https://bugs.python.org/file49484/best_frac.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41513] High accuracy math.hypot()

[Raymond Hettinger]

Change by Raymond Hettinger :


Removed file: https://bugs.python.org/file49435/best_frac.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41905] add update_abstractmethods function to update an ABC's abstract methods

[Raymond Hettinger]

Raymond Hettinger  added the comment:

The downside of this proposal is that it is tightly coupled with class creation 
process.  It requires all existing class decorators, metaclasses, and code 
generators to start caring about something that isn't part of their core 
functionality (very few tools are ABC aware).

I'm usually dubious about proposed solutions to problems that 1) rarely occur 
in practice and 2) require everyone else to change their code.

Am not sure why total_ordering was mentioned?  No current collections ABC 
requires the ordering methods and hand-rolled numeric classes typically don't 
use total_ordering because it is simpler, faster, and cleaner to implement the 
methods directly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41889] enum: Mixin and int base class regression in 3.8.6

[Ethan Furman]

Change by Ethan Furman :


--
assignee:  -> ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41889] enum: Mixin and int base class regression in 3.8.6

[Ethan Furman]

Change by Ethan Furman :


--
keywords: +patch
pull_requests: +21505
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22487

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41613] get_type_hints regression for 3.9 and 3.10

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

Commit 3ff51d425ecd98b7ba5a12ca9f77eda73fbf9f53 set the __builtins__ to None in 
namedtuple's creation, but the problem is that it is causing harm when the 
point came to the point of getting type hints on typing.NamedTuple.__new__ (we 
had a test about it), I'm changing that test (in the PR 20434) to work directly 
with the namedtuple itself instead of __new__ (since the annotations get 
proxied).

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17490] Improve ast.literal_eval test suite coverage

[Guido van Rossum]

Guido van Rossum  added the comment:

Hm, I don't think PR 22469 is relevant to this issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41905] add update_abstractmethods function to update an ABC's abstract methods

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41905] add update_abstractmethods function to update an ABC's abstract methods

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40455] GCC 10 compiler warnings

[STINNER Victor]

STINNER Victor  added the comment:

There are still warnings on compile.c:
https://buildbot.python.org/all/#/builders/448/builds/130

Python/compile.c:5660:19: warning: writing 1 byte into a region of size 0 
[-Wstringop-overflow=]
Python/compile.c:5660: warning: writing 1 byte into a region of size 0 
[-Wstringop-overflow=]
Python/compile.c:5660: warning: writing 1 byte into a region of size 0 
[-Wstringop-overflow=]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Ammar Askar]

Ammar Askar  added the comment:

Out of curiosity, what is the motivation for this optimization? 

Is there reason to believe that range.index is performed often enough that it's 
worth the extra code and maintenance cost here?

--
nosy: +ammar2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17490] Improve ast.literal_eval test suite coverage

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
keywords: +patch
pull_requests: +21504
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/22469

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17490] Improve ast.literal_eval test suite coverage

[Guido van Rossum]

Guido van Rossum  added the comment:

I agree that it's better to just close this issue, the patch is 7 years old.

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41692] Deprecate immortal interned strings: PyUnicode_InternImmortal()

[STINNER Victor]

STINNER Victor  added the comment:

I proposed PR 22486 to deprecate the function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41692] Deprecate immortal interned strings: PyUnicode_InternImmortal()

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +21503
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22486

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41834] Remove _Py_CheckRecursionLimit variable

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38605] [typing] PEP 563: Postponed evaluation of annotations: enable it by default in Python 3.10

[STINNER Victor]

STINNER Victor  added the comment:

> Issue 41314 changed the __future__ annotations default version to 3.10.  
> 3.10.0a1 is scheduled for next Oct 5, less than 2 months from now.  It would 
> be good if PR 20434 were merged before that.

Python 3.9.0 final is also around the corner. Any update on this issue?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41905] add update_abstractmethods function to update an ABC's abstract methods

[Ben Avrahami]

Ben Avrahami  added the comment:

for the functionality to work, `total_ordering` needs to change to also 
override abstract methods. I believe this is an OK change since total_ordering 
implicitly dictates that the comparison methods are interchangable. Thus, 
implementing some comparison operators, but leaving others unchanged is 
undesired behaviour.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29893] create_subprocess_exec doc doesn't match software

[STINNER Victor]

STINNER Victor  added the comment:

> This was fixed in https://github.com/python/cpython/pull/12598

Right, thanks. I close the issue.

commit 1328375ad1c91f25a1500945a67b0ef36e387527
Author: Dima Tisnek 
Date:   Fri Apr 5 23:02:28 2019 +0900

Fix doc for create_subprocess_exec (GH-12598)



Add missing `program` argument to asyncio.create_subprocess_exec 
documentation.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41905] add update_abstractmethods function to update an ABC's abstract methods

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
nosy: +python-dev
nosy_count: 2.0 -> 3.0
pull_requests: +21502
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22485

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40956] Use Argument Clinic in sqlite3

[Erlend Egeberg Aasland]

Change by Erlend Egeberg Aasland :


--
pull_requests: +21501
pull_request: https://github.com/python/cpython/pull/22484

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29127] Incorrect reference names in asyncio.subprocess documentation

[Carl Bordum Hansen]

Carl Bordum Hansen  added the comment:

I do not think this is the case any longer

--
nosy: +carlbordum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29893] create_subprocess_exec doc doesn't match software

[Carl Bordum Hansen]

Carl Bordum Hansen  added the comment:

This was fixed in https://github.com/python/cpython/pull/12598

--
nosy: +carlbordum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41901] Added some explaining to pickle errors.

[Marco Sulla]

Marco Sulla  added the comment:

I do not remember the problem I had, but when I experimented with frozendict I 
get one of these errors. I failed to understand the problem so I added the 
additional info.

Maybe adding an assert in debug mode? It will be visible only to devs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21955] ceval.c: implement fast path for integers with a single digit

[STINNER Victor]

STINNER Victor  added the comment:


New changeset bd0a08ea90e4c7a2ebf29697937e9786d4d8e5ee by Victor Stinner in 
branch 'master':
bpo-21955: Change my nickname in BINARY_ADD comment (GH-22481)
https://github.com/python/cpython/commit/bd0a08ea90e4c7a2ebf29697937e9786d4d8e5ee


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41904] datetime.datetime.today makes no sense and should be removed

[Damian Yurzola]

Damian Yurzola  added the comment:

Thanks for your prompt answer Steven.

I was inspired to file this bug after reading through a multiplicity of bugs 
introduced by folks confused by the library's behavior. So there's good 
precedent.

While granted, the documentation is explicit and the inheritance chain 
substantiates it. There's nothing more explicit than the function/type names 
and saying datetime.today() brings, as you say, arbitrary time to the 
conversation. Which I claim, subjectively, that it should not.


Gratuitous breakage, is debatable. It would not be the first or last. It could 
be a chance to remove a lot of code that works around potentially incorrect 
mental models.

But since both points are to some extent subjective. I'm OK to have left this 
on the record and move on.


What do you say about the unnecessarily redefined properties?

Lines Lib/datetime.py#L1606 to datetime.py#L1620

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21955] ceval.c: implement fast path for integers with a single digit

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +21500
pull_request: https://github.com/python/cpython/pull/22481

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[STINNER Victor]

STINNER Victor  added the comment:

Mark: "I'd much prefer not. Every extra fast path check costs time for the 
general case, and there's no strong reason to expect people to be dividing by 
one. The range code seems like the right place for this optimization, not the 
long-divide code."

In this case, I suggest to add a comment in long_div() to explain why there is 
no fast-path in long_div(). Otherwise, I am likely to suggest the exact same 
optimization in 6 months :-D

I'm thinking at something similar to my comment in ceval.c:

case TARGET(BINARY_ADD): {
PyObject *right = POP();
PyObject *left = TOP();
PyObject *sum;
/* NOTE(haypo): Please don't try to micro-optimize int+int on
   CPython using bytecode, it is simply worthless.
   See http://bugs.python.org/issue21955 and
   http://bugs.python.org/issue10044 for the discussion. In short,
   no patch shown any impact on a realistic benchmark, only a minor
   speedup on microbenchmarks. */

This comment is the outcome of 2 years of hard work by many developers :-D See 
bpo-21955.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26680] Incorporating float.is_integer into the numeric tower and Decimal

[Mark Dickinson]

Mark Dickinson  added the comment:


New changeset 58a7da9e125422323f79c4ee95ac5549989d8162 by Robert Smallshire in 
branch 'master':
bpo-26680: Incorporate is_integer in all built-in and standard library numeric 
types (GH-6121)
https://github.com/python/cpython/commit/58a7da9e125422323f79c4ee95ac5549989d8162


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Mark Dickinson]

Mark Dickinson  added the comment:

> Do we need another fast-path in long_div(a, b) when b == _PyLong_One? Just 
> return a in this case.

I'd much prefer not. Every extra fast path check costs time for the general 
case, and there's no strong reason to expect people to be dividing by one. The 
range code seems like the right place for this optimization, not the 
long-divide code.

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39337] codecs.lookup() ignores non-ASCII characters, whereas encodings.normalize_encoding() copies them

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +21499
pull_request: https://github.com/python/cpython/pull/17997

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39337] codecs.lookup() ignores non-ASCII characters, whereas encodings.normalize_encoding() copies them

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-37751 and my PR 17997.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41904] datetime.datetime.today makes no sense and should be removed

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Even if I agreed that this method "makes no sense", and I don't, removing it 
would be gratuitous breakage.

Why should we break potentially thousands of people's code who are happily 
using this method, merely because you say that people who use it have "the 
wrong semantic mental model"?

Considered as a datetime, having `today` return the date and time *now* makes 
good sense to me. Obviously the date components must be today's date, not 
yesterday or tomorrow; and the time component could be any arbitrary time, with 
the current time an obvious choice.

And since datetime inherits from date, it would be very odd if it didn't 
inherit the `today` method. It would also violate the Liskov Substitution 
Principle.

But even if you disagree with my opinions, you still have to justify breaking 
backwards compatibility.

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

PR 22479 and PR 22480 are different approaches.

I (and Victor) want to check which approach might be better.

PR 22480 would affect overall long division performance
PR 22479 assumes that step=1 case is very often (e.g range(100), range(-100, 
100))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41905] add update_abstractmethods function to update an ABC's abstract methods

[Eric V. Smith]

Change by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41905] add update_abstractmethods function to update an ABC's abstract methods

[Ben Avrahami]

New submission from Ben Avrahami :

python-ideas discussion:
https://mail.python.org/archives/list/python-id...@python.org/thread/6BNJ3YSEBPHEPGXSAZGBW3TJ64ZGZIHE/

In order to allow "decorator mixins" (most notably dataclass and 
total_ordering) to implement ABCs, new functionality is needed. I propose a new 
python function in `abc.py` called `update_abstractmethods`. The function will 
accept a class and, if the class is an instance of ABCMeta, will update the 
class's `__abstractmethods__` attribute to not include implemented attributes, 
and to include new abstractmethods (proposed implementation in thread).

Both dataclass and total_ordering will be modified to call this function on the 
subject class before returning it, and 3rd-party libraries which implement 
mixin decorators (like attrs) will be to do the same.

Also, the function can be used as a decorator individually, this is especially 
useful in cases where 3rd party decorators do not call the function.

--
components: Library (Lib)
messages: 377769
nosy: avrahami.ben
priority: normal
severity: normal
status: open
title: add update_abstractmethods function to update an ABC's abstract methods
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41904] datetime.datetime.today makes no sense and should be removed

[Damian Yurzola]

New submission from Damian Yurzola :

Last night I discovered we have datetime.datetime.today alongside
datetime.datetime.now and datetime.date.today.

- datetime.now
- date.today

Both make semantic sense.

datetime.datetime.today returns a datetime, which make no semantic sense and 
causes confusion.

On further inspection of the code, this is due to the fact that datetime 
inherits from date.

so datetime.today is practically an implementation of datetime.now minus the 
"tz".

I think we should implement a datetime.today only to rise an AttributeError or 
some other way to stop people from using the wrong semantic mental model.
We'd also need to remove the documentation entry: 
https://docs.python.org/3/library/datetime.html#datetime.datetime.today


>From this inspection we also find that:

datetime.hour/minute/second are unnecessarily redefined.
lines Lib/datetime.py#L1606 to datetime.py#L1620

could be removed without any ill effect.






date.today:
https://github.com/python/cpython/blob/256e54acdbdb26745d4bbb5cf366454151e42773/Lib/datetime.py#L833



https://docs.python.org/3/library/datetime.html#datetime.datetime.today

--
components: Library (Lib)
messages: 377768
nosy: yurzo
priority: normal
severity: normal
status: open
title: datetime.datetime.today makes no sense and should be removed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +21498
pull_request: https://github.com/python/cpython/pull/22480

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41903] PyNumber_InPlacePower ignores o3 if o1 implements __ipow__

[Ammar Askar]

Ammar Askar  added the comment:

Whoops, forgot the numpy link: 
https://github.com/numpy/numpy/blob/33e1dbee8d9e11a3e96efaae822ff6f3c44e3cef/numpy/core/src/multiarray/number.c#L729-L741

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41903] PyNumber_InPlacePower ignores o3 if o1 implements __ipow__

[Ammar Askar]

Ammar Askar  added the comment:

Also of note, there is actually no way to call `PyNumber_InPlacePower` from 
pure python from what I can tell. Even libraries like numpy don't implement the 
three-argument version of the nb_inplace_power slot.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

For the record with optimization

Mean +- std dev: [range_master] 112 ns +- 3 ns -> [range_opt] 99.3 ns +- 1.8 
ns: 1.13x faster (-12%)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

> do you build Python with PGO+LTO optimizations

Nope, but I will try to run the benchmark

> What is your OS and CPU? Do you use CPU isolation on Linux?

macOS, intel i9 with 8 cores

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41892] use both "for in" and "ElementTree.remove" has a index bug

[Eric V. Smith]

Eric V. Smith  added the comment:

I can't say how ElementTree works without more checking, but this solution 
cannot work in general. Given a pointer to an object that's in a list, how 
would you get to the next item? Say the parent list-like object has a C array 
of pointers to the objects it contains, and removing one of the objects 
re-shuffles that list. How would keeping a pointer to the current object help 
you?

In any event, I don't think this behavior is going to change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41901] Added some explaining to pickle errors.

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It was intentionally left a generic "cannot pickle 'type' object". Because 
there are many ways to make an object pickleable, and it is non-pickleable in 
all other cases.

* "missing `__getstate__()` on variable-length object"

It creates false impression that implementing __getstate__() is the only way to 
solve the problem. Actually you can make that object pickleable by:

- implementing __reduce__() or __reduce_ex__()
- implementing __getnewargs__() or __getnewargs_ex__()
- implementing __getstate__()
- make it a subclass of list or dict

So more correct error message should contain "missing __reduce__(), 
__reduce_ex__(), __getnewargs__(), __getnewargs_ex__() and __getstate__() on 
variable-length object which is not a list or dict". And even this is not 
completely correct, because it has __reduce__() and __reduce_ex__() inherited 
from class object, but it is not enough. Completely correct error message can 
be hundreds characters long.

And this still may be misleading error message, because the root cause is not 
that some object is not pickleable, but that you try to pickle an object which 
has an unexpected non-pickleable component.

* "basic size greater than expected"

Basic size is an implementation detail, and the end user can have no guess what 
is it. Actually it means that an object implemented in C likely has some inner 
state and the pickle module does not have access to it because the object does 
not implement __reduce__(), __reduce_ex__(), __getnewargs__(), 
__getnewargs_ex__() or __getstate__().

* "object has no `__new__()`"

Every object has __new__ at Python level, even if its tp_new slot is NULL. 
tp_new is the C implementation detail and should not be mentioned in the error 
message. And the object without tp_new still can be pickleable, if it 
implements __reduce__() or __reduce_ex__().

It is all too complicated. It is better to omit details in error messages than 
provide not correct and direct user in wrong way.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41903] PyNumber_InPlacePower ignores o3 if o1 implements __ipow__

[Ammar Askar]

Change by Ammar Askar :


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41903] PyNumber_InPlacePower ignores o3 if o1 implements __ipow__

[Ammar Askar]

Ammar Askar  added the comment:

Huh, this is weird. I wonder why the data model provides the signature

  object.__ipow__(self, other[, modulo])

if the slot always seems to drop the third argument.

Adding Brett to the nosy who found another issue around the special casing of 
pow/** recently. See also issue38302

--
nosy: +ammar2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41903] PyNumber_InPlacePower ignores o3 if o1 implements __ipow__

[Simon Cross]

Simon Cross  added the comment:

The documentation for __ipow__ [4] suggests that the intention was to support 
the modulus argument, so perhaps that argues for fixing the behaviour of 
PyNumber_InPlacePower.

[4] https://docs.python.org/3/reference/datamodel.html#object.__ipow__

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[STINNER Victor]

STINNER Victor  added the comment:

About your benchmark, do you build Python with PGO+LTO optimizations? What is 
your OS and CPU? Do you use CPU isolation on Linux? It would be good to use 
PGO+LTO optimizations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41903] PyNumber_InPlacePower ignores o3 if o1 implements __ipow__

[Simon Cross]

New submission from Simon Cross :

The documentation for PyNumber_InPlacePower [1] reads:

This is the equivalent of the Python statement o1 **= o2 when o3 is Py_None, or 
an in-place variant of pow(o1, o2, o3) otherwise. If o3 is to be ignored, pass 
Py_None in its place (passing NULL for o3 would cause an illegal memory access).

However, if a class A implements __ipow__ then PyNumber_InPlacePower(o1, o2, 
o3) ALWAYS ignores o3 if o1 is an instance of A.

This happens because if A implements __ipow__ then PyNumber_InPlacePower always 
calls the nb_inplace_power slot [2] and the slot always drops the third 
argument [3].

This appears to have always been the case in Python, so likely a small 
documentation patch is all that is required. If people agree, I will open a 
documentation pull request.

[1] 
https://docs.python.org/3/c-api/number.html?highlight=pynumber_inplacepower#c.PyNumber_InPlacePower

[2] https://github.com/python/cpython/blob/master/Objects/abstract.c#L1164

[3] 
https://github.com/python/cpython/blob/master/Objects/typeobject.c#L6631-L6636

--
components: C API
messages: 377758
nosy: hodgestar
priority: normal
severity: normal
status: open
title: PyNumber_InPlacePower ignores o3 if o1 implements __ipow__
type: behavior
versions: Python 3.10, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 
3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[Erlend Egeberg Aasland]

Change by Erlend Egeberg Aasland :


--
nosy: +erlendaasland
nosy_count: 17.0 -> 18.0
pull_requests: +21497
pull_request: https://github.com/python/cpython/pull/22478

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

For practical use case

>>> a = range(-2, 10)
>>> a.index(2)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41892] use both "for in" and "ElementTree.remove" has a index bug

[WoodyWoo]

WoodyWoo  added the comment:

Could I say the mutable sequence containing not the object but the pointer like 
C++.
So they can changed in def functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[STINNER Victor]

STINNER Victor  added the comment:

PR 22479 avoids calling PyNumber_FloorDivide(a, b) if b == 1 (if b == 
_PyLong_One): it makes range.index(a, 1) call 214 ns faster. I'm surprised that 
PyNumber_FloorDivide(a, 1) takes 214 ns. Does it spend most time in 
binary_op1()? Or PyNumber_FloorDivide()?

long_div(a, 1) is quite simple:

CHECK_BINOP(a, b);

if (Py_ABS(Py_SIZE(a)) == 1 && Py_ABS(Py_SIZE(b)) == 1) {
return fast_floor_div((PyLongObject*)a, (PyLongObject*)b);

with:

static PyObject *
fast_floor_div(PyLongObject *a, PyLongObject *b)
{
sdigit left = a->ob_digit[0];
sdigit right = b->ob_digit[0];
sdigit div;
if (Py_SIZE(a) == Py_SIZE(b)) {
div = left / right;
}
else {
div = -1 - (left - 1) / right;
}
return PyLong_FromLong(div);
}

Do we need another fast-path in long_div(a, b) when b == _PyLong_One? Just 
return a in this case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41861] Convert sqlite3 to PEP 384

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 256e54acdbdb26745d4bbb5cf366454151e42773 by Erlend Egeberg 
Aasland in branch 'master':
bpo-41861: Convert _sqlite3 CursorType and ConnectionType to heap types 
(GH-22478)
https://github.com/python/cpython/commit/256e54acdbdb26745d4bbb5cf366454151e42773


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

For optimization case,

>>> a = range(0, 1)
>>> a.index(3)
3

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Change by Dong-hee Na :


--
keywords: +patch
pull_requests: +21496
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22479

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

Change by Dong-hee Na :


Added file: https://bugs.python.org/file49483/bench_range_step.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41902] Micro optimization for range.index if step is 1

[Dong-hee Na]

New submission from Dong-hee Na :

if we declare range without setting step, we don't have to process divide 
operation.

here is the benchmark result both setting step and without step,

With my patch, there is no performance degrade with range.index when the step 
is not one and showing 19% enhancement when the step is the default value (1) .

Mean +- std dev: [range_master] 1.11 us +- 0.01 us -> [range_opt] 896 ns +- 23 
ns: 1.24x faster (-19%)

Mean +- std dev: [range_step_master] 1.12 us +- 0.02 us -> [range_step_opt] 
1.11 us +- 0.01 us: 1.01x faster (-1%)

--
assignee: corona10
components: Interpreter Core
files: bench_range_index.py
messages: 377752
nosy: corona10, pablogsal, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Micro optimization for range.index if step is 1
type: performance
versions: Python 3.10
Added file: https://bugs.python.org/file49482/bench_range_index.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41861] Convert sqlite3 to PEP 384

[Erlend Egeberg Aasland]

Change by Erlend Egeberg Aasland :


--
pull_requests: +21495
pull_request: https://github.com/python/cpython/pull/22478

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41896] Moving index with wordstart expression includes non-alphanumberic and underline characters if word is tagged and iat the edge of a text widget

[pyTama]

pyTama  added the comment:

I'm running on windows. The patch level on python 3.6 is (8, 6, 6, 'final', 0), 
while on python 3.5 is (8, 6, 4, 'final', 0).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41861] Convert sqlite3 to PEP 384

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9031bd4fa42dbb0f25aee9286154ad4bf60df3f8 by Erlend Egeberg 
Aasland in branch 'master':
bpo-41861: Convert _sqlite3 RowType and StatementType to heap types (GH-22444)
https://github.com/python/cpython/commit/9031bd4fa42dbb0f25aee9286154ad4bf60df3f8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

The test still fails randomly: AMD64 Windows8.1 Refleaks 3.8
https://buildbot.python.org/all/#/builders/200/builds/48

test_subinterpreter failed twice on Refleaks which runs tests 6 times in a row.

First run:

0:33:58 load avg: 4.78 [204/423/1] test__xxsubinterpreters failed -- running: 
test_multiprocessing_spawn (12 min 42 sec), test_io (7 min 28 sec), 
test_tarfile (39.2 sec)
beginning 6 repetitions
123456
...Exception in thread Thread-38:
Traceback (most recent call last):
  File "D:\buildarea\3.8.ware-win81-release.refleak\build\lib\threading.py", 
line 932, in _bootstrap_inner
self.run()
  File "D:\buildarea\3.8.ware-win81-release.refleak\build\lib\threading.py", 
line 870, in run
self._target(*self._args, **self._kwargs)
  File 
"D:\buildarea\3.8.ware-win81-release.refleak\build\lib\test\test__xxsubinterpreters.py",
 line 51, in run
interpreters.run_string(interp, dedent(f"""
RuntimeError: unrecognized interpreter ID 250
test test__xxsubinterpreters failed -- Traceback (most recent call last):
  File 
"D:\buildarea\3.8.ware-win81-release.refleak\build\lib\test\test__xxsubinterpreters.py",
 line 495, in test_subinterpreter
self.assertTrue(interpreters.is_running(interp))
AssertionError: False is not true

Second run:

FAIL: test_subinterpreter (test.test__xxsubinterpreters.IsRunningTests)
--
Traceback (most recent call last):
  File 
"D:\buildarea\3.8.ware-win81-release.refleak\build\lib\test\test__xxsubinterpreters.py",
 line 495, in test_subinterpreter
self.assertTrue(interpreters.is_running(interp))
AssertionError: False is not true

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41804] test_epoll fails test_control_and_wait() randomly on aarch64 RHEL8 Refleaks 3.9

[STINNER Victor]

STINNER Victor  added the comment:

The test also fails on aarch64 RHEL7 Refleaks 3.x:
https://buildbot.python.org/all/#/builders/131/builds/46

FAIL: test_control_and_wait (test.test_epoll.TestEPoll)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41901] Added some explaining to pickle errors.

[Marco Sulla]

New submission from Marco Sulla :

All pickle error messages in typeobject.c was a generic "cannot pickle 'type' 
object". Added some explaining for every individual error.

--
components: Interpreter Core
messages: 377747
nosy: Marco Sulla
priority: normal
pull_requests: 21494
severity: normal
status: open
title: Added some explaining to pickle errors.
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22864] Add filter to multiprocessing.Pool

[Christian Leichsenring]

Christian Leichsenring  added the 
comment:

The main point the OP didn't make is exactly the issue that Pool.map returns a 
list which is potentially very large given that multiprocessing is used to 
process large amounts of data.

So IMHO either there should be the possibility to exclude elements from being 
saved in memory (i.e. Pool.filter) or Pool.map shouldn't return a list but just 
an iterable.

--
nosy: +christian.leichsenring

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41896] Moving index with wordstart expression includes non-alphanumberic and underline characters if word is tagged and iat the edge of a text widget

[E. Paine]

E. Paine  added the comment:

Thank you for reporting this issue, however it is ultimately a Tk issue as 
shown by running the attached script in wish. I assume the Tk version was 
updated in Python 3.6, though I don't know which platform you are on so don't 
know which of Ned or Steve to nosy to verify this theory (you can check 
yourself by calling `tkinter.test.support.get_tk_patchlevel()` in both 3.5 & 
3.6).

I am currently in the process of digging through the Tk source looking for the 
cause of the problem (so I can decide whether to report it to the Tk team - you 
can do so yourself if you prefer). This issue should be closed as third party, 
but I will link the Tk issue here if I decide to create one.

--
Added file: https://bugs.python.org/file49481/wordstart with tag.tcl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41871] Add PyList_Remove() in listobject.c

[STINNER Victor]

STINNER Victor  added the comment:

> it is more convenient to just call the method

Calling a Python method in Python is not convenient nor efficient.

For example, PyObject_CallMethod() has to decode a name from UTF-8 and then 
needs a lookup into the type dictionary. The list.remove() raises a exception 
if the item is not found, whereas PyCodec_Unregister() does nothing if the 
codec search function is not found: the exception is ignored.

It's also about correctness, looking for an item in a list can be tricky if the 
list size changes during the lookup, or Python can crash if it uses a borrowed 
reference whereas a list item is destroyed. There were tons of such bugs in the 
dict implementation. list.remove() doesn't look safe, but I would prefer to 
avoid having to fix the same bugs in copies of similar functions.

If a public function is not justified, what about an internal function?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41896] Moving index with wordstart expression includes non-alphanumberic and underline characters if word is tagged and iat the edge of a text widget

[E. Paine]

Change by E. Paine :


--
nosy: +epaine

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41900] XML C14N serialisation fails with default namespace

[Stefan Behnel]

Change by Stefan Behnel :


--
keywords: +patch
pull_requests: +21493
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/22474

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41900] XML C14N serialisation fails with default namespace

[Stefan Behnel]

New submission from Stefan Behnel :

import xml.etree.ElementTree as ET
xml="""
http://soap.sforce.com/2006/04/metadata;>

"""
print(ET.canonicalize(xml))

Fails with:
ValueError: Namespace "" is not declared in scope
when trying to build the QName of the unnamespaced "targets" attribute.

Originally reported for lxml here:
https://bugs.launchpad.net/lxml/+bug/1869455

--
components: XML
messages: 377743
nosy: scoder
priority: normal
severity: normal
stage: needs patch
status: open
title: XML C14N serialisation fails with default namespace
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31256] xml.etree.ElementTree: add support for doctype in tostring method

[Stefan Behnel]

Stefan Behnel  added the comment:

Yes, it fixed already. Thanks!

--
nosy: +scoder
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41899] Poor example for Element.remove()

[Stefan Behnel]

Stefan Behnel  added the comment:

Closing as duplicate of issue 41891. Let's keep the discussion there.

--
nosy: +scoder
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41899] Poor example for Element.remove()

[Stefan Behnel]

Change by Stefan Behnel :


--
Removed message: https://bugs.python.org/msg377740

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41899] Poor example for Element.remove()

[Stefan Behnel]

Stefan Behnel  added the comment:

Closing as duplicate of issue 41892. Let's keep the discussion there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37294] concurrent.futures.ProcessPoolExecutor state=finished raised error

[DanilZ]

DanilZ  added the comment:

I think you have correctly estimated the problem in the last part of your 
message: "as it could possibly indicate an issue with running out of memory 
when the dataframe is converted to pickle format (which often increases the 
total size) within the process associated with the job”

The function pd.read_csv performs without any problems inside a process, the 
error appears only when I try to extract it from the finished process via:
for f in concurrent.futures.as_completed(results):
data = f.result()

or

data = results.result()

It just does not pass a large file from the results object.

I am sure that inside of a multiprocess everything works correctly for 2 
reasons:
1. If I change in function inside a process to just save the file (that had 
been read in memory) to disk.
2. If I recuse the file size, then it gets extracted from results.result() 
without error.

So I guess then that my question narrows down to: 
1. Can I increase the memory allocated to a process? 
2. Or at least understand what would is the limit.

Regards,
Danil

> On 1 Oct 2020, at 03:11, Kyle Stanley  wrote:
> 
> 
> Kyle Stanley  added the comment:
> 
> DanilZ, could you take a look at the superseding issue 
> (https://bugs.python.org/issue37297) and see if your exception raised within 
> the job is the same?  
> 
> If it's not, I would suggest opening a separate issue (and linking to it in a 
> comment here), as I don't think it's necessarily related to this one. 
> "state=finished raised error" doesn't indicate the specific exception that 
> occurred. A good format for the name would be something along the lines of:
> 
> "ProcessPoolExecutor.submit()  while reading 
> large object (4GB)"
> 
> It'd also be helpful in the separate issue to paste the full exception stack 
> trace, specify OS, and multiprocessing start method used (spawn, fork, or 
> forkserver). This is necessary to know for replicating the issue on our end.
> 
> In the meantime, I workaround I would suggest trying would be to use the  
> *chunksize* parameter (or *Iterator*) in pandas.read_csv(), and split it 
> across several jobs (at least 4+, more if you have additional cores) instead 
> of within a single one. It'd also be generally helpful to see if that 
> alleviates the problem, as it could possibly indicate an issue with running 
> out of memory when the dataframe is converted to pickle format (which often 
> increases the total size) within the process associated with the job.
> 
> --
> nosy: +aeros
> 
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37294] concurrent.futures.ProcessPoolExecutor state=finished raised error

[DanilZ]

DanilZ  added the comment:

.

> On 1 Oct 2020, at 03:11, Kyle Stanley  wrote:
> 
> 
> Kyle Stanley  added the comment:
> 
> DanilZ, could you take a look at the superseding issue 
> (https://bugs.python.org/issue37297) and see if your exception raised within 
> the job is the same?  
> 
> If it's not, I would suggest opening a separate issue (and linking to it in a 
> comment here), as I don't think it's necessarily related to this one. 
> "state=finished raised error" doesn't indicate the specific exception that 
> occurred. A good format for the name would be something along the lines of:
> 
> "ProcessPoolExecutor.submit()  while reading 
> large object (4GB)"
> 
> It'd also be helpful in the separate issue to paste the full exception stack 
> trace, specify OS, and multiprocessing start method used (spawn, fork, or 
> forkserver). This is necessary to know for replicating the issue on our end.
> 
> In the meantime, I workaround I would suggest trying would be to use the  
> *chunksize* parameter (or *Iterator*) in pandas.read_csv(), and split it 
> across several jobs (at least 4+, more if you have additional cores) instead 
> of within a single one. It'd also be generally helpful to see if that 
> alleviates the problem, as it could possibly indicate an issue with running 
> out of memory when the dataframe is converted to pickle format (which often 
> increases the total size) within the process associated with the job.
> 
> --
> nosy: +aeros
> 
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com