[issue24076] sum() several times slower on Python 3 64-bit

2021-09-22 Thread Stefan Behnel


Stefan Behnel  added the comment:

Sorry for that, Pablo. I knew exactly where the problem was, the second I read 
your notification. Thank you for resolving it so quickly.

--

___
Python tracker 
<https://bugs.python.org/issue24076>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24076] sum() several times slower on Python 3 64-bit

2021-09-21 Thread Stefan Behnel


Change by Stefan Behnel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue24076>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24076] sum() several times slower on Python 3 64-bit

2021-09-21 Thread Stefan Behnel


Stefan Behnel  added the comment:

Old, with PGO:
$ ./python -m timeit -s 'd = list(range(2**61, 2**61 + 1))' 'sum(d)'
1000 loops, best of 5: 340 usec per loop
$ ./python -m timeit -s 'd = list(range(2**30, 2**30 + 1))' 'sum(d)'
2000 loops, best of 5: 114 usec per loop
$ ./python -m timeit -s 'd = list(range(2**29, 2**29 + 1))' 'sum(d)'
5000 loops, best of 5: 73.4 usec per loop
$ ./python -m timeit -s 'd = list(range(1))' 'sum(d)'
5000 loops, best of 5: 73.3 usec per loop
$ ./python -m timeit -s 'd = [0] * 1' 'sum(d)'
5000 loops, best of 5: 78.7 usec per loop


New, with PGO:
$ ./python -m timeit -s 'd = list(range(2**61, 2**61 + 1))' 'sum(d)'
1000 loops, best of 5: 305 usec per loop
$ ./python -m timeit -s 'd = list(range(2**30, 2**30 + 1))' 'sum(d)'
2000 loops, best of 5: 115 usec per loop
$ ./python -m timeit -s 'd = list(range(2**29, 2**29 + 1))' 'sum(d)'
5000 loops, best of 5: 52.4 usec per loop
$ ./python -m timeit -s 'd = list(range(1))' 'sum(d)'
5000 loops, best of 5: 54 usec per loop
$ ./python -m timeit -s 'd = [0] * 1' 'sum(d)'
5000 loops, best of 5: 45.8 usec per loop

The results are a bit more mixed with PGO optimisation (I tried a couple of 
times), not sure why. Might just be normal fluctuation, bad benchmark value 
selection, or accidental PGO tuning, can't say. In any case, the 1-digit case 
(1, 2**29) is again about 28% faster and none of the other cases seems 
(visibly) slower.

I think this is a very clear net-win.

--

___
Python tracker 
<https://bugs.python.org/issue24076>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24076] sum() several times slower on Python 3 64-bit

2021-09-21 Thread Stefan Behnel


Stefan Behnel  added the comment:

Hmm, thanks for insisting, Serhiy. I was accidentally using a debug build this 
time. I'll make a PGO build and rerun the microbenchmarks.

--

___
Python tracker 
<https://bugs.python.org/issue24076>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24076] sum() several times slower on Python 3 64-bit

2021-09-21 Thread Stefan Behnel


Stefan Behnel  added the comment:

Original:
$ ./python -m timeit -s 'd = list(range(2**61, 2**61 + 1))' 'sum(d)'
500 loops, best of 5: 712 usec per loop
$ ./python -m timeit -s 'd = list(range(2**30, 2**30 + 1))' 'sum(d)'
2000 loops, best of 5: 149 usec per loop
$ ./python -m timeit -s 'd = list(range(2**29, 2**29 + 1))' 'sum(d)'
2000 loops, best of 5: 107 usec per loop
$ ./python -m timeit -s 'd = list(range(1))' 'sum(d)'
2000 loops, best of 5: 107 usec per loop

New:
$ ./python -m timeit -s 'd = list(range(2**61, 2**61 + 1))' 'sum(d)'
500 loops, best of 5: 713 usec per loop
$ ./python -m timeit -s 'd = list(range(2**30, 2**30 + 1))' 'sum(d)'
2000 loops, best of 5: 148 usec per loop
$ ./python -m timeit -s 'd = list(range(2**29, 2**29 + 1))' 'sum(d)'
5000 loops, best of 5: 77.4 usec per loop
$ ./python -m timeit -s 'd = list(range(1))' 'sum(d)'
5000 loops, best of 5: 77.2 usec per loop

Seems to be 28% faster for the single digit case and exactly as fast as before 
with larger integers.
Note that these are not PGO builds.

--

___
Python tracker 
<https://bugs.python.org/issue24076>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24076] sum() several times slower on Python 3 64-bit

2021-09-21 Thread Stefan Behnel


Change by Stefan Behnel :


--
Removed message: https://bugs.python.org/msg402301

___
Python tracker 
<https://bugs.python.org/issue24076>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24076] sum() several times slower on Python 3 64-bit

2021-09-21 Thread Stefan Behnel


Stefan Behnel  added the comment:

Original:
$ ./python -m timeit -s 'd = list(range(2**61, 2**61 + 1))' 'sum(d)'
500 loops, best of 5: 712 usec per loop
$ ./python -m timeit -s 'd = list(range(2**30, 2**30 + 1))' 'sum(d)'
2000 loops, best of 5: 149 usec per loop
$ ./python -m timeit -s 'd = list(range(2**29, 2**29 + 1))' 'sum(d)'
2000 loops, best of 5: 107 usec per loop
$ ./python -m timeit -s 'd = list(range(1))' 'sum(d)'
2000 loops, best of 5: 107 usec per loop

New:
stefan@flup:~/ablage/software/Python/python-git$ ./python -m timeit -s 'd = 
list(range(2**61, 2**61 + 1))' 'sum(d)'
500 loops, best of 5: 713 usec per loop
$ ./python -m timeit -s 'd = list(range(2**30, 2**30 + 1))' 'sum(d)'
2000 loops, best of 5: 148 usec per loop
$ ./python -m timeit -s 'd = list(range(2**29, 2**29 + 1))' 'sum(d)'
5000 loops, best of 5: 77.4 usec per loop
$ ./python -m timeit -s 'd = list(range(1))' 'sum(d)'
5000 loops, best of 5: 77.2 usec per loop

Seems to be 28% faster for the single digit case and exactly as fast as before 
with larger integers.
Note that these are not PGO builds.

--

___
Python tracker 
<https://bugs.python.org/issue24076>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24076] sum() several times slower on Python 3 64-bit

2021-09-21 Thread Stefan Behnel


Stefan Behnel  added the comment:


New changeset debd80403721b00423680328d6adf160a28fbff4 by scoder in branch 
'main':
bpo-24076: Inline single digit unpacking in the integer fastpath of sum() 
(GH-28469)
https://github.com/python/cpython/commit/debd80403721b00423680328d6adf160a28fbff4


--

___
Python tracker 
<https://bugs.python.org/issue24076>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24076] sum() several times slower on Python 3 64-bit

2021-09-20 Thread Stefan Behnel


Stefan Behnel  added the comment:

> The patch looks fine, but it looks a bit like benchmark chasing. Is the speed 
> of builtin sum() of a sequence of integers important enough to do this bit of 
> inlining?

Given that we already accepted essentially separate loops for the int, float 
and everything else cases, I think the answer is that it doesn't add much to 
the triplication.


> It may break if we change the internals of Py_Long, as Mark Shannon has been 
> wanting to do for a while

I would assume that such a structural change would come with suitable macros to 
unpack the special 0-2 digit integers. Those would then apply here, too. As it 
stands, there are already some modules distributed over the source tree that 
use direct digit access: ceval.c, _decimal.c, marshal.c. They are easy to find 
with grep and my PR just adds one more.

--

___
Python tracker 
<https://bugs.python.org/issue24076>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

2021-09-20 Thread Stefan Behnel

Stefan Behnel  added the comment:

> The code in the examples given above are using `tstate->use_tracing` assuming 
> that its meaning is to determine whether tracing is turned on or not.

No, actually not. It is using the field in the same way as CPython, simply 
because most of this code was originally copied from CPython, and we also 
copied the optimisation of avoiding to check the other fields (for the obvious 
reason of being an optimisation).


> I propose adding back `tstate->use_tracing` as a copy of 
> `tstate->cframe->us_tracing`.

Cython 0.29.24 has already been adapted to this change and will use the new 
field in CPython 3.10b1+.


> Any code that reads `tstate->use_tracing` will work correctly.

Any code that reads and /writes/ the field would probably also continue to work 
correctly, which is what older Cython versions did.


> if_you_use_this_then_we_will_break_your_code_in_some_way_that_will_ruin_your_reputation_as_a_library_developer…

The thing is, new APIs can only be added to new CPython releases. Supporting 
features in older CPython versions (currently 2.7+) means that we always *have 
to* use the existing fields, and can only switch to new APIs by duplicating 
code based on a PY_VERSION_HEX preprocessor check. Even if a new low-latency 
profiling API was added in CPython 3.11, we'd have to wait until there is at 
least an alpha release that has it before enabling this code switch.

And if the new API proves to be slower, we may end up keeping the old code 
around and adding a C compile-time configuration option for users to enable (or 
disable) its use. Cython has lots of those these days, mostly to support the 
different C-API capabilities of different Python implementations, e.g. to take 
advantage of the PyLong or PyUnicode internals if available, and use generic 
C-API calls if not.

--

___
Python tracker 
<https://bugs.python.org/issue43760>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24076] sum() several times slower on Python 3 64-bit

2021-09-20 Thread Stefan Behnel


Stefan Behnel  added the comment:

I created a PR from my last patch, inlining the unpacking of single digit 
integers. Since most integers should fit into a single digit these days, this 
is as fast a path as it gets.

https://github.com/python/cpython/pull/28469

--
versions: +Python 3.11 -Python 3.7

___
Python tracker 
<https://bugs.python.org/issue24076>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24076] sum() several times slower on Python 3 64-bit

2021-09-20 Thread Stefan Behnel


Change by Stefan Behnel :


--
pull_requests: +26868
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/28469

___
Python tracker 
<https://bugs.python.org/issue24076>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

2021-07-19 Thread Stefan Behnel


Stefan Behnel  added the comment:

The obvious work-around is to not use a default namespace. The result is just a 
visual difference, not a semantic one.

If someone wants to continue with the existing PR, I'll try to free some time 
to review any improvements.

--
versions: +Python 3.10, Python 3.11 -Python 3.9

___
Python tracker 
<https://bugs.python.org/issue17088>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23609] Export PyModuleObject in moduleobject.h

2021-07-05 Thread Stefan Behnel


Stefan Behnel  added the comment:

I only just noticed that the original request is from 2015. My guess is that 
this is a resolved problem, given the available options. I'll close this ticket 
as outdated. Please comment if this is still an issue.

--
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue23609>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23609] Export PyModuleObject in moduleobject.h

2021-07-05 Thread Stefan Behnel


Change by Stefan Behnel :


--
components: +C API -Interpreter Core
type:  -> enhancement
versions: +Python 3.11 -Python 3.5

___
Python tracker 
<https://bugs.python.org/issue23609>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23609] Export PyModuleObject in moduleobject.h

2021-07-05 Thread Stefan Behnel


Stefan Behnel  added the comment:

Could you explain a bit why you want to subclass "PyModuleObject" in C? Why 
isn't a Python subclass or an independent (non-module) C extension type enough?

--

___
Python tracker 
<https://bugs.python.org/issue23609>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19832] XML version is ignored

2021-06-18 Thread Stefan Behnel


Stefan Behnel  added the comment:

After reading up a bit, version "X" should probably be rejected, whereas 
"1.[0-9]+" is meant to be allowed also by a 1.0 parser, according to the spec:

https://www.w3.org/TR/REC-xml/#sec-prolog-dtd

"""
When an XML 1.0 processor encounters a document that specifies a 1.x version 
number other than '1.0', it will process it as a 1.0 document. This means that 
an XML 1.0 processor will accept 1.x documents provided they do not use any 
non-1.0 features.
"""

However, this is not so much an issue with the SAX framework but with the 
underlying parser, which would be expat. Not sure why that doesn't care about 
the version.

Personally, I don't really care. There are only two XML versions, 1.0 and 1.1, 
and an XML 1.x parser is supposed to deal with both of them nicely. Anyone who 
writes something different in their XML version probably does so deliberately 
and wrongly. As long as the rest is XML, I don't see a reason to reject such an 
input document.

I'll close this as "won't fix", since there is no practical effect, it would 
need effort, and it doesn't look like anyone cared in almost 8 years.

--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue19832>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43420] Optimize rational arithmetics

2021-05-17 Thread Stefan Behnel


Stefan Behnel  added the comment:

Just FYI, I applied the same changes to the quicktions [1] module, a Cython 
compiled (and optimised) version of fractions.Fraction.

[1] https://github.com/scoder/quicktions/

The loss in performance for small values is much higher there, almost 2x for 
the example given (compared to 10-20% for CPython):

$ python3.8 -m timeit -r11 -s 'from fractions import Fraction as F' -s 
'a=F(10,3)' -s 'b=F(6, 5)' 'a * b'
10 loops, best of 11: 1.94 usec per loop

Original:
$ python3.8 -m timeit -r11 -s 'from quicktions import Fraction as F' -s 
'a=F(10,3)' -s 'b=F(6, 5)' 'a * b'
100 loops, best of 11: 214 nsec per loop

Patched:
$ python3.8 -m timeit -r11 -s 'from quicktions import Fraction as F' -s 
'a=F(10,3)' -s 'b=F(6, 5)' 'a * b'
50 loops, best of 11: 391 nsec per loop


For the larger values example, the gain is tremendous, OTOH:

$ python3.8 -m timeit -r11 -s 'from fractions import Fraction as F' -s 'import 
random' -s 'n = [random.randint(1, 100) for _ in range(1000)]' -s 'd = 
[random.randint(1, 100) for _ in range(1000)]' -s 'a=list(map(lambda x: 
F(*x), zip(n, d)))' 'sum(a)'
2 loops, best of 11: 150 msec per loop

Original:
$ python3.8 -m timeit -r11 -s 'from quicktions import Fraction as F' -s 'import 
random' -s 'n = [random.randint(1, 100) for _ in range(1000)]' -s 'd = 
[random.randint(1, 100) for _ in range(1000)]' -s 'a=list(map(lambda x: 
F(*x), zip(n, d)))' 'sum(a)' 
2 loops, best of 11: 135 msec per loop

Patched:
$ python3.8 -m timeit -r11 -s 'from quicktions import Fraction as F' -s 'import 
random' -s 'n = [random.randint(1, 100) for _ in range(1000)]' -s 'd = 
[random.randint(1, 100) for _ in range(1000)]' -s 'a=list(map(lambda x: 
F(*x), zip(n, d)))' 'sum(a)'
50 loops, best of 11: 9.65 msec per loop

I'll have to see if the slowdown can be mitigated somehow.

Interesting enough, the telco benchmark seems to benefit slightly from this:

Original:
$ python3.8 benchmark/telco_fractions.py -n 200   # avg
0.0952927148342

Patched:
$ python3.8 benchmark/telco_fractions.py -n 200   # avg
0.0914235627651

--
nosy: +scoder

___
Python tracker 
<https://bugs.python.org/issue43420>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

2021-05-14 Thread Stefan Behnel


Change by Stefan Behnel :


--
Removed message: https://bugs.python.org/msg393667

___
Python tracker 
<https://bugs.python.org/issue43760>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

2021-05-14 Thread Stefan Behnel


Stefan Behnel  added the comment:

Should we allow empty CFrame entries (with a NULL Frame pointer) that only use 
their "use_tracing" flag? That would make it easy for code to append and remove 
such an entry in order to temporarily disable tracing.

--

___
Python tracker 
<https://bugs.python.org/issue43760>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

2021-05-14 Thread Stefan Behnel


Stefan Behnel  added the comment:

I just noticed that new C-API functions are probably useless for Cython since I 
think it will have to maintain the CFrame stack, so not to enable "use_tracing" 
for the (Python) caller but the current (Cython) function. This then means that 
we own The current CFrame as well as its "use_tracing" field and don't need any 
help from CPython in order to change the state.

I'm not sure if this is any different for other users of the "use_tracing" 
field.

--
type:  -> performance

___
Python tracker 
<https://bugs.python.org/issue43760>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2021-05-11 Thread Stefan Behnel


Stefan Behnel  added the comment:

Just a comment regarding the change to "PyCode_NewWithPosOnlyArgs()". As Pablo 
mentioned, this has happened before. And that's OK! Exactly because this has 
happened before, it's clearly not a part of the API that is meant to be stable.

I can easily adapt Cython to make this work in the next patch-level release of 
CPython 3.11 (or the current one, since alpha-1 seems not so close), but any 
adaptation will be patch-level dependent. Meaning, for each such change, there 
will be a couple of weeks or months until the C preprocessor makes the code 
compile again. And during that time, people won't be able to test their code to 
report issues.

So, I'd rather have compatibility broken and stay that way, than going one way 
now and changing it back later, thus going through the same adaptation period 
twice.

That being said, any such change means that maintainers will have to rebuild 
their packages with a new Cython release to adapt them to Py3.11. Many will, 
but some won't, for whatever reason.

--
nosy: +scoder

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

2021-05-11 Thread Stefan Behnel


Stefan Behnel  added the comment:

For the same reason that motivated this ticket, I think the functions should be 
inline functions. They should also take the current thread-state as argument, 
because that's probably known on the caller side already.

I guess a macro would be fine, too. :)

Cython previously used "use_tracing" directly because it needs to implement the 
exact same tracing/profiling behaviour as CPython, regardless of who called a 
Cython implemented function (Cython or CPython).

Naming nit: Get/Is/UsesTracing?

Also, given that a common use case seems to be "make sure tracing is disabled, 
do something, enable tracing if it was enabled", I think DisableTracing() 
should return the previous state.

--
nosy: +scoder

___
Python tracker 
<https://bugs.python.org/issue43760>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44054] 2**53+1 != float(2**53+1)

2021-05-06 Thread Stefan Behnel


Stefan Behnel  added the comment:

> I really don't know why you would desire a different result.

I found it surprising that a comparison uses a different method of conversion 
than the (obvious) user-side conversion, with a different outcome. This seems 
to be implementation details leaking into the user side.


> "That's also what the code says."

I wasn't referring to a specific comment. What I meant was that the code in 
float_richcompare() goes to great length trying to convert the integer to a 
float in a safe way so that it can compare the two values.

https://github.com/python/cpython/blob/985ac016373403e8ad41f8d563c4355ffa8d49ff/Objects/floatobject.c#L403

I now see that it goes the other way at the end, though. If both values have 
the same order of magnitude, then it actually converts the float to a PyLong 
instead, thus choosing one of the integer values out of the value range that 
the float spans and comparing that. That's where the difference originates.


> If you want to compare values approximately as floats, you should explicitly 
> convert them to floats.

As I wrote, "I'm not sure if I should consider this a bug", because it's an 
area that we could just define as "out of bounds behaviour" and "user, you're 
on your own".

The net effect is that some integers will never equal a floating point value, 
even though the floating point value does its very best to represent that 
integer.

I can live with considering the current behaviour "as good as it gets, because 
there is no right way to do it".

Thank you for your comments.

--

___
Python tracker 
<https://bugs.python.org/issue44054>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44054] 2**53+1 != float(2**53+1)

2021-05-06 Thread Stefan Behnel


Change by Stefan Behnel :


--
nosy: +tim.peters

___
Python tracker 
<https://bugs.python.org/issue44054>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44054] 2**53+1 != float(2**53+1)

2021-05-06 Thread Stefan Behnel


New submission from Stefan Behnel :

I'm not sure if I should consider this a bug, but I'd at least frown at the 
behaviour, so I thought I'd bring this up here.

Python 3.8.5 (default, Jan 27 2021, 15:41:15) 
[GCC 9.3.0] on linux
>>> 2**53 == float(2**53)
True
>>> float(2**53+1) == float(2**53+1)
True
>>> 2**53+1 == float(2**53+1)
False

This probably has something to do with the 52bit exponent of double precision 
floats. But the way I would have expected this to work is that a comparison of 
an integer to a float would first convert the integer to a float, and then 
compare the two floating point values. That's also what the code says. However, 
comparing the actual two floating point values gives the expected result, 
whereas letting the comparison do the conversion internally leads to a 
different outcome. The code in float_richcompare() uses a vastly more complex 
implementation than PyLong_AsDouble(), which is likely the reason for this 
difference in behaviour.

I found this on the system Python on 64bit Ubuntu 20.04, but also tried with a 
self-built 3.10a7+, giving the same result. I'm only setting the target to 
3.10/11 since a potential behavioural change would likely not find its way back 
to 3.9 and earlier any more.

--
messages: 393077
nosy: mark.dickinson, rhettinger, scoder
priority: normal
severity: normal
status: open
title: 2**53+1  != float(2**53+1)
type: behavior
versions: Python 3.10, Python 3.11

___
Python tracker 
<https://bugs.python.org/issue44054>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28254] Add C API for gc.enable, gc.disable, and gc.isenabled

2021-04-28 Thread Stefan Behnel


Change by Stefan Behnel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue28254>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43959] Improve documentation of PyContextVar C-API

2021-04-28 Thread Stefan Behnel


Change by Stefan Behnel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue43959>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43959] Improve documentation of PyContextVar C-API

2021-04-28 Thread Stefan Behnel


Stefan Behnel  added the comment:


New changeset ff7266efd0ef6b42dad30c9c0d210f843cc44f39 by Miss Islington (bot) 
in branch '3.8':
bpo-43959: clarify the documentation of the PyContextVar C-API (GH-25671) 
(GH-25690)
https://github.com/python/cpython/commit/ff7266efd0ef6b42dad30c9c0d210f843cc44f39


--

___
Python tracker 
<https://bugs.python.org/issue43959>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43959] Improve documentation of PyContextVar C-API

2021-04-28 Thread Stefan Behnel


Stefan Behnel  added the comment:


New changeset fdb11897d7c8f9e6bdf96fcef802f784ef90f726 by Miss Islington (bot) 
in branch '3.9':
bpo-43959: clarify the documentation of the PyContextVar C-API (GH-25671) 
(GH-25689)
https://github.com/python/cpython/commit/fdb11897d7c8f9e6bdf96fcef802f784ef90f726


--

___
Python tracker 
<https://bugs.python.org/issue43959>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28254] Add C API for gc.enable, gc.disable, and gc.isenabled

2021-04-28 Thread Stefan Behnel


Stefan Behnel  added the comment:

I just remembered that it's usually helpful to return the previous state, so 
that callers know whether they need to re-enable or disable the GC when they're 
done. I'll add that.

Also, returning an "int" may allow us to add a "-1" error code at some point, 
if it turns out to become necessary.

--

___
Python tracker 
<https://bugs.python.org/issue28254>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28254] Add C API for gc.enable, gc.disable, and gc.isenabled

2021-04-28 Thread Stefan Behnel


Change by Stefan Behnel :


--
nosy: +scoder
nosy_count: 3.0 -> 4.0
pull_requests: +24377
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/25687

___
Python tracker 
<https://bugs.python.org/issue28254>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43959] Improve documentation of PyContextVar C-API

2021-04-27 Thread Stefan Behnel


Change by Stefan Behnel :


--
keywords: +patch
pull_requests: +24362
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25671

___
Python tracker 
<https://bugs.python.org/issue43959>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43959] Improve documentation of PyContextVar C-API

2021-04-27 Thread Stefan Behnel


New submission from Stefan Behnel :

The documentation of the PyContextVar C-API is unclear in a couple of places.

- It's not clear whether PyContextVar_Get() always returns an owned reference 
also for default values or only if a value was found.
- It's not explicit that "optional" actually means "pass an object reference or 
NULL".
- The return value of PyContextVar_Set() is not explained.

--
assignee: docs@python
components: Documentation
messages: 392128
nosy: docs@python, scoder
priority: normal
severity: normal
status: open
title: Improve documentation of PyContextVar C-API
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue43959>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22079] Ensure in PyType_Ready() that base class of static type is static

2021-04-20 Thread Stefan Behnel


Stefan Behnel  added the comment:

Coming back to this after a while. I would like to get rid of the work-around 
(read: huge hack) that we have in Cython for this check and thus would ask for 
the check to be removed in Py3.10.

According to the discussion, no-one seemed to remember why it was added in the 
first place, just that "bad things happened", but not what or how.

With the fix for issue 35810 in place since Py3.8, my guess is that the 
situation that this check is preventing has actually become safe now. Unless 
someone can motivate that the risks were other than a premature deallocation of 
the base type.

--

___
Python tracker 
<https://bugs.python.org/issue22079>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28254] Add C API for gc.enable, gc.disable, and gc.isenabled

2021-04-17 Thread Stefan Behnel


Stefan Behnel  added the comment:

I support that there should be a simple way to do this from C. The way via 
importing the "gc" module and then looking up an attribute (possibly building a 
Unicode string) and calling a function in it involves several operations that 
can take some time and require useless error handling since each of them will 
practically never fail but may. An operation as simple as changing the GC 
status shouldn't require that.

A use case is building large data structures, or critical rearrangements of 
data structures that involve object operations that might trigger a GC run, 
maybe even including temporarily invalid object states and graphs. When larger 
data structures are involved but no collectable cycles, then the GC will 
probably not do anything useful except slowing down the creation process and/or 
running arbitrary code in between.

I consider the need to disable garbage collection in critical sections of C 
code similar to locking and GIL handling. It isn't quite the same as locking, 
since other code can enable it again when it runs, which isn't the case for a 
lock, but especially in that case, being able to detect quickly whether it was 
re-enabled when my own code gains back control seems beneficial. Having to call 
a Python function for that and taking care of the object result is way too much 
overhead for this case.

The fact that this is a rare request may not necessarily mean that it's rarely 
needed. There is certainly a bunch of C code out there that would benefit from 
temporarily disabling the GC in critical sections. I would imagine that people 
simply don't think of doing it and fail to notice any resulting slow-downs or 
even crashes since those often require elaborate circumstances to occur, and 
thus may not become visible at all in test or benchmark scenarios.

Note that the GC state is now part of the PyInterpreterState, so a patch would 
need to do what "gc_enable_impl" and "gc_disable_impl" do in 
Modules/gcmodule.c, or, rather, become their implementation to share code.

--
components: +C API -Extension Modules
nosy: +scoder
resolution: rejected -> 
stage:  -> needs patch
status: closed -> open
type:  -> resource usage
versions: +Python 3.10 -Python 3.7

___
Python tracker 
<https://bugs.python.org/issue28254>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43816] Missing 'extern "C"' for _Py_ctype_table

2021-04-13 Thread Stefan Behnel


Stefan Behnel  added the comment:

The macros were moved to "Includes/cpython/", not to "Includes/internal/". That 
suggests to me that they should use "extern C", so that C++ code that wants to 
make use of CPython internals can use them.

Basically, the way I see it, *all* header files in "Includes/" and 
"Includes/cpython/" should work with C++ code and thus have an "extern C". Only 
the header files in "Includes/internal/" should not have them.

Regardless, I've removed the macro usage from Cython so that the current state 
doesn't hurt our users.

--

___
Python tracker 
<https://bugs.python.org/issue43816>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40066] Enum: modify __repr__, __str__; update docs

2021-04-12 Thread Stefan Behnel

Stefan Behnel  added the comment:

> why /shouldn't/ we make the change?

It breaks doctests, and probably some other unit tests, too, e.g. for output 
formatting.

What should we suggest users to do? Replace

>>> get_flag(…)


by

>>> get_flag(…) == app_enums.TrickyFlag  or get_flag(…)  # (also show 
result on failure)
True

and

assertEqual(
"You caught the %s flag!" % result,
"You caught the app_enums.TrickyFlag flag!")

by

assertEqual(
("You caught the %s flag!" % result).replace("app_enums.", ""),
"You caught the TrickyFlag flag!")

?

Note that using "%r" does not help, since it's also backwards incompatible.

For their own enums, users can probably backport (or forward-port) "__str__()" 
and "__repr__()" themselves in order to work around this difference. But it's 
something they would have to do.

I certainly understand the reasoning, and it also makes new Py3.10-only 
doctests nicer, actually, but IMHO it counts as deliberate breakage.

--
nosy: +scoder

___
Python tracker 
<https://bugs.python.org/issue40066>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43816] Missing 'extern "C"' for _Py_ctype_table

2021-04-12 Thread Stefan Behnel


Stefan Behnel  added the comment:

These macros are a sort-of documented part of the C-API. At least, they were 
mentioned in a What's New document:

https://docs.python.org/3/whatsnew/2.7.html?highlight=py_isspace#build-and-c-api-changes

They were added here, for the Py2.7 release:

https://github.com/python/cpython/commit/8374981fb4d781d5ddbca313656bd3f32b49cef4

It looks to me like the header file should use "extern C".
@Eric, do you agree?

--
nosy: +eric.smith, scoder
type:  -> compile error
versions: +Python 3.10

___
Python tracker 
<https://bugs.python.org/issue43816>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43618] random.shuffle loses most of the elements

2021-03-25 Thread Stefan Behnel


Stefan Behnel  added the comment:

Yes, this is neither a bug in CPython (or its stdlib) nor in lxml. It's how 
things work. Don't use these two together.

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue43618>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43616] random.shuffle() crashes with Unhandled exception

2021-03-25 Thread Stefan Behnel


Stefan Behnel  added the comment:

Almost certainly not a bug in CPython's stdlib.

Possibly something worth investigating in lxml, although, as stated in issue 
43618, random.shuffle() simply does not work on lxml.etree Elements.

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue43616>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43399] xml.etree.ElementTree.extend does not work with iterators when using the Python implementation

2021-03-04 Thread Stefan Behnel


Stefan Behnel  added the comment:

Thanks for the report and the PR. I would argue that this can still be fixed in 
Py3.8, given the low impact (but not earlier). The behaviour is clearly wrong 
(in a subtle way) and most users won't notice it because they won't switch off 
the C accelerator module.

--
versions:  -Python 3.6, Python 3.7

___
Python tracker 
<https://bugs.python.org/issue43399>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40624] add support for != (not-equals) in ElementTree XPath

2020-11-08 Thread Stefan Behnel


Change by Stefan Behnel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue40624>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40624] add support for != (not-equals) in ElementTree XPath

2020-11-08 Thread Stefan Behnel


Stefan Behnel  added the comment:


New changeset 97e8b1eaeaf3aa325c84ff2e13417c30414d0269 by Ammar Askar in branch 
'master':
bpo-40624: Add support for the XPath != operator in xml.etree (GH-22147)
https://github.com/python/cpython/commit/97e8b1eaeaf3aa325c84ff2e13417c30414d0269


--

___
Python tracker 
<https://bugs.python.org/issue40624>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42151] Pure Python xml.etree.ElementTree is missing default attribute values

2020-10-28 Thread Stefan Behnel


Stefan Behnel  added the comment:

In general, since the C accelerator is enabled by default, and few people would 
consider disabling it explicitly, I generally consider the behaviour of the C 
implementation to be "right", if both implementations differ.

As a single data point, the reason why the difference was found in this case 
was differing behaviour in PyPy (which uses only the Python implementation). It 
was only later found to be a problem on the CPython side.

Changing the behaviour of the C implementation would certainly break a lot more 
code than changing the Python implementation.

--

___
Python tracker 
<https://bugs.python.org/issue42151>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42151] Pure Python xml.etree.ElementTree is missing default attribute values

2020-10-26 Thread Stefan Behnel


Stefan Behnel  added the comment:

The patch looks right. I'm not sure if this can still be changed in Py3.8, 
though, since that has been around for quite a while now.

Admittedly, few people will disable the C accelerator module and thus whitness 
this issue, but for them, this is a breaking change, and some code might rely 
on the current behaviour. I have no way to tell how much, and whether it 
intentionally relies on it.

I'd definitely change this for 3.9 and later. Maybe for 3.8, but it's at least 
a bit of a risk, given that there will only be very few more minor releases for 
it, and given that this is how things have been working for years. So, rather 
not, unless there is a convincing argument for backporting the change.

--
components: +XML -Library (Lib)
versions:  -Python 3.6, Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue42151>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-10-11 Thread Stefan Behnel


Stefan Behnel  added the comment:

> Don't forget to remove PyGen_Send()

That's the function that allows sending data into a generator. It's also used 
internally by "PyIter_Send()". Are you really suggesting to remove it, or to 
make it underscore-private again? (As it was before.)

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41892] use both "for in" and "ElementTree.remove" has a index bug

2020-10-04 Thread Stefan Behnel


Stefan Behnel  added the comment:

Closing since this works as designed.

Users are responsible for avoiding concurrent tree modifications during 
iteration.

--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41892] use both "for in" and "ElementTree.remove" has a index bug

2020-10-04 Thread Stefan Behnel


Change by Stefan Behnel :


--
pull_requests: +21547
pull_request: https://github.com/python/cpython/pull/22546

___
Python tracker 
<https://bugs.python.org/issue41892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33802] Regression in logging configuration

2020-10-03 Thread Stefan Behnel


Change by Stefan Behnel :


--
nosy:  -scoder

___
Python tracker 
<https://bugs.python.org/issue33802>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6721] Locks in the standard library should be sanitized on fork

2020-10-03 Thread Stefan Behnel


Change by Stefan Behnel :


--
nosy:  -scoder

___
Python tracker 
<https://bugs.python.org/issue6721>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33802] Regression in logging configuration

2020-10-03 Thread Stefan Behnel


Change by Stefan Behnel :


--
pull_requests:  -21521

___
Python tracker 
<https://bugs.python.org/issue33802>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

2020-10-03 Thread Stefan Behnel


Change by Stefan Behnel :


--
nosy:  -scoder

___
Python tracker 
<https://bugs.python.org/issue36533>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6721] Locks in the standard library should be sanitized on fork

2020-10-03 Thread Stefan Behnel


Change by Stefan Behnel :


--
pull_requests:  -21519

___
Python tracker 
<https://bugs.python.org/issue6721>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

2020-10-03 Thread Stefan Behnel


Change by Stefan Behnel :


--
pull_requests:  -21520

___
Python tracker 
<https://bugs.python.org/issue36533>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33802] Regression in logging configuration

2020-10-03 Thread Stefan Behnel


Change by Stefan Behnel :


--
nosy: +scoder
nosy_count: 9.0 -> 10.0
pull_requests: +21521
pull_request: https://github.com/python/cpython/pull/22474

___
Python tracker 
<https://bugs.python.org/issue33802>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

2020-10-03 Thread Stefan Behnel


Change by Stefan Behnel :


--
nosy: +scoder
nosy_count: 9.0 -> 10.0
pull_requests: +21520
pull_request: https://github.com/python/cpython/pull/22474

___
Python tracker 
<https://bugs.python.org/issue36533>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6721] Locks in the standard library should be sanitized on fork

2020-10-03 Thread Stefan Behnel


Change by Stefan Behnel :


--
nosy: +scoder
nosy_count: 29.0 -> 30.0
pull_requests: +21519
pull_request: https://github.com/python/cpython/pull/22474

___
Python tracker 
<https://bugs.python.org/issue6721>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41900] XML C14N serialisation fails with default namespace

2020-10-03 Thread Stefan Behnel


Change by Stefan Behnel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41900>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41900] XML C14N serialisation fails with default namespace

2020-10-03 Thread Stefan Behnel


Stefan Behnel  added the comment:


New changeset a0f2b664335eb689abdecc677e09a193f503af59 by Miss Skeleton (bot) 
in branch '3.9':
bpo-41900: C14N 2.0 serialisation failed for unprefixed attributes when a 
default namespace was defined. (GH-22474) (GH-22507)
https://github.com/python/cpython/commit/a0f2b664335eb689abdecc677e09a193f503af59


--

___
Python tracker 
<https://bugs.python.org/issue41900>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41900] XML C14N serialisation fails with default namespace

2020-10-03 Thread Stefan Behnel


Stefan Behnel  added the comment:


New changeset cfed5343331350c5737b464970a31f7588319e8b by Miss Skeleton (bot) 
in branch '3.8':
bpo-41900: C14N 2.0 serialisation failed for unprefixed attributes when a 
default namespace was defined. (GH-22474) (GH-22508)
https://github.com/python/cpython/commit/cfed5343331350c5737b464970a31f7588319e8b


--

___
Python tracker 
<https://bugs.python.org/issue41900>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41900] XML C14N serialisation fails with default namespace

2020-10-03 Thread Stefan Behnel


Stefan Behnel  added the comment:


New changeset 6a412c94b6b68e7e3632562dc7358a12ffd1447f by scoder in branch 
'master':
bpo-41900: C14N 2.0 serialisation failed for unprefixed attributes when a 
default namespace was defined. (GH-22474)
https://github.com/python/cpython/commit/6a412c94b6b68e7e3632562dc7358a12ffd1447f


--

___
Python tracker 
<https://bugs.python.org/issue41900>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41892] use both "for in" and "ElementTree.remove" has a index bug

2020-10-02 Thread Stefan Behnel


Change by Stefan Behnel :


--
versions: +Python 3.10, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue41892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41900] XML C14N serialisation fails with default namespace

2020-10-01 Thread Stefan Behnel


Change by Stefan Behnel :


--
keywords: +patch
pull_requests: +21493
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/22474

___
Python tracker 
<https://bugs.python.org/issue41900>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41900] XML C14N serialisation fails with default namespace

2020-10-01 Thread Stefan Behnel


New submission from Stefan Behnel :

import xml.etree.ElementTree as ET
xml="""
http://soap.sforce.com/2006/04/metadata;>

"""
print(ET.canonicalize(xml))

Fails with:
ValueError: Namespace "" is not declared in scope
when trying to build the QName of the unnamespaced "targets" attribute.

Originally reported for lxml here:
https://bugs.launchpad.net/lxml/+bug/1869455

--
components: XML
messages: 377743
nosy: scoder
priority: normal
severity: normal
stage: needs patch
status: open
title: XML C14N serialisation fails with default namespace
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue41900>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31256] xml.etree.ElementTree: add support for doctype in tostring method

2020-10-01 Thread Stefan Behnel


Stefan Behnel  added the comment:

Yes, it fixed already. Thanks!

--
nosy: +scoder
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue31256>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41899] Poor example for Element.remove()

2020-10-01 Thread Stefan Behnel


Stefan Behnel  added the comment:

Closing as duplicate of issue 41891. Let's keep the discussion there.

--
nosy: +scoder
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41899>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41899] Poor example for Element.remove()

2020-10-01 Thread Stefan Behnel


Change by Stefan Behnel :


--
Removed message: https://bugs.python.org/msg377740

___
Python tracker 
<https://bugs.python.org/issue41899>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41899] Poor example for Element.remove()

2020-10-01 Thread Stefan Behnel


Stefan Behnel  added the comment:

Closing as duplicate of issue 41892. Let's keep the discussion there.

--

___
Python tracker 
<https://bugs.python.org/issue41899>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41892] use both "for in" and "ElementTree.remove" has a index bug

2020-09-30 Thread Stefan Behnel


Stefan Behnel  added the comment:

@WoodyWoo, you can (and should) do something like this:

for ELEMchild in list(etroot):

Modifying a container while iterating over it is usually not a good idea. The 
same applies to Python lists and dicts, for example.

--
stage: patch review -> 

___
Python tracker 
<https://bugs.python.org/issue41892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41892] use both "for in" and "ElementTree.remove" has a index bug

2020-09-30 Thread Stefan Behnel


Change by Stefan Behnel :


--
stage:  -> patch review

___
Python tracker 
<https://bugs.python.org/issue41892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41892] use both "for in" and "ElementTree.remove" has a index bug

2020-09-30 Thread Stefan Behnel


Change by Stefan Behnel :


--
keywords: +patch
pull_requests: +21489
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22464

___
Python tracker 
<https://bugs.python.org/issue41892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41892] use both "for in" and "ElementTree.remove" has a index bug

2020-09-30 Thread Stefan Behnel


Stefan Behnel  added the comment:

> That example is especially problematic.

No, it's not. It uses .findall(), which returns a list. It's like when you make 
a copy of a list to iterate over, when you want to modify the original list in 
the loop.

That could be made explicit in the text that introduces the example, but I 
think it's a very good example.

--

___
Python tracker 
<https://bugs.python.org/issue41892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41295] CPython 3.8.4 regression on __setattr__ in multiinheritance with metaclasses

2020-09-29 Thread Stefan Behnel


Change by Stefan Behnel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41295>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38225] iscoroutinefunction broken with cython - allow tagging of functions as async?

2020-09-22 Thread Stefan Behnel


Stefan Behnel  added the comment:

FYI, https://github.com/cython/cython/pull/3427 has been merged into Cython. In 
Cython 3.0, compiled coroutines will disguise as non-compiled coroutines, from 
the PoV of asyncio.

Restricting this ticket to Py3.10 since we're rather discussing a new feature 
now.

--
versions: +Python 3.10 -Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue38225>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39960] Using typename.__setattr__ in extension type with Py_TPFLAGS_HEAPTYPE is broken (hackcheck too eager?)

2020-09-20 Thread Stefan Behnel


Stefan Behnel  added the comment:

Closing again since GH-21528 has been merged in issue 41295.

--
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39960>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Stefan Behnel


Stefan Behnel  added the comment:

BTW, just to give this a house number, I remember having measured a performance 
improvement of up to 70% at some point when switching from "generators always 
raise a StopIteration at the end" to "generators just return NULL" in Cython. 
For short-running generators and coroutines, this can make a big difference.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Stefan Behnel


Stefan Behnel  added the comment:

I would also have preferred a more type specific function, but yeah, as long as 
the types for which the function would normally be used are special cased early 
enough in the implementation, it makes no big difference.

Fine with me, too.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Stefan Behnel


Stefan Behnel  added the comment:

I'm happy to see this moving forward.

Not convinved of the "PyIter_Send()" name, though. I don't consider this part 
of the iterator protocol. It's specific to generators and coroutines. Cython 
would probably guard its usage by "PyGen_CheckExact()" or 
"PyCoro_CheckExact()", and not use it for arbitrary iterators.

Since coroutines inherit the generator protocol more or less, I think 
"PyGen_Send()" is a more suitable name, better than "PyCoro_Send()".

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-11 Thread Stefan Behnel


Stefan Behnel  added the comment:

Copying some of the design discussion from the PR here 
(https://github.com/python/cpython/pull/22196/files#r486730457), because it 
belongs into the ticket.

Yury Selivanov proposed to add a new C-API function for this (naming changes by 
me):

typedef enum {PYGEN_RETURN, PYGEN_ERROR, PYGEN_NEXT} PyGenSendStatus;

PyGenSendStatus PyGen_Send(PyGenObject *gen, PyObject *arg, PyObject 
**result);

Mark Shannon and I agreed that the status code should be the return value, with 
some confusion whether "PyGen_" or "PyCoro_" would be appropriate prefixes.

Mark Shannon wrote: I don't think [the C-API function] should be public, as a 
possible further improvement is to stop passing exceptions through a side 
channel, but in result. Maybe we don't want to do that, but lets' not add to 
the (already rather large) C-API.

However, I think this will be demanded and used by extensions, including Cython 
implemented ones, so it seems better to make them use a public function than a 
private one.

Let's continue these lines of discussion here.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-11 Thread Stefan Behnel


Stefan Behnel  added the comment:

Big +1 from me, too, for the same reasons Yury gave.

--
nosy: +scoder

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20198] xml.etree.ElementTree.ElementTree.write attribute sorting

2020-09-07 Thread Stefan Behnel


Change by Stefan Behnel :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue20198>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19483] Allow more low-level parser configuration in ElementTree

2020-09-07 Thread Stefan Behnel


Stefan Behnel  added the comment:

Changing subject to make it clear what this ticket is really about.

--
title: Pure-Python ElementTree classes no longer available since 3.3 -> Allow 
more low-level parser configuration in ElementTree
versions: +Python 3.10 -Python 3.3, Python 3.4, Python 3.5

___
Python tracker 
<https://bugs.python.org/issue19483>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39714] ElementTree parser limitation of input string size

2020-09-07 Thread Stefan Behnel


Change by Stefan Behnel :


--
title: ElementTree limitation -> ElementTree parser limitation of input string 
size

___
Python tracker 
<https://bugs.python.org/issue39714>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39714] ElementTree limitation

2020-09-07 Thread Stefan Behnel


Stefan Behnel  added the comment:

I'd suggest feeding the data into the parser in chunks, or letting it read from 
a file-like object, or something like that.

Also, you probably want to do incremental processing on the data (see the 
XMLPullParser and iterparse), because reading 3.5GB of XML data into an 
in-memory tree can easily result in 10x the memory usage. You may have 40GB of 
RAM on your machine, but even then, I'd still recommend processing the data in 
incrementally.

--
nosy: +scoder
versions: +Python 3.10, Python 3.9 -Python 3.7

___
Python tracker 
<https://bugs.python.org/issue39714>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40624] add support for != (not-equals) in ElementTree XPath

2020-09-07 Thread Stefan Behnel


Stefan Behnel  added the comment:

Agreed that adding "!=" would be a nice improvement, probably not very 
invasive. PR welcome.

https://www.w3.org/TR/xpath-10/#booleans

--
nosy: +scoder
stage:  -> needs patch
type:  -> enhancement
versions: +Python 3.10 -Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40624>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41566] Include much faster DEFLATE implementations in Python's gzip and zlib libraries. (isa-l)

2020-08-20 Thread Stefan Behnel


Stefan Behnel  added the comment:

> On the whole I think the arguments to make a module are very strong. So I 
> think that is the appropriate way forward.

If you take this route, please don't write it directly against the CPython 
C-API (as you would for a CPython stdlib module). It is much easier to get 
something working, correct, testable and feature-rich with Cython. Once you 
have that, put it on PyPI to give it more testing. If you then decide to come 
back and provide a patch to get it integrated into CPython's zlib module, it's 
relatively easy to manually translate the Cython code to equivalent C-API code. 
But by that time, your code will be known to work, will have a test suite, and 
we can run real benchmarks on it before investing the time into spelling it out 
in C.

--

___
Python tracker 
<https://bugs.python.org/issue41566>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41566] Include much faster DEFLATE implementations in Python's gzip and zlib libraries. (isa-l)

2020-08-17 Thread Stefan Behnel


Stefan Behnel  added the comment:

> libdeflate and isa-l use different compression ratio's for the levels.

I don't see why these would need to translate 1:1. The zlib module is a Python 
API wrapper, it can do its own mapping here, or use the libraries selectively 
only for some compression levels. Python code also cannot rely on an exact bit 
pattern coming out of the zlib/gzip compressor, since that might change with 
the zlib version that is available. So I think we're fine when replacing the 
underlying implementation, as long as the API does not change and the output is 
strictly zlib/gzip compatible (and there are no visible performance/size 
regressions, as your numbers seem to suggest, but that would need some broader 
testing).


You also wrote on python-ideas that

> It is packaged in linux distros already

That might be an option then. CPython could use the existing library if it is 
available. It doesn't have to ship the sources. Most Linux distributions 
already build some standard library modules against the system-wide installed 
libraries rather than whatever CPython ships in its sources. And those 
distributions could then make the library a fixed dependency of their CPython 
packages.

--

___
Python tracker 
<https://bugs.python.org/issue41566>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41566] Include much faster DEFLATE implementations in Python's gzip and zlib libraries. (isa-l)

2020-08-17 Thread Stefan Behnel


Stefan Behnel  added the comment:

What about building the library? The readme says it needs nasm? That's not a 
standard dependency for the CPython build currently, which relies solely on a C 
compiler.

--

___
Python tracker 
<https://bugs.python.org/issue41566>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41566] Include much faster DEFLATE implementations in Python's gzip and zlib libraries. (isa-l)

2020-08-17 Thread Stefan Behnel


Stefan Behnel  added the comment:

> This has to be in a PEP

No, the bug tracker seems fine for this.

--
nosy: +scoder
resolution: not a bug -> 
stage: resolved -> needs patch
status: closed -> open
type:  -> performance

___
Python tracker 
<https://bugs.python.org/issue41566>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35018] Sax parser provides no user access to lexical handlers

2020-08-09 Thread Stefan Behnel


Change by Stefan Behnel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue35018>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35018] Sax parser provides no user access to lexical handlers

2020-08-09 Thread Stefan Behnel


Stefan Behnel  added the comment:


New changeset e28b8c93878072dc02b116108ef5443084290d47 by Zackery Spytz in 
branch 'master':
bpo-35018: Sax parser should provide user access to lexical handlers (GH-20958)
https://github.com/python/cpython/commit/e28b8c93878072dc02b116108ef5443084290d47


--

___
Python tracker 
<https://bugs.python.org/issue35018>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41295] CPython 3.8.4 regression on __setattr__ in multiinheritance with metaclasses

2020-07-19 Thread Stefan Behnel


Stefan Behnel  added the comment:

> test_asyncio altered the execution environment

That happened several times before in previous builds. I think there's just a 
part of the asyncio tests that is unreliable. I left a comment in issue 41273 
since it might be related, the sporadic failures started appearing in the build 
following the merge.

--

___
Python tracker 
<https://bugs.python.org/issue41295>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41273] asyncio: proactor read transport: use recv_into instead of recv

2020-07-19 Thread Stefan Behnel


Stefan Behnel  added the comment:

There have been sporadic buildbot failures in "test_asyncio" since this change, 
message being "1 test altered the execution environment", e.g.

https://buildbot.python.org/all/#/builders/129/builds/1443

Could someone please check if they are related?

--
nosy: +scoder

___
Python tracker 
<https://bugs.python.org/issue41273>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41295] CPython 3.8.4 regression on __setattr__ in multiinheritance with metaclasses

2020-07-18 Thread Stefan Behnel


Stefan Behnel  added the comment:

PR 21528 works for all four test cases that we now have (1x test_capi.py, 3x 
test_descr.py).

Any comments? We need to merge a fix before Monday to meet the deadline of the 
planned hotfix release.

@kam193, could you please also test that change with your real code?

--

___
Python tracker 
<https://bugs.python.org/issue41295>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39960] Using typename.__setattr__ in extension type with Py_TPFLAGS_HEAPTYPE is broken (hackcheck too eager?)

2020-07-18 Thread Stefan Behnel


Stefan Behnel  added the comment:

I pushed PR 21528 with a new proposal. See issue 41295.

--

___
Python tracker 
<https://bugs.python.org/issue39960>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41295] CPython 3.8.4 regression on __setattr__ in multiinheritance with metaclasses

2020-07-18 Thread Stefan Behnel


Change by Stefan Behnel :


--
pull_requests: +20664
pull_request: https://github.com/python/cpython/pull/21528

___
Python tracker 
<https://bugs.python.org/issue41295>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39960] Using typename.__setattr__ in extension type with Py_TPFLAGS_HEAPTYPE is broken (hackcheck too eager?)

2020-07-18 Thread Stefan Behnel


Stefan Behnel  added the comment:

> intermediate base type "object" in the hierarchy

Sorry, I meant an intermediate base type "B", which inherits its setattr from 
"object".

--

___
Python tracker 
<https://bugs.python.org/issue39960>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39960] Using typename.__setattr__ in extension type with Py_TPFLAGS_HEAPTYPE is broken (hackcheck too eager?)

2020-07-18 Thread Stefan Behnel


Stefan Behnel  added the comment:

The problem in the test added in PR 21473 is that there is an intermediate base 
type "object" in the hierarchy:

class A(type):
def __setattr__(cls, key, value):
type.__setattr__(cls, key, value)

class B:
pass

class C(B, A):
pass

>>> [c for c in C.__mro__]
[, , , , ]

>>> [c.__setattr__ for c in C.__mro__]
[, , , , ]

I think the change to the algorithm there is too broad, it disables much of 
what the check was written for (or against).

Given Guido's second (negative) test case, I think it would also not be correct 
to add "object.__setattr__" to the list of allowed (intermediate) slot methods:

class A(type):
def __setattr__(cls, key, value):
object.__setattr__(cls, key, value)   # this should fail!

class B:
pass

class C(B, A):
pass

It's difficult to turn this into an algorithm. Is the MRO really the place to 
look? For "A", we're only really interested in the C-level bases, aren't we?

--

___
Python tracker 
<https://bugs.python.org/issue39960>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39960] Using typename.__setattr__ in extension type with Py_TPFLAGS_HEAPTYPE is broken (hackcheck too eager?)

2020-07-05 Thread Stefan Behnel


Stefan Behnel  added the comment:

Fixed in 3.8+. Closing.
Thanks for the feedback.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39960>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   3   4   5   6   7   8   9   10   >