Stefan Behnel added the comment:
Thanks, Yury!
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25082>
___
___
Python-bugs-list
Stefan Behnel added the comment:
Could I get a little note in the collections.abc section that the new ABCs
"Generator", "Coroutine" and "Awaitable" are available as backports in the
"backports_abc" package on PyPI?
https://pypi.python.org/pypi/backpo
Stefan Behnel added the comment:
Patch and test look correct. They fix a bug that produces incorrect output, so
I vote for backporting them. Most code won't see the difference as whitespace
control characters are rare in attribute values. And code that uses them will
benefit from correctness
Stefan Behnel added the comment:
> The only thing that was a bit mystifying were the errors during the
> initial profile run. There is so much that floats by in the terminal
> window that I completely missed the warnings about errors during the
> test run not being anything to
Stefan Behnel added the comment:
Issue 24915 suggests PGO and comes with an actual patch. I suggest rejecting
this ticket as too broad.
--
nosy: +scoder
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17781
Stefan Behnel added the comment:
Please upload your patches as separate, uncompressed files for review.
PGO was already proposed here, but nothing came out of it:
https://bugs.python.org/issue17781
I suggest rejecting that old ticket and sticking with this one since it has an
actual patch
Stefan Behnel added the comment:
The can store arbitrary objects sentence is now duplicated, and still way too
visible. I have to read three sentences until it tells me what I need to know.
--
___
Python tracker rep...@bugs.python.org
http
Stefan Behnel added the comment:
I think the first two sentences can simply be removed to fix this, without loss
of readability or information.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24079
Stefan Behnel added the comment:
Also, the uptake for SNs are nearly zero.
That suggests to me that pointing users to it could help. It's currently hidden
in the types module and if I didn't know it existed, I could end up looking in
collections, and failing to find it there, write my own
Stefan Behnel added the comment:
The fix wasn't applied yet, so the current code in 3.4 and later branches is
still incorrect. Any of the last two patches (*_value) will fix it, with my
preference on the last one.
--
versions: +Python 3.6
Stefan Behnel added the comment:
Regarding tests, it looks like iteration isn't currently tested at the C
level at all. At least, the xx test modules don't have any types that use
it. I can write one up next week, or add it to one of the existing types
(Xxo_Type?). Unlikely that I'll make
Stefan Behnel added the comment:
It would be nice to have this applied to 3.5 even. I'm aware that this is a
new feature that shouldn't go in any more at this point, but if it could
get added in 3.5.1 at least, I think it would fill a rather clear and
annoying gap when it comes to tool
Stefan Behnel added the comment:
I find it questionable to mix await and threads, as I have said several
times in the discussion about Nick Coghlan's proposal to introduce
helper functions with a similar function.
There are a couple of use cases in the context of asyncio, but definitely
also
Stefan Behnel added the comment:
Funny. I just thought about this yesterday and came up with pretty much the
same idea. +1 for the patch.
I don't think there are more classes to support here. Quite the contrary,
other tools should try to integrate via concurrent.futures.Future instead
Stefan Behnel added the comment:
Whenever you feel a need for using range(len(...)), you are almost always doing
something wrong. The problem in your code is that you are modifying the tree
while iterating over it. However, please ask this question on the mailing list
as the bug tracker
Stefan Behnel added the comment:
could we apply this patch, please?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24079
___
___
Python-bugs
Stefan Behnel added the comment:
Here's a complete drop-in replacement.
More specific constraints on which elements to look for and where to look for
them in the tree can be expressed in :ref:`XPath elementtree-xpath`.
:meth:`Element.iterfind` iterates over all elements matching such a path
Stefan Behnel added the comment:
The proposed patch downplays that generality.
That is completely intentional. Almost all readers of the documentation will
first need to understand the difference between text and tail before they can
go and think about any more advanced use cases
Stefan Behnel added the comment:
Note that the expected usage is not as a function but as a decorator. That
should be stated in the docs as well. IMHO, users should only do two things
with whatever the result is: either use it as a Generator (as before), or pass
it as an argument to await
Stefan Behnel added the comment:
Hm, I think there's little need for new exceptions...
While I agree with Yuri that the names are a bit awkward, I actually second
this. The StopIteration is almost an implementation detail of how the return
value is passed on to become the (Future) result
Stefan Behnel added the comment:
Thanks for updating the micro-benchmark. Just FYI (and sorry for hijacking this
ticket), I ran it through Cython. Here are the numbers:
Cython 0.23 (latest master)
binary(21) * 3: total 1.609s
abinary(21) * 3: total 1.514s
CPython 3.5 (latest branch)
binary(21
Stefan Behnel added the comment:
Benchmark results look good to me (although a header line is missing) and seem
to match my expectations. Sounds like we should allow this change.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org
Stefan Behnel added the comment:
Hmm, right. I see your point and also the analogy with yield. I could live
with (maybe) giving it a better error message suggesting to use parentheses for
clarity and otherwise keeping it a SyntaxError
Stefan Behnel added the comment:
It looks like perfectly valid syntax to me and I cannot see why it should be
semantically rejected. I disagree that it shouldn't be changed. I think it's a
(minor) bug that should eventually get fixed.
--
nosy: +scoder
Stefan Behnel added the comment:
Your benchmarks are not affected by this change see the other issue.
Then you ran them, I assume? Could you show the output here that you got?
They are also not representative of every workload out there.
Certainly not, but they do provide a certain variety
Stefan Behnel added the comment:
I'm witnessing a crash in the C implementation during garbage collection.
Interestingly, it only shows in the Py3.6 branch, not in Py3.5 (both latest).
Here's the gdb session:
Program received signal SIGSEGV, Segmentation fault.
lru_cache_tp_traverse (self
Stefan Behnel added the comment:
It's not actually my own code using the lru_cache here. From a quick grep
over the code tree, the only potentially related usage I found was in
Python's fnmatch module, on the _compile_pattern() function. Commenting
that out then made the crash go away, so
Stefan Behnel added the comment:
test_fnmatch.py also passes, BTW.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14373
___
___
Python-bugs-list
Stefan Behnel added the comment:
It's generally worth running the benchmark suite for this kind of optimisation.
Being mostly Python code, it should benefit quite clearly from dictionary
improvements, but it should also give an idea of how much of an improvement
actual Python code
Stefan Behnel added the comment:
the undecorated form will never be visible to any code other than the
decorator
Assuming that 1) it's the first and/or only decorator, 2) it's used to
decorate a generator function that returns its own generator and 3) it's
really used as a decorator
Stefan Behnel added the comment:
1. abc.Coroutine and abc.Awaitable will guarantee that objects that implement
them have '__await__' and it's safe to access it (that means that they will
fail for generator-based coroutines).
Absolutely. That was the main theme behind the whole type split
Stefan Behnel added the comment:
This is not purely about speeding up the code. It's also about avoiding to
replace the code object of a function, which is essentially a big and clumsy
hack only to achieve setting a flag. Some tools, namely line_profiler, use the
current code object as a dict
Stefan Behnel added the comment:
isawaitable(), however, should continue using abc.Awaitable, since it only
checks for __await__ presence on the type (or should we just drop it?)
I'd really remove it. It's not referring to an actual type, so it doesn't fit
the purpose of the inspect module
Stefan Behnel added the comment:
I originally planned to make the next Cython release patch the Generator
and Coroutine ABCs into collections.abc, but I now think it would be worth
uploading an abc_backports package to PyPI instead that does that and on
which asyncio, tornado and other
Stefan Behnel added the comment:
I added three more comments to the review. The rest looks good to me, too.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24400
New submission from Stefan Behnel:
A Cython user noticed a memory leak when C-inheriting from int.
http://thread.gmane.org/gmane.comp.python.cython.devel/15689
The Cython code to reproduce this is simply this:
cdef class ExtendedInt(int): pass
for j in xrange(1000
Stefan Behnel added the comment:
No problem for Cython either.
The change in issue 24400 that makes coroutines real Awaitables also removes
surprises for a cr_await return value being a coroutine and previously *not*
an Awaitable.
The contract for gi_yieldfrom is only that the returned value
Changes by Stefan Behnel sco...@users.sourceforge.net:
--
nosy: +scoder
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24450
___
___
Python-bugs
Changes by Stefan Behnel sco...@users.sourceforge.net:
--
nosy: +pitrou, serhiy.storchaka
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24469
Stefan Behnel added the comment:
1) The intended solution is to require that int subclasses override tp_free.
In the way I showed? By calling either PyObject_GC_Del() or PyObject_Del()
directly? I'll happily do that (Py2.7 and Py2 int being dead ends makes
that pretty future proof), but it's
Stefan Behnel added the comment:
I agree that a typedef is a good idea. It doesn't cost anything but gives
us more freedom for future changes.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24400
Changes by Stefan Behnel sco...@users.sourceforge.net:
Added file: http://bugs.python.org/file39692/fix_stopiteration_value.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23996
Stefan Behnel added the comment:
Here are two patches that fix this case, one with special casing, one without.
Please choose and apply one.
--
Added file: http://bugs.python.org/file39691/fix_stopiteration_value_slow.patch
___
Python tracker rep
Stefan Behnel added the comment:
I currently do isinstance(x, types.GeneratorType), which will fail if x
is actually a GeneratorWrapper. I can change this to use
collections.abc.Generator when it exists, but then I'd have to have some
conditional logic to switch between collections.abc
Stefan Behnel added the comment:
See issue 24400 regarding a split of yield-based generators and async-def
coroutines at a type level.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24017
Stefan Behnel added the comment:
I added some review comments. The main thing is that the coroutine type should
be awaitable.
For reference, here's my current implementation for Cython. It's a bit more
involved as it needs to support Python 2.6+. I may also add some special casing
Stefan Behnel added the comment:
(Copying my review comment here so that it doesn't get lost as it's more of a
general design thing than a specific review comment.)
Having Future know and deal with asyncio seems wrong to me. asyncio should be
able to deal *somehow* with a Future that is being
Stefan Behnel added the comment:
Looks good to me.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24079
___
___
Python-bugs-list mailing list
Stefan Behnel added the comment:
Cython functions that return a generator aren't special in any way, so it's
actually correct that isgeneratorfunction() always returns False. I mean, I
could set the CO_COROUTINE flag on the code object that we fake, but that
wouldn't help much as Cython's
Stefan Behnel added the comment:
Hmm, I just noticed that this only started failing when I disabled the asyncio
module patching for 3.5beta1+, assuming that it now all works. A side effect of
that was that also the inspect module is no longer patched into returning True
from isgenerator
Stefan Behnel added the comment:
... except that the returned object may not actually be a Cython generator but
a GeneratorWrapper, now that the patch from issue 24316 is in place. :)
Then I guess we're back to the point where duck-typing and calling __await__()
is a good idea
Stefan Behnel added the comment:
Works for me.
Why is the AwaitableABC type check needed, in addition to looking up the
relevant method? IIUC, the type check will just do a lookup of the same method
if the type hasn't been registered as Awaitable explicitly.
--
status: pending - open
Stefan Behnel added the comment:
I found one more place in asyncio.coroutines, around line 190 in the
coroutine() decorator:
if inspect.isgeneratorfunction(func):
coro = func
else:
@functools.wraps(func)
def coro(*args, **kw):
res = func(*args, **kw
Stefan Behnel added the comment:
I would still like to see a patch in Py3.5 that only flips the bit in C when
_types is available and otherwise falls back to the existing Python code that
creates a new code object. This part is much cleaner and faster to do in C than
in the current Python
Stefan Behnel added the comment:
I'm seeing crashes with this assertion in Cython test modules:
python: ./Python/importdl.c:75: get_encoded_name: Assertion
`(((PyObject*)(encoded))-ob_refcnt) == 1' failed.
The problem seems to be that the module name is only one character long
Stefan Behnel added the comment:
Patch LGTM and it fixes the problem (tried it on my side). Please make sure it
gets applied in time for beta 2.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24268
Stefan Behnel added the comment:
Tried it, works for me. Thanks!
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24017
___
___
Python-bugs-list
Stefan Behnel added the comment:
I just noticed that I hadn't used the real types.coroutine in my Py3.5 tests
when reporting back in issue 24017. When I pass a Cython generator through it,
I get
Traceback (most recent call last):
File tests/run/test_coroutines_pep492.pyx, line 245
Stefan Behnel added the comment:
One failing test in test_coroutines: test_func_5. The reason is that the
GeneratorWrapper is not iterable (and there is no reason it shouldn't be, given
that it wraps a Generator). That was my fault, I had already added an __iter__
method but didn't copy
Stefan Behnel added the comment:
BTW, it's not only for compiled generators but also for normal Python functions
that construct Python generators internally and return them, or that delegate
the generator creation in some way. With this change, it's enough to decorate
the constructor function
Stefan Behnel added the comment:
IMHO less clear and less correct than the previous suggestions.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24079
Stefan Behnel added the comment:
Seems like a good idea to explain text and tail in one section, though.
That makes tail easier to find for those who are not used to this kind of
split (and that's basically everyone who needs to read the docs in the first
place
Stefan Behnel added the comment:
Ok, now the problem with *this* patch is that __iter__ and __await__ are
special methods that are being looked up on the type, not the instance.
Similarly __next__, I think, as it also has its own (type) slot. But I think
you're right that __next__ is also
Stefan Behnel added the comment:
Your latest patch works for me.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24316
___
___
Python-bugs-list
Stefan Behnel added the comment:
Yield-from iterates, and a coroutine is not supposed to be iterable, only
awaitable (at least, that's what all error messages tell me when I try it).
So why should yield from work on them? What if foo() was not an Iterable
but a Coroutine? Should yield
Stefan Behnel added the comment:
BTW, given that iter(iterator) works and returns the iterator, should we also
allow await x.__await__() to work? I guess that would be tricky to achieve
given that __await__() is only required to return any kind of arbitrary
Iterator, and Iterators cannot
Stefan Behnel added the comment:
Can't your Coroutine object return itself from its __await__, and implement
__next__? Like genobject in CPython simply returns self from its __iter__.
That was my first try, sure, and it mostly worked. It has a drawback,
though: it's an incomplete
Stefan Behnel added the comment:
Thanks Yury, I'll give it a try ASAP.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24017
___
___
Python-bugs
Stefan Behnel added the comment:
It *is* correct, see PEP 492. Awaitable is either a coroutine *or* an object
with an __await__ method.
coroutine, yes. But Coroutine? Shouldn't the Coroutine ABC then require
__await__ to be implemented? Maybe even by inheriting from Awaitable?
Just
Changes by Stefan Behnel sco...@users.sourceforge.net:
--
nosy: +scoder
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23275
___
___
Python-bugs
Stefan Behnel added the comment:
I added a couple of review comments to patch 6, but since no-one has responded
so far, I guess they simply haven't been noticed. So I'll just repeat them here.
1)
getawaitablefunc / aiternextfunc / getaiterfunc
Is there a reason why these need to have
Stefan Behnel added the comment:
Another question: is it ok if Cython implements and uses the tp_as_async slot
in all Py3.x versions (3.2+)? It shouldn't hurt, but it would speed up
async/await in Cython at least under Py3.x. Only Py2.6/7 would then have to
resort to calling __await__() etc
Stefan Behnel added the comment:
FTR, lxml's Element class has addnext() and addprevious() methods which are
commonly used for this purpose. But ET can't adopt those due to its different
tree model.
I second Martin's comment that ET.append() is a misleading name. It suggests
adding stuff
Stefan Behnel added the comment:
I noticed that my patch isn't entirely correct. If the exception value is a
tuple, both PyErr_SetObject() and PyErr_NormalizeException() use it directly as
*argument tuple* for the exception instantiation call, i.e. they essentially
unpack it into separate
Changes by Stefan Behnel sco...@users.sourceforge.net:
--
nosy: +scoder
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17239
___
___
Python-bugs
Stefan Behnel added the comment:
Eventually, I think a corresponding function should be added to cmath.
math.isclose() shouldn't deal with complex values by itself (other than
rejecting them as non-floatish input).
--
___
Python tracker rep
Stefan Behnel added the comment:
The cast is correct and required (the actual signature is determined by the
METH_* flags).
Patch LGTM, FWIW.
--
components: +Library (Lib)
nosy: +scoder
type: - enhancement
___
Python tracker rep...@bugs.python.org
Stefan Behnel added the comment:
There are calls to PyObject_RichCompareBool() in the loops, which means that
user code might get executed. What if that's evil code that modifies the heap
list? Couldn't that lead to list resizing and thus invalidation of the list
item pointer
Stefan Behnel added the comment:
Ah, sorry, yes. I somehow misread the arguments being passed *into* richcompare
as subsequent array access. LGTM too. Sorry for the noise.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org
Stefan Behnel added the comment:
Well, as I've shown in issue 24076 (I'm copying the numbers here), even simple
arithmetic expressions can benefit from a free-list. Basically anything that
uses temporary integer results.
Original:
$ ./python -m timeit 'sum(range(1, 10))'
1000 loops, best
Stefan Behnel added the comment:
Issue 24165 was created to pursue the path of a free-list for PyLong objects.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24076
Stefan Behnel added the comment:
I got similar results on 64bits for my original patch (very similar to what
Serhiy used now). The numbers are not really conclusive.
Report on Linux leppy 3.13.0-46-generic #77-Ubuntu SMP Mon Mar 2 18:23:39 UTC
2015 x86_64 x86_64
Total CPU cores: 4
### 2to3
Stefan Behnel added the comment:
We also need a Coroutine ABC. Both the GeneratorType and CO_COROUTINE
checks are too restrictive. Also see issue 24018, which this one should in fact
depend on.
--
___
Python tracker rep...@bugs.python.org
http
Stefan Behnel added the comment:
This is blocking issue 24017 (async/await syntax).
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24018
Stefan Behnel added the comment:
Thanks! Minor grouch: it should say collections.*abc*.Generator in the NEWS
entry.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24018
Stefan Behnel added the comment:
I get a test failure in Cython's compatibility tests which seems to be
attributable to this change:
def sideeffect(x):
... L.append(x)
... return x
def unhashable(x):
... L.append(x)
... return [x]
L
Stefan Behnel added the comment:
See issue 24076.
--
nosy: +scoder
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24138
___
___
Python-bugs-list
Stefan Behnel added the comment:
I'm actually not sure how it relates to the minimum. The more runs you have,
the higher the chance of hitting the actual minimum at least once. And if none
of the runs hits the real minimum, you're simply out of luck.
However, it should tend to give a much
Stefan Behnel added the comment:
Well, we can apply a kludge, or apply statistics.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22881
Stefan Behnel added the comment:
for the even number case, I think you shouldn't do // 2, but / 2.
Right. I updated the patch.
--
Added file: http://bugs.python.org/file39279/show_median.patch
___
Python tracker rep...@bugs.python.org
http
Stefan Behnel added the comment:
I tried implementing a freelist. Patch attached, mostly adapted from the one in
dictobject.c, but certainly needs a bit of cleanup.
The results are not bad, about 10-20% faster:
Original:
$ ./python -m timeit 'sum(range(1, 10))'
1000 loops, best of 3
Stefan Behnel added the comment:
I don't think it's irrelevant. Throw-away integers are really not uncommon.
For-loops use them quite often, non-trivial arithmetic expressions can create a
lot of intermediate temporaries. Speeding up the create-delete cycle of PyLong
sounds like a very
Stefan Behnel added the comment:
I don't think it's necessary to have slots for __aiter__, __anext__,
__aenter__ and __aexit__. Async iteration will never be as fast as regular
iteration, and there is plenty overhead in it.
You seem to be assuming that the outer loop is the asyncio I/O
Stefan Behnel added the comment:
I agree that the wording in the documentation isn't great:
text
The text attribute can be used to hold additional data associated with the
element. As the name implies this attribute is usually a string but may be any
application-specific object
Stefan Behnel added the comment:
there are three ingredients here - sum, (x)range and the integer addition
that sum will be performing at each iteration.
... not to forget the interpreter startup time on his machine. :)
I did a tiny bit of profiling and about 90% of the time seems
Stefan Behnel added the comment:
Please either
1) drop the throw() method entirely or
2) make throw an abstractmethod()
Ok, as I already said, I think it's important to provide the complete protocol
as code will usually expect that. Also, close() has a helpful implementation
Stefan Behnel added the comment:
why not spell them sys.exit.FAILURE and sys.exit.SUCCESS ?
--
nosy: +scoder
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24053
Stefan Behnel added the comment:
Actually, my guess is also that these constants will not be used. Not because
they are not helpful, but because they'd only be available in Python 3.5+.
Meaning that if you use them, your code won't work with long time supported
CPython versions like 3.4
Stefan Behnel added the comment:
Any more comments on this? The deadlines for new features in Py3.5 are getting
closer. It seems we're just discussing details here, but pretty much everyone
wants this feature.
So, what are the things that still need to be done? Serhiy submitted working
Stefan Behnel added the comment:
PEP 489 (Redesigning extension module loading) includes the proposal to fix
this by using punycode.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20485
601 - 700 of 1270 matches
Mail list logo