[issue45123] PyAiter_Check & PyObject_GetAiter issues

2021-09-06 Thread Yury Selivanov


Change by Yury Selivanov :


--
keywords: +patch
pull_requests: +26618
pull_request: https://github.com/python/cpython/pull/28194

___
Python tracker 
<https://bugs.python.org/issue45123>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45123] PyAiter_Check & PyObject_GetAiter issues

2021-09-06 Thread Yury Selivanov


New submission from Yury Selivanov :

Per discussion on python-dev (also see the linked email), PyAiter_Check should 
only check for `__anext__` existence (and not for `__aiter__`) to be consistent 
with `Py_IterCheck`.

While there, I'd like to rename PyAiter_Check to PyAIter_Check and 
PyObject_GetAiter to PyObject_GetAIter (i -> I).  First, we should apply 
CamelCase convention correctly, here "async" and "iter" are separate words; 
second, "Aiter" is marked as invalid spelling by spell checkers in IDEs which 
is annoying.

See 
https://mail.python.org/archives/list/python-...@python.org/message/BRHMOFPEKGQCCKEKEEKGSYDR6NOPMRCC/
 for more details.

--
components: Interpreter Core
messages: 401206
nosy: pablogsal, yselivanov
priority: release blocker
severity: normal
stage: patch review
status: open
title: PyAiter_Check & PyObject_GetAiter issues
type: behavior
versions: Python 3.10, Python 3.11

___
Python tracker 
<https://bugs.python.org/issue45123>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45088] Coroutines & async generators disagree on the iteration protocol semantics

2021-09-02 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy: +gvanrossum

___
Python tracker 
<https://bugs.python.org/issue45088>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45088] Coroutines & async generators disagree on the iteration protocol semantics

2021-09-02 Thread Yury Selivanov


New submission from Yury Selivanov :

See this script:

  https://gist.github.com/1st1/eccc32991dc2798f3fa0b4050ae2461d

Somehow an identity async function alters the behavior of manual iteration 
though the wrapped nested generator.

This is a very subtle bug and I'm not even sure if this is a bug or not. 
Opening the issue so that I don't forget about this and debug sometime later.

--
components: Interpreter Core
messages: 400951
nosy: lukasz.langa, pablogsal, yselivanov
priority: normal
severity: normal
stage: needs patch
status: open
title: Coroutines & async generators disagree on the iteration protocol 
semantics
type: behavior
versions: Python 3.10, Python 3.11

___
Python tracker 
<https://bugs.python.org/issue45088>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22239] asyncio: nested event loop

2021-07-26 Thread Yury Selivanov


Yury Selivanov  added the comment:

> nest-asyncio library (https://github.com/erdewit/nest_asyncio)

Seeing that the community actively wants to have support for nested loops I'm 
slowly changing my opinion on this.

Guido, maybe we should allow nested asyncio loops disabled by default?

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44604] [Enhancement] Asyncio task decorator to provide interface to define async DAGs (similar to dask's delayed interface)

2021-07-13 Thread Yury Selivanov


Yury Selivanov  added the comment:

Yeah, I'm also not convinced this has to be part of asyncio, especially since 
we'll likely have task groups in 3.11.

--

___
Python tracker 
<https://bugs.python.org/issue44604>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38323] asyncio: MultiLoopWatcher has a race condition (test_asyncio: test_close_kill_running() hangs on AMD64 RHEL7 Refleaks 3.x)

2021-06-08 Thread Yury Selivanov


Yury Selivanov  added the comment:

>  MultiLoopChildWatcher must ensures that the event loop is awaken when it 
> receives a signal by using signal.setwakeup(). This is done by 
> _UnixSelectorEventLoop.add_signal_handler(). Maybe MultiLoopChildWatcher 
> could reuse this function, rather than calling directly signal.signal().

I think this is a good idea. MultiLoopChildWatcher could use setwakeupfd with 
some no-op callback just to wakeup the loop.

--

___
Python tracker 
<https://bugs.python.org/issue38323>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37146] opcode cache for LOAD_GLOBAL emits false alarm in memory leak hunting

2021-02-19 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Giving the ability to control the cache size, at least at Python startup, is 
> one option.

I'd really prefer not to allow users to control cache sizes. There's basically 
no point in that; the only practically useful thing is to enable or disable it.

--

___
Python tracker 
<https://bugs.python.org/issue37146>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

2021-02-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> So you think that even a dedicated "LEN" opcode would not be any faster? 
> (This is getting in Paul Sokolovsky territory -- IIRC he has a variant of 
> Python that doesn't allow overriding builtins.)

Yeah, a dedicated LEN opcode could only be faster if it would not be possible 
to shadow builtins (or if there was a "len" operator in Python).  If that's not 
the case, this hypothetical LEN opcode would still have to check if "len" was 
shadowed or not, and that's slower than the optimized LOAD_GLOBAL we have now.

--

___
Python tracker 
<https://bugs.python.org/issue42990>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

2021-02-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I tried to implement such optimization in my old 
> https://faster-cpython.readthedocs.io/fat_python.html project. I implemented 
> guards to de-optimize the code if a builtin is overriden.

FWIW the globals opcode cache handles all of this now. There's no point in 
specifically optimizing the builtins lookup since we optimize all global 
lookups for a code object that's hot enough.

--

___
Python tracker 
<https://bugs.python.org/issue42990>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42927] Inline cache for slots

2021-01-14 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Some microbenchmarks:


Can you add a new one `read_slots`?

--

___
Python tracker 
<https://bugs.python.org/issue42927>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42927] Inline cache for slots

2021-01-14 Thread Yury Selivanov


Yury Selivanov  added the comment:

> So it seems that everything is in the noise range except the "float" 
> benchmark that is 1.11x faster

Yeah, this is why. 
https://github.com/python/pyperformance/blob/master/pyperformance/benchmarks/bm_float.py#L12

This is a great result, IMO. I'm +1 to merge this.

--

___
Python tracker 
<https://bugs.python.org/issue42927>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2021-01-05 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Do we have good intuition or data about which operations need speeding up 
> most? Everybody always assumes it's BINARY_ADD, but much Python code isn't 
> actually numeric, and binary operations aren't all that common.

IMO, we shouldn't focus too much on optimizing binops. Regular webapps/network 
kind of code wouldn't benefit from that, and scientific code that uses numpy/ml 
libraries already offsets most of expensive computation to outside of CPython 
eval. Instead, I think, we should continue focusing on lowering the cost of 
function/method calls and attribute access.

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2021-01-05 Thread Yury Selivanov


Yury Selivanov  added the comment:

> The gist seems to be to have extra opcodes that only work for certain 
> situations (e.g. INT_BINARY_ADD). In a hot function we can rewrite opcodes 
> with their specialized counterpart. The new opcode contains a guard that 
> rewrites itself back if the guard fails (and then it stays unoptimized).

This is also roughly what I suggested in https://bugs.python.org/msg379333. 
Except that I don't think it's necessary to add new opcodes.

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41891] asyncio.wait_for does not wait for task/future to be completed in all cases

2020-12-18 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 82dbfd5a04863d8b6363527e6a34a90c9aa5691b by Miss Islington (bot) 
in branch '3.9':
bpo-41891: ensure asyncio.wait_for waits for task completion (GH-22461) (#23840)
https://github.com/python/cpython/commit/82dbfd5a04863d8b6363527e6a34a90c9aa5691b


--

___
Python tracker 
<https://bugs.python.org/issue41891>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42600] Cancelling tasks waiting for asyncio.Conditions crashes w/ RuntimeError: Lock is not acquired.

2020-12-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

This has been actually first fixed in #41891 but somehow wasn't yet merged. 
Yurii, thanks so much for working on this and making a PR, there was just 
another PR to fix the same issue that was there first, so I had to merge that 
one.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
superseder:  -> asyncio.wait_for does not wait for task/future to be completed 
in all cases

___
Python tracker 
<https://bugs.python.org/issue42600>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41891] asyncio.wait_for does not wait for task/future to be completed in all cases

2020-12-18 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy:  -miss-islington
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41891>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41891] asyncio.wait_for does not wait for task/future to be completed in all cases

2020-12-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

Thanks for the PR, Richard!

--
nosy:  -miss-islington

___
Python tracker 
<https://bugs.python.org/issue41891>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41891] asyncio.wait_for does not wait for task/future to be completed in all cases

2020-12-18 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 17ef4319a34f5a2f95e7823dfb5f5b8cff11882d by Richard Kojedzinszky 
in branch 'master':
bpo-41891: ensure asyncio.wait_for waits for task completion (#22461)
https://github.com/python/cpython/commit/17ef4319a34f5a2f95e7823dfb5f5b8cff11882d


--

___
Python tracker 
<https://bugs.python.org/issue41891>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42600] Cancelling tasks waiting for asyncio.Conditions crashes w/ RuntimeError: Lock is not acquired.

2020-12-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

Hey Hynek! :) Can you submit a PR?

--

___
Python tracker 
<https://bugs.python.org/issue42600>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-12-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I will try to do some prototyping around that to see how much can we gain in 
> that route. In any case, adding LOAD_METHOD support for all kinds of calls 
> should be an improvement by itself even without caching, no?

Exactly.

As one argument for generalizing of LOAD_METHOD is that I, for example, try not 
to use kwargs in hot paths because I know it will be slower, which feels very 
wrong. I shouldn't care of internal implementation details of CPython and focus 
on writing readable code.

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-12-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I think I am closing the PR as it seems that the gains are not good enough 
> (and there is quite a lot of noise by comparing the benchmarks together).

IMO you need to implement LOAD_METHOD support for all kinds of calls, including 
the ones that use kwargs, to see any improvement.

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-24 Thread Yury Selivanov


Yury Selivanov  added the comment:

Sorry, there are a few things in the committed patch that should be fixed. See 
the PR for my comments.

--
resolution: fixed -> 
stage: resolved -> patch review
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-19 Thread Yury Selivanov


Yury Selivanov  added the comment:

> The safe code can look like:
>
> global_lock = threading.Lock()  like GIL

SGTM. We can use this strategy for all synchronization primitives and for 
objects like asyncio.Queue.

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Despite the fact that asyncio.get_running_loop() never returns None but 
> raises RuntimeError (and maybe other tiny cleanups), 

It never raises when called from inside a coroutine. And I propose to do that. 
So it will never fail.

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42085] Add dedicated slot for sending values

2020-11-18 Thread Yury Selivanov


Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42085>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42085] Add dedicated slot for sending values

2020-11-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Is anything left to do?

I think we can close this now.

--

___
Python tracker 
<https://bugs.python.org/issue42085>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

Yeah, I follow the reasoning.

My use case:

   class Something:
  def __init__(self):
  self._lock = asyncio.Lock()

  async def do_something():
  async with self._lock:
  ...

And `Something` won't be created in a coroutine. So now I have to jump through 
the hoops to implement lazy lock instantiation.


> But the lock is tightly coupled with a loop instance. In other words, the 
> loop belongs to the loop.
>The lock cannot be used after the loop dies (stopped and closed).

I agree. Maybe the solution should be this then:


class asyncio.Lock:

   def _get_loop(self):
   loop = asyncio.get_running_loop()
   if self._loop is None:
self._loop = loop
   if loop is not self._loop: raise
   if not loop.is_running(): raise

   async def acquire(self):
   loop = self._get_loop()
   ...

This is what would guarantee all protections you want and would also allow to 
instantiate `asyncio.Lock` in class constructors, simplifying code.

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> That's good to know and I think more convenient to work with, so +1 from me. 
> I guess my remaining question though is whether it's okay to `await 
> lock.acquire()` on a single lock instance from multiple different running 
> event loops (presumably each in their own separate threads) or if there's a 
> need to restrict it to only one event loop.

No, it's not OK to use one lock across multiple loops at the same time. But 
most asyncio code out there doesn't have protections against that, and we never 
advertised that having multiple loops run in parallel is a good idea.  So while 
we could build protections against that, I'm not sure its needed. 

Andrew, thoughts?


> From what I've seen of asyncio user code, it seems reasonably common to 
> create async primitives (lock, semaphore, queue, etc.) in the __init__ for 
> some class prior to using the event loop, which would fail with usage of 
> `get_running_loop()` in the __init__ for the primitives. So, if it's not an 
> issue to wait until accessing the event loop until it's actually needed (e.g. 
> in the lock.acquire() or queue.get/put()), I think we should definitely try 
> to be conscious about when we call `get_running_loop()` going forward to 
> ensure we're not imposing arbitrary inconveniences on users.

Yep. This sums up how I think of this now.

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

Oh my.  FWIW I think that we need to implement this differently. I don't think 
it matters where, say, an asyncio.Lock was instantiated. It can be created 
anywhere. So IMO its __init__ shouldn't try to capture the current loop -- 
there's no need for that. The loop can be and should be captured when the first 
`await lock.acquire()` is called.

I'm writing a piece of code right now that would need to jump through the hoops 
to simply create a new `asyncio.Lock()` in a place where there's no asyncio 
loop yet.

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

Kyle, lmk if you want to work on this.

--
nosy: +aeros

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

(or alternatively they can cache the running `loop`, but the first loop lookup 
should be performed with `asyncio.get_running_loop()`)

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-17 Thread Yury Selivanov


New submission from Yury Selivanov :

asyncio.Lock and other primitives should no longer accept the `loop` parameter. 
They should also stop storing the current loop in the `self._loop` attribute. 
Instead, they should use `get_running_loop()` whenever they need to access the 
loop.

--
components: asyncio
messages: 381275
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: remove the 'loop' parameter from __init__ in all classes in asyncio.locks
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42140] asyncio.wait function creates futures set two times

2020-11-10 Thread Yury Selivanov


Yury Selivanov  added the comment:

Thanks for the change and for the discussion to everybody involved. The PR 
makes sense and I've already merged it.

--

___
Python tracker 
<https://bugs.python.org/issue42140>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42085] Add dedicated slot for sending values

2020-11-10 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 1e996c3a3b51e9c6f1f4cea8a6dbcf3bcb865060 by Vladimir Matveev in 
branch 'master':
bpo-42085: Introduce dedicated entry in PyAsyncMethods for sending values 
(#22780)
https://github.com/python/cpython/commit/1e996c3a3b51e9c6f1f4cea8a6dbcf3bcb865060


--

___
Python tracker 
<https://bugs.python.org/issue42085>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42085] Add dedicated slot for sending values

2020-11-10 Thread Yury Selivanov


Yury Selivanov  added the comment:

Vladimir, please do a follow up PR documenting Py_TPFLAGS_HAVE_AM_SEND.

--

___
Python tracker 
<https://bugs.python.org/issue42085>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24165] Free list for single-digits ints

2020-10-22 Thread Yury Selivanov


Yury Selivanov  added the comment:

Inada-san, how do you interpret the results? Looks like it's 
performance-neutral.

--

___
Python tracker 
<https://bugs.python.org/issue24165>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24165] Free list for single-digits ints

2020-10-22 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy: +pablogsal

___
Python tracker 
<https://bugs.python.org/issue24165>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42117] asyncio calls from sync/async, better docs or api support

2020-10-22 Thread Yury Selivanov


Yury Selivanov  added the comment:

I'm not sure what you're actually proposing, and only have a vague 
understanding of what you're complaining about. This is perhaps related to 
https://bugs.python.org/issue33523? If so then maybe leave a comment there?

--

___
Python tracker 
<https://bugs.python.org/issue42117>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-10-22 Thread Yury Selivanov


Change by Yury Selivanov :


--
Removed message: https://bugs.python.org/msg379330

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-10-22 Thread Yury Selivanov


Yury Selivanov  added the comment:

> This idea can be implemented without opcode cache. I will try it.

I'd actually encourage trying to use the opcode cache because this way the 
optimization will be more generic. E.g. `decimal + decimal` would also be 
specialized via the cache, because you'd cache a pointer to the specific `+` 
operator implementation. I'm really not sure that adding specialized byte code 
is a good idea.

> * PHP uses scalar type for float and int

While at it, maybe we push the number of bits per int digit to 60?

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-10-22 Thread Yury Selivanov


Yury Selivanov  added the comment:

> This idea can be implemented without opcode cache. I will try it.

I'd actually encourage trying to use the opcode cache because this way the 
optimization will be more generic. E.g. `decimal + decimal` would also be 
specialized via the cache, because you'd cache a pointer to the specific `+` 
operator implementation. I'm really not sure that adding specialized byte code 
is a good idea.

> * PHP uses scalar type for float and int

While at it, maybe we push the number of bits per int digit to 60?

While at it, I'd also increase the digit size

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-10-21 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Imagine that we have a secondary copy of the bytecode in the cache inside the 
> code object and we mutate that instead. The key difference with the current 
> cache infrastructure is that we don't accumulate all the optimizations on the 
> same opcode, which can be very verbose. Instead, we change the generic opcode 
> to a more specialised to optimize and we change it back to deoptimize.

Yeah, I follow. As long as we keep the original list of opcodes we're good ;)

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-10-21 Thread Yury Selivanov


Yury Selivanov  added the comment:

Few thoughts in no particular order:

- I'd suggest implementing the cache for 2-3 more opcodes on top of the 
existing infrastructure to get more experience and then refactoring it to make 
it more generic.

- Generalizing LOAD_METHOD to work for methods with **kwargs, caching concrete 
operator implementations for opcodes like BINARY_ADD etc. are all possible on 
top of the current infra.

- Rewriting code objects in place is wrong, IMO: you always need to have a way 
to deoptimize the entire thing, so you need to keep the original one. It might 
be that you have well defined and static types for the first 1 invocations 
and something entirely different on 10001. So IMO we need a SpecializedCode 
object with the necessary bailout guards. But that's not a simple thing to 
implement, so unless someone will be working on this fulltime for a long time 
I'd suggest working off what we have now. (That said I'd take a close look at 
what Dino is building).

- There are multiple different approaches we can choose for optimizing CPython, 
ranging from hidden classes to a full blown JIT. I hope someone will do them 
one day. But IMO the current simple "opcode cache" (I wish we had a better 
name) mechanism we have would allow us to squeeze up to 15-25% median 
improvement in our benchmarks with relatively limited dev time. Maybe that's 
good enough for 3.10.

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42113] Replace _asyncio.TaskWakeupMethWrapper with PyCFunction

2020-10-21 Thread Yury Selivanov


Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42113>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-10-13 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset cfb0f57ff876ab3d04ff144f19eda58844981643 by Vladimir Matveev in 
branch 'master':
bpo-41756: Export PyGen_Send and wrap it in if-defs (#22677)
https://github.com/python/cpython/commit/cfb0f57ff876ab3d04ff144f19eda58844981643


--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-10-12 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Also, in Include/abstract.h it should only be available for limited C API >= 
> 3.10:

Vladimir, could you please submit a PR?

> We now should discuss the behavior of PySend_Iter() if value is NULL. It is 
> not documented, the current behavior differs from the behavior for non-NULL 
> value, and it is used in the ceval loop.

IMO: I'd keep the behavior and just document it.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-10-12 Thread Yury Selivanov


Change by Yury Selivanov :


--
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-10-12 Thread Yury Selivanov


Yury Selivanov  added the comment:

With the latest PR now merged this issue can be closed. Please reopen if there 
are any other action items left.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-10-09 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 037245c5ac46c3436f617a1f5d965929754be239 by Vladimir Matveev in 
branch 'master':
bpo-41756: Add PyIter_Send function (#22443)
https://github.com/python/cpython/commit/037245c5ac46c3436f617a1f5d965929754be239


--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-21 Thread Yury Selivanov


Yury Selivanov  added the comment:

> If you disagree with me, please say why, don't just merge the PR.

Apologies, Mark. I didn't intend to merge something bypassing your opinion; 
just missed your comment between reviewing multiple PRs in a few unrelated 
repos. I'm sorry.

On the actual naming subject, you proposed:

> `PySendResult PyIter_Send(PyObject *obj, PyObject *arg, PyObject **result);`

The problem with using this name is that ideally we should also support 
non-native coroutine and generator implementations (i.e. resolve the "send" 
attribute and call it using Python calling convention). Ideally we should have 
two C APIs: one low-level supporting only native objects and a high level one, 
supporting all kinds of them.

Can we perhaps add both `PyGen_Send()` and `PyCoro_Send()` for now that would 
only accept generators and coroutines respectively? After that we can discuss 
adding a more generic `PyIter_Send`?


> Would you revert the PR, please.

Since this is in 3.10/master that nobody uses right now except us (Python core 
devs), can we just issue a follow up PR to fix whatever is there to fix? I'd 
like to avoid the churn of reverting, and again, I apologize for pushing this a 
bit hastily.  Let me know if you actually want a revert and I'll do that.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

Vladimir, could you please submit a PR to update 3.10/whatsnew? Need to mention 
both the new C API and the new perf boost in relevant sections.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

Thanks, Vladimir! Also huge thanks to Mark, Serhiy, and Stefan. If there are 
any nits to fix let's do that in follow up PRs.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 2b05361bf7cbbd76035206fd9befe87f37489f1e by Vladimir Matveev in 
branch 'master':
bpo-41756: Introduce PyGen_Send C API (GH-22196)
https://github.com/python/cpython/commit/2b05361bf7cbbd76035206fd9befe87f37489f1e


--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Sounds like a good middleground to start: add ``PySendResult `` and 
> `PySendResult PyGen_Send(PyGenObject*, PyObject* PyObject**)` specific to 
> generators and coroutines. 

Yes, it seems that everybody agreed on that. I can give the PR another review 
-- is it ready?

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I would also have preferred a more type specific function, but yeah, as long 
> as the types for which the function would normally be used are special cased 
> early enough in the implementation, it makes no big difference.

Maybe add two API funcs: PyGen_Send (specific to generators & coroutines) and 
PyIter_Send?

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Does it sound correct?

It does, but given the amount of back and forth on this, I'd wait for Serhiy 
and Stefan to confirm if they're OK.

IMO the `PyIter_Send` name is OK (given how generic the implementation will be) 
and returning an enum, while a tad unconventional for the C API is way more 
convenient for the developer.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I guess `PyIter_Send` would imply that this function should work for all 
> inputs (like in https://bugs.python.org/msg377007) which also sounds 
> reasonable.

I like `PyIter_Send` (and why I initially proposed it myself) because it will 
also work with the "send" slot defined on some types if we end up adding one.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I think it should be specific to generators and coroutines. Calling 
> `PyObject_CallMethodIdOneArg(coro, _send, arg);` and interpreting 
> exceptions to emulate the low level API seems a bit too much.

To add to my point: typically higher-level APIs go under the `PyObject_*` 
namespace, whereas `Py{Type}_*` is more concrete. So I'd make `PyGen_Send` to 
only work with `PyGen` and `PyCoro`.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Since coroutines inherit the generator protocol more or less, I think 
> "PyGen_Send()" is a more suitable name, better than "PyCoro_Send()".

OK, +1. PyGen_Send it is.

> Also should it be specific to generators/coroutines and accept PyGenObject*

I think it should be specific to generators and coroutines. Calling 
`PyObject_CallMethodIdOneArg(coro, _send, arg);` and interpreting 
exceptions to emulate the low level API seems a bit too much.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> As for returned value, I propose to return -1 in case of error, 1 for yielded 
> value and 0 for returned value (i.e. define PYGEN_RETURN = 0, PYGEN_YIELD = 1 
> and PYGEN_ERROR = -1, but without exposing public names).

Sure, that works.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

Mark, Stefan,

I don't want this to be stale so I propose to move with my suggestions:

1. We make the new API public. Mark, if you have objections to that - please 
elaborate with some details. IMO, the corresponding Python API is long public 
and there's no harm in exposing a C version of it. Especially given the fact 
that uvloop, cython, and even asyncio itself will be relying on that API.

2. I propose to name the new API `PyIter_Send`. Motivation: it will work with 
both generators and coroutines and plays nicely with `PyIter_Next`.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-11 Thread Yury Selivanov


Yury Selivanov  added the comment:

@Mark
> Mark Shannon wrote: I don't think [the C-API function] should be public, as a 
> possible further improvement is to stop passing exceptions through a side 
> channel, but in result. Maybe we don't want to do that, but lets' not add to 
> the (already rather large) C-API.

Yeah, we can add it as a "private" function, I'm not entirely opposed to that. 
But... it would be great if Cython and C code could still depend on it and use 
it. And then... why should it be private? The corresponding Python API 
"gen.send()" and "gen.throw()" is public, why can't the C API be public too?

We will not fundamentally change generators (it would be a major backwards 
incompatible change), so committing to a good C API sounds reasonable.

@Mark
> Remember that PyIter_Next() is pretty much the same, though, and it has the 
> standard "return PyObject*" interface. These two would diverge then.

Maybe we should call it `_PyIter_Send()`?  While `.send()` is mostly about 
coroutines, regular generators have the method too, and it would be weird to 
call `_PyCoro_Send` on a generator object.

@Vladimir
> PYGEN_ERROR  | NULL | Regular PyErr_* functions should be used to work 
> with error case

Correct.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-10 Thread Yury Selivanov


Yury Selivanov  added the comment:

Big +1 from me. This is something I always wanted to do myself (since the time 
of PEP 492 & 525 implementations) and I think this is a necessary change. It's 
great that this isn't just a C API UX improvement but also yields a big perf 
improvement.

--
nosy: +Mark.Shannon, lukasz.langa, vstinner

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41733] ContextVar get value is unexpected

2020-09-08 Thread Yury Selivanov


Yury Selivanov  added the comment:

> my expected result is all objects should be different, because the code runs 
> in different context.

Contexts "branch", so if you store something in the context before asyncio.run 
it will store it in the current thread state. And then asyncio.run will pick it 
up.

Also I'd recommend using something else than object IDs - those can be reused 
and can be very confusing when browsing logs -- it might look like it was the 
same object, but in fact it could be to different objects sharing the same ID 
at different points of time.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue41733>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41687] sendfile implementation is not compatible with Solaris

2020-09-05 Thread Yury Selivanov

Yury Selivanov  added the comment:


New changeset 8c0be6fd9101746235b63ddfb84106d1e9ca286b by Jakub Kulík in branch 
'master':
bpo-41687: Fix sendfile implementation to work with Solaris (#22040)
https://github.com/python/cpython/commit/8c0be6fd9101746235b63ddfb84106d1e9ca286b


--
nosy: +yselivanov

___
Python tracker 
<https://bugs.python.org/issue41687>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41687] sendfile implementation is not compatible with Solaris

2020-09-05 Thread Yury Selivanov


Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue41687>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 40e2444c364eede59f193979df5a02ed958f56e9 by Miss Islington (bot) 
in branch '3.8':
bpo-39010: Improve test shutdown (GH-22066) (#22083)
https://github.com/python/cpython/commit/40e2444c364eede59f193979df5a02ed958f56e9


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset a2680058ee1f43cab282e9f4c2b35fd89464ebb6 by Miss Islington (bot) 
in branch '3.9':
bpo-41696: Fix handling of debug mode in asyncio.run (GH-22069) (#22071)
https://github.com/python/cpython/commit/a2680058ee1f43cab282e9f4c2b35fd89464ebb6


--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 1f5f12737755f246fee83a54c600779ad76934a4 by Miss Islington (bot) 
in branch '3.8':
bpo-41696: Fix handling of debug mode in asyncio.run (GH-22069) (#22072)
https://github.com/python/cpython/commit/1f5f12737755f246fee83a54c600779ad76934a4


--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 8d68e59f11267ded765d4af6d71a49784a12fad5 by Miss Islington (bot) 
in branch '3.9':
bpo-39010: Improve test shutdown (GH-22066) (#22082)
https://github.com/python/cpython/commit/8d68e59f11267ded765d4af6d71a49784a12fad5


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset a986b061a3cd4661eb9f857e2936291f1847bd15 by Miss Islington (bot) 
in branch '3.8':
bpo-39010: Fix errors logged on proactor loop restart (GH-22017) (#22035)
https://github.com/python/cpython/commit/a986b061a3cd4661eb9f857e2936291f1847bd15


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 49571c0b0e57e20d85727c738d9a0fe342dd2938 by Miss Islington (bot) 
in branch '3.9':
bpo-39010: Fix errors logged on proactor loop restart (GH-22017) (#22034)
https://github.com/python/cpython/commit/49571c0b0e57e20d85727c738d9a0fe342dd2938


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-02 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset be435ae2b064dc64f04475bec632862e1dbf605f by Ben Darnell in branch 
'master':
bpo-39010: Improve test shutdown (#22066)
https://github.com/python/cpython/commit/be435ae2b064dc64f04475bec632862e1dbf605f


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-02 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 0770ad948cb6d9f7f6c4002efd83e27c27069808 by Shantanu in branch 
'master':
bpo-41696: Fix handling of debug mode in asyncio.run (#22069)
https://github.com/python/cpython/commit/0770ad948cb6d9f7f6c4002efd83e27c27069808


--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-02 Thread Yury Selivanov


Yury Selivanov  added the comment:

Yes, this is a bug. The default for the `debug` parameter should be `None` and 
it should only call `loop.set_debug` if a non-None bool value was explicitly 
passed. Care to submit a PR?

--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-08-31 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset ea5a6363c3f8cc90b7c0cc573922b10f296073b6 by Ben Darnell in branch 
'master':
bpo-39010: Fix errors logged on proactor loop restart (#22017)
https://github.com/python/cpython/commit/ea5a6363c3f8cc90b7c0cc573922b10f296073b6


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32751] wait_for(future, ...) should wait for the future (even if a timeout occurs)

2020-08-26 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 57b698886b47bb81c782c2ba80a8a72fe66c7aad by Elvis Pranskevichus 
in branch '3.8':
[3.8] bpo-32751: Wait for task cancel in asyncio.wait_for() when timeout <= 0 
(GH-21895) (#21967)
https://github.com/python/cpython/commit/57b698886b47bb81c782c2ba80a8a72fe66c7aad


--

___
Python tracker 
<https://bugs.python.org/issue32751>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37658] In some cases asyncio.wait_for can lead to socket leak.

2020-08-26 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 6e1954cd8286e083e7f8d09516d91b6b15769a4e by Miss Islington (bot) 
in branch '3.8':
bpo-37658: Fix asyncio.wait_for() to respect waited task status (GH-21894) 
(#21965)
https://github.com/python/cpython/commit/6e1954cd8286e083e7f8d09516d91b6b15769a4e


--

___
Python tracker 
<https://bugs.python.org/issue37658>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37658] In some cases asyncio.wait_for can lead to socket leak.

2020-08-26 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset a2118a14627256197bddcf4fcecad4c264c1e39d by Elvis Pranskevichus 
in branch 'master':
bpo-37658: Fix asyncio.wait_for() to respect waited task status (#21894)
https://github.com/python/cpython/commit/a2118a14627256197bddcf4fcecad4c264c1e39d


--

___
Python tracker 
<https://bugs.python.org/issue37658>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32751] wait_for(future, ...) should wait for the future (even if a timeout occurs)

2020-08-26 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset c517fc712105c8e5930cb42baaebdbe37fc3e15f by Elvis Pranskevichus 
in branch 'master':
bpo-32751: Wait for task cancel in asyncio.wait_for() when timeout <= 0 (#21895)
https://github.com/python/cpython/commit/c517fc712105c8e5930cb42baaebdbe37fc3e15f


--

___
Python tracker 
<https://bugs.python.org/issue32751>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41543] contextlib.nullcontext doesn't work with async context managers

2020-08-13 Thread Yury Selivanov


Yury Selivanov  added the comment:

I typically don't like objects that support both `with` and `async with`, but 
in this case I think that's appropriate. Nathaniel, Andrew, what do you think?

--
nosy: +njs

___
Python tracker 
<https://bugs.python.org/issue41543>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41197] Async magic methods in contextlib.closing

2020-08-10 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy: +asvetlov, njs

___
Python tracker 
<https://bugs.python.org/issue41197>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41197] Async magic methods in contextlib.closing

2020-08-10 Thread Yury Selivanov


Yury Selivanov  added the comment:

I'm OK with adding this, but the close method should be `aclose()`

--

___
Python tracker 
<https://bugs.python.org/issue41197>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41438] TimeoutError behavior changes on async.wait_for from python3.7

2020-07-29 Thread Yury Selivanov


Yury Selivanov  added the comment:

> In python 3.8.4 running this script will print:

> asyncio.TimeoutError happened

This is the expected behavior in 3.8.

> This is a breaking change which I didn't see announced in the changelog.

Yeah, looks like this wasn't mentioned in what's new or elsewhere in the docs. 
:(

Do you want to submit a PR to fix that?

--

___
Python tracker 
<https://bugs.python.org/issue41438>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41317] sock_accept() does not remove server socket reader on cancellation

2020-07-23 Thread Yury Selivanov

Yury Selivanov  added the comment:


New changeset 0dd98c2d00a75efbec19c2ed942923981bc06683 by Alex Grönholm in 
branch 'master':
bpo-41317: Remove reader on cancellation in asyncio.loop.sock_accept() (#21595)
https://github.com/python/cpython/commit/0dd98c2d00a75efbec19c2ed942923981bc06683


--

___
Python tracker 
<https://bugs.python.org/issue41317>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41317] sock_accept() does not remove server socket reader on cancellation

2020-07-22 Thread Yury Selivanov


Yury Selivanov  added the comment:

Alex, do you want to submit a PR?

--

___
Python tracker 
<https://bugs.python.org/issue41317>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41305] Add StreamReader.readinto()

2020-07-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> By the way if we will eventually combine StreamReader and StreamWriter won't 
> this function (readinto) be useful then?

Yes.  But StreamReader and StreamWriter will stay for the foreseeable future 
for backwards compatibility pretty much frozen in time.

So I'd like to start treating them as legacy APIs now.

--

___
Python tracker 
<https://bugs.python.org/issue41305>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41305] Add StreamReader.readinto()

2020-07-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Is it simply combining stream reader and stream writer into a single object 
> and changing the write() function to always wait the write (thus deprecating 
> drain) and that's it?

Pretty much. We might also rename a few APIs here and there, like "close()" 
should become an "async def aclose()"

We also will likely want to define a few ABCs.

Which brings me to the most important point: what we need it not coding it 
(yet), but rather drafting the actual proposal and posting it to 
https://discuss.python.org/c/async-sig/20.  Once a formal proposal is there we 
can proceed with the implementation.

It's a bit of a process to follow, but we have to do it this way.

--
nosy: +aeros, njs

___
Python tracker 
<https://bugs.python.org/issue41305>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41305] Add StreamReader.readinto()

2020-07-15 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Im interested in learning about the new api.

There are two problems with the current API:

1. Reader and writer are separate objects, while they should be one.
2. Writer.write is not a coroutine, although it should be one.

There are other minor nits, but that's the crux of the problem. So we need a 
new streams design + a new set of APIs to work with it (and streams are in many 
places, like in the subprocess APIs).

Trio was going to stabilize their own streaming API and we thought it would be 
great if our new API was compatible with it (not sure if they did stabilize it 
or not).

I was going to lead the project myself (and still am) but dropped the ball and 
we missed 3.9 to do this. If you want to start working on this I'd be glad to 
assist with discussions & reviews.

--

___
Python tracker 
<https://bugs.python.org/issue41305>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41305] Add StreamReader.readinto()

2020-07-15 Thread Yury Selivanov


Yury Selivanov  added the comment:

We don't want to extend StreamReader with new APIs as ultimately we plan to 
deprecate it. A new streams API is needed, perhaps inspired by Trio. Sadly, I'm 
-1 on this one.

--

___
Python tracker 
<https://bugs.python.org/issue41305>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41273] asyncio: proactor read transport: use recv_into instead of recv

2020-07-14 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 568fb0ff4aa641329261cdde20795b0aa9278175 by Tony Solomonik in 
branch 'master':
bpo-41273: asyncio's proactor read transport's better performance by using 
recv_into instead of recv (#21442)
https://github.com/python/cpython/commit/568fb0ff4aa641329261cdde20795b0aa9278175


--
nosy: +yselivanov

___
Python tracker 
<https://bugs.python.org/issue41273>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37179] asyncio loop.start_tls() provide support for TLS in TLS

2020-07-09 Thread Yury Selivanov


Yury Selivanov  added the comment:

> The aiohttp issue says they won't fix this until asyncio supports it. Kinda 
> understand that.

I saw you opened an issue with aiohttp to allow this and they're open to it. I 
hope that will get some movement. It also would be a big test for uvloop's (and 
3.10 asyncio) TLS implementation.

--

___
Python tracker 
<https://bugs.python.org/issue37179>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41247] asyncio.set_running_loop() cache running loop holder

2020-07-09 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 0b6169e391ce6468aad711f08ffb829362293ad5 by Tony Solomonik in 
branch '3.8':
bpo-41247: asyncio.set_running_loop() cache running loop holder (#21406)
https://github.com/python/cpython/commit/0b6169e391ce6468aad711f08ffb829362293ad5


--

___
Python tracker 
<https://bugs.python.org/issue41247>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41202] Allow to provide custom exception handler to asyncio.run()

2020-07-09 Thread Yury Selivanov


Yury Selivanov  added the comment:

> So how about maybe:   

That wouldn't work. You still haven't explained what's wrong with calling `loop 
= asyncio.get_running_loop()` inside `async def main()`. That literally solves 
all problems without the need of us modifying any APIs.

--

___
Python tracker 
<https://bugs.python.org/issue41202>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37179] asyncio loop.start_tls() provide support for TLS in TLS

2020-07-08 Thread Yury Selivanov


Yury Selivanov  added the comment:

Looks like https://github.com/python/cpython/pull/17975 was forgotten and was 
never committed to 3.9. So it's 3.10 now.

Best bet for you is to use uvloop which should support the feature.

--

___
Python tracker 
<https://bugs.python.org/issue37179>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41247] asyncio.set_running_loop() cache running loop holder

2020-07-08 Thread Yury Selivanov


Yury Selivanov  added the comment:

> n python to know if there could be a context switch to get_running_loop while 
> set_running_loop is running.


No, it's protected by the GIL.

Good catch, and merged.

--
nosy: +yselivanov
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41247>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41244] Change to use str.join() instead of += when concatenating string

2020-07-08 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Changes in bpo-41242 were rejected not because the new code is worse, but 
> because it is not obviously and significantly better that the existing code. 
> Here status quo wins for the same reasons. This rule saves us from endless 
> rewriting the code and allows to focus on important things.

+1.

--

___
Python tracker 
<https://bugs.python.org/issue41244>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22239] asyncio: nested event loop

2020-07-07 Thread Yury Selivanov


Yury Selivanov  added the comment:

> The community is hurting *A LOT* right now because asyncio is intentionally 
> non-compatible with the traditional blocking approach that is not only still 
> prevalent it's one that a lot of us think is *easier* to work with.

Mike, I'm super happy with having you here and I encourage you to propose 
feature requests etc. That said, please don't use arguments like this here. 
Everyone has their own point of view and I, for example, haven't seen the "A 
LOT of community hurt" you're describing. I'm not implying that what you're 
saying is wrong, or that asyncio is perfect; the point is that it's just very 
subjective. The bug tracker is not the medium for these kind of remarks.

> That SQLAlchemy internally is not using this coding style, whether or not 
> that leads to new kinds of bugs, there are new kinds of bugs no matter what 
> kind of code a library uses, I don't think this hurts the user community.

You're free to use whatever approach you want in SQLAlchemy. We're here to 
share our advice and perspective (if we have any) and/or to discuss concrete 
proposals for API improvements or changes.

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22239] asyncio: nested event loop

2020-07-07 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Yeah, writing a trivial "event loop" to drive actually-synchronous code is 
> easy. Try it out:

This is exactly the approach I used in edgedb-python.

> I guess there's technically some overhead, but it's tiny.

Correct, the overhead isn't even detectable in microbenchmarks. In most async 
programs regular function calls dominate awaits by a few factors of magnitude.

> I think dropping 'await' syntax has two major downsides:

For the extra context, in the case of using this approach for something like 
edgedb-python these downsides don't really apply, because we're adapting 
async/await implementation to be sync. The async/await code can handle 
cancellation etc. whereas the sync code only needs to support the general 
protocol parsing flow. FWIW I don't think it would be possible to apply my 
approach to SQLA without a very invasive rewrite, which isn't probably worth it.

> tl;dr: I think switching from async/await -> greenlets would make it much 
> easier to write programs that are 90% correct, and much harder to write 
> programs that are 100% correct. That might be a good tradeoff in some 
> situations, but it's a lot more complicated than it seems.

Yeah, this sums up my opinion on this topic.

Also, having spent a couple of years writing and debugging big and small 
greenlet-heavy code bases I wouldn't want to touch them ever again. Your 
mileage may vary, Mike.

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   3   4   5   6   7   8   9   10   >