[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

I'm very happy that the issue is finally resolved.

But a bit offended that it took Andrew only 4 days to push it :)

> On 07 Nov 2016, at 17:04, Yury Selivanov  wrote:
> 
> 
> Yury Selivanov added the comment:
> 
> See https://github.com/python/asyncio/pull/452. I believe that the issue is 
> not resolved (almost).
> 
> What's left is documenting how to use get_event_loop and when to use 
> explicit/implicit loop passing.
> 
> Closing this issue.  Will open a new one for the documentation update.
> 
> --
> resolution:  -> fixed
> stage:  -> resolved
> status: open -> closed
> 
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

See https://github.com/python/asyncio/pull/452. I believe that the issue is not 
resolved (almost).

What's left is documenting how to use get_event_loop and when to use 
explicit/implicit loop passing.

Closing this issue.  Will open a new one for the documentation update.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Vincent Michel]

Vincent Michel added the comment:

I agree with Yury's ideas about the implementation of this feature. However, it 
is a bit confusing to have `asyncio.get_event_loop` defined as:

def get_event_loop():
policy = get_event_loop_policy()
return policy.get_running_loop() or policy.get_event_loop()

One would expect `asyncio.get_event_loop` to simply work as a shortcut for:

get_event_loop_policy().get_event_loop()

The root of the problem is that we're trying to define 3 concepts with only 2 
wordings. I think it is possible to solve this issue quite easily by renaming 
`AbstractLoopPolicy.get_event_loop` with `AbstractLoopPolicy.get_default_loop`. 
We'd end up with the following definitions:

- default_loop: current default loop as defined in the policy
- running_loop: current running loop (thread-wise) if any 
- event_loop: running loop if any, default_loop otherwise

Changing the API is always annoying, but in this case it would only affect the 
event loop policies. This is a pretty specific use case, and they'll have to be 
updated anyway in order to implement `set_running_loop`. asyncio.set_event_loop 
might be affected too, although it could be kept or deprecated.

Do you think it's worth the trouble?

--
nosy: +vxgmichel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

Yury, as you suggested posted to python-ideas 
(https://groups.google.com/forum/#!topic/python-ideas/ABOe22Mib44)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

> If I understood it correctly, get_event_loop() would never return "default" 
> event loop (in terms of current implementation) for a running task, because 
> it always be overridden with "running" event loop. If it's so, are you sure 
> it won't raise any problems with backward compatibility?

I think this is extremely unlikely.  I hope that there is not much code out 
there that relies on this behaviour.  But you have a point.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

Yury, 

> `get_event_loop()` will then try to use the `running_loop` object first, and 
> if nothing is there, fall back to its current implementation.

Do you think hiding "default" event loop completly from a currently executing 
coroutine is safe?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

Yury,

> Not sure I understand the question.

If I understood it correctly, get_event_loop() would never return "default" 
event loop (in terms of current implementation) for a running task, because it 
always be overridden with "running" event loop. If it's so, are you sure it 
won't raise any problems with backward compatibility?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

> Do you think hiding "default" event loop completly from a currently executing 
> coroutine is safe?

Not sure I understand the question.  Anyways, let's see what Guido thinks about 
this all.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

>> I now think that we don't need a new function for getting the currently 
>> running event loop.

> May I ask you to elaborate on this?

In my last message I propose to modify 'get_event_loop' to return the running 
loop reliably.  To me this is to me the most important thing -- this will make 
things like Task.current more reliable.

Adding another function (get_running_loop) won't change much. It would have a 
slightly different behaviour, but likely it won't be used anywhere in asyncio, 
or other libraries.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

Yury,

> I now think that we don't need a new function for getting the currently 
> running event loop.

May I ask you to elaborate on this? Asynchronous API I'm aware of (including  
other languages) typically allows to get "main" (which in asyncio is lazily 
created for main thread only), and "current". Sometimes there is also a 
convenient "global" which can be used when you want to do some async work 
outside. E.g. take a look at public API of Apple's Grand Central Dispatch, 
specifically `dispatch_get_current_queue`.

Even Python's thread module allows to get "current_thread" instead of passing 
it into run callback and requiring a user to carry it around by hand.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

> TBH, I don't fully understand Ilya's case with threads, synchronous 
> coroutines, possible deadlocks etc.

I feel sorry for sharing that example. It didn't help and made my points 
regarding original issue unclear, hidden behind example's complexity.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

> Yury, what do you think of this yourself? Maybe you can explain it better
than Ilya?

TBH, I don't fully understand Ilya's case with threads, synchronous coroutines, 
possible deadlocks etc.  So I'll just explain what I would want out of this.

I don't like a few aspects of the current design of 'get_event_loop'.  Mainly 
because it creates an event loop implicitly, or it can just fail with a 
RuntimeError under some conditions.  When one has a few event loops, implicitly 
called 'get_event_loop' from Future and other asyncio functions won't work 
correctly.  IMO, this is something that we at least should try to fix (even 
though it's not an idiomatic use case for asyncio).

I now think that we don't need a new function for getting the currently running 
event loop.  Instead we can try to improve the `get_event_loop()` function a 
little.

The core idea is to add a thread-local object `running_loop`, which will point 
to the currently running event loop in the current thread.  And `Loop._run` 
method will set/reset a reference to the loop to `running_loop`.

`get_event_loop()` will then try to use the `running_loop` object first, and if 
nothing is there, fall back to its current implementation.

Now, this is a point where everything becomes complicated. To quote 
myself,--"Yes, the patch part is easy",--I don't think it is that easy anymore 
;)

Currently, event loops are decoupled from policies.  This means that we can't 
make loops to use some hidden shared thread-local object (`running_loop`) that 
policies and loops will work with.  There has to be another new public policy 
APIs that loops will use, for instance: 'set_running_event_loop()'.  This won't 
be the first case of event loop policy APIs being called from event loops -- 
another example is the `get_child_watcher` method.

With this new method, `Loop._run` will then look like this:

  def _run(self):
  policy = get_event_loop_policy()
  policy.set_running_event_loop(self)
  try:
 ... # current Loop._run implementation
  finally:
  policy.set_running_event_loop(None)

`policy.set_running_event_loop` call would raise a RuntimeError if it's called 
if another event loop is currently running.

And the `get_event_loop()` function will look like this:

  def get_event_loop():
  policy = get_event_loop_policy()
  loop = policy._get_running_event_loop()
  if loop is not None:
  return loop
  # ... current get_event_loop implementation

So it all boils down to:

1. Adding a new policy API 'set_running_event_loop'.

2. Updating 'get_event_loop' to return the currently running event loop if it's 
available, or else resorting to its current behaviour.

With this change it will be completely safe to use "get_event_loop" from 
running coroutines or just about any code that runs in a context of an event 
loop.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Guido van Rossum]

Guido van Rossum added the comment:

As long as there are still things you find confusing about asyncio's model,
you should probably consider yourself unqualified to start proposing new
features...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

Yury, we're building our own CPython anyway (and we just updated to 3.5.1). I'd 
be glad to test the patch during the next iteration.

Guido, I think my use case mixes up other things I find confusing about 
asyncio: e.g. inablitity to synchronously perform code written as standalone 
coroutine. It deserves its own task and discussion.

The purpose of this request is that access to a running loop is handy within 
coroutine. It's already there, since coroutine is executed within its context. 
However, instead of "self" library provides API to only access some globally 
default loop.

Python does not require a user to pass self whenever instance methods needs to 
be called. I consider relationship between event loop and awaitables alike.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Guido van Rossum]

Guido van Rossum added the comment:

Yury, what do you think of this yourself? Maybe you can explain it better
than Ilya?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

Ilya, at this point I'd suggest you to craft a very clear and simple to digest 
email to python-ideas describing 'get_running_loop()', how it can be used, your 
case, other cases, etc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Guido van Rossum]

Guido van Rossum added the comment:

Still no green light. Ilya just seems to want something misguided.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

> Yury, could you submit a patch implements this feature?

Yes, the patch part is easy. The hard part is to get a green light from Guido 
on adding the 'get_running_loop()' function...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

Yury, could you submit a patch implements this feature?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

> If you're worried about blocking the event loop just to acquire the threading 
> lock that's part of threading.Event, you can use run_in_executor()

I'm more worried to delay invocation of a "synchronous" report because of 
outstanding tasks in event loop's queue. Given my situtation, that may not even 
happen because outstanding tasks may stop the loop by raising their own 
exceptions.

I'm willing to sacrifice them (i.e. I'm ok with not sending them) in order to 
send information about exception "for sure".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Guido van Rossum]

Guido van Rossum added the comment:

Instead of starting a new event loop, you should figure out a way to wait
for an event in the existing loop. IIUC that loop runs in a different
thread -- I think you can solve this by using a threading.Event that you
set from a wrapper coroutine running in your event loop and waited for in
the main loop. (If you're worried about blocking the event loop just to
acquire the threading lock that's part of threading.Event, you can use
run_in_executor().

Good luck!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

> You don't know what else that coroutine is waiting for, and it may be waiting 
> for some I/O whose socket is registered with the other event loop. Since the 
> other event loop won't make progress, you'd be deadlocked.

As an end user I know what my coroutines may be waiting for. It's out of 
question to expect an ability to reuse coroutine and its associated context in 
multiple loops.

> PS. If you have something you sometimes want to run synchronously and 
> sometimes as a coroutine, you probably need to refactor it somehow.

We have a service that is responsible for reporting various stages about our 
application's lifetime. Most of the time app does not care about when (and 
whether) data will be actually sent, so we run this service in its own thread, 
the rest of the app just schedules coroutines in its event loop (hidden and 
managed by service's API). However sometimes it does care and moreover needs to 
do that synchronously (i.e. when we handle fatal / unhandled exception. In 
order to reuse remaining code that constructs coroutines, in the place of 
invocation we create a temporary event loop and use run_until_complete.

This is all practical problem. The conceptual problem is that current API is 
not flexible enough to create an event policy that would do something more 
useful that changing default type of an event loop.

Having the ability to get currently running event loop from within executing 
coroutine sounds pretty convenient and in my opinion would greatly reduce the 
amount of passing loops everywhere for end users (library developery, 
unfortunately, have no chance to avoid this).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Guido van Rossum]

Guido van Rossum added the comment:

No, that specifically sounds like asking for trouble to me. You don't know
what else that coroutine is waiting for, and it may be waiting for some I/O
whose socket is registered with the other event loop. Since the other event
loop won't make progress, you'd be deadlocked.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Guido van Rossum]

Guido van Rossum added the comment:

PS. If you have something you sometimes want to run synchronously and
sometimes as a coroutine, you probably need to refactor it somehow.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

Guido, Probably a legitimate example of having multiple event loops in a single 
thread: you want to run certain coroutine synchronously without running 
thread's own event loop.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Guido van Rossum]

Guido van Rossum added the comment:

FWIW asyncio's own test suite makes sure that the loop is indeed passed
everywhere by setting the default loop to None. If a library chooses to
pass the loop around like this it should structure its tests the same way.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

>> Update some places in asyncio where we currently use "get_event_loop()", 
>> such as Future constructor, Task.current_task, etc.

> Yury, do you have an idea how it could be done?

I think I do.  To keep the backwards compatibility, we'll need to update the 
"get_event_loop()" function as follows:

  def get_event_loop():
 loop = _running_loop.loop
 if loop is None:
 loop = get_event_loop_old_impl()
 return loop

No need to actually update Future or Task or any other asyncio code.

The get_running_loop() would look like this:

  def get_running_loop():
 loop = _running_loop.loop
 if loop is None:
 raise RuntimeError('no running event loop')
 return loop

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[R. David Murray]

R. David Murray added the comment:

The way I avoid the errors of failing to always pass the loop through in my 
application library, currently, is to run my tests in a thread with no event 
loop set up, so that get_event_loop will raise an error.  Take that for what it 
is worth, since I don't claim to know what best practices are in asyncio 
programming.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

> Update some places in asyncio where we currently use "get_event_loop()", such 
> as Future constructor, Task.current_task, etc.

Yury, do you have an idea how it could be done?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

Yury, that would do it.

Guido, that's indeed might be an anti-pattern. But it looks like passing event 
loop around is just a worse version of it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

> Honestly I think it's pretty crazy and out there to have multiple event
loops in the same thread. That feels like an anti-pattern inspired by some
other event loop APIs (in other languages) that encourage this. But asyncio
was not designed for that.

I agree.

OTOH, if you're designing a library for asyncio, you want it to be as foolproof 
as possible, so many people simply pass an event loop everywhere.  It's 
especially annoying when you have a huge chunk of code that didn't need the 
loop, and then when something needs it you have to refactor everything or use 
"get_event_loop".

In terms of performance, I don't think this is gonna affect anything, it's just 
a couple of additional thread-local sets in the Loop.run method.

And we can design "get_running_loop" to raise a clear exception if no loop is 
currently running.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Guido van Rossum]

Guido van Rossum added the comment:

Honestly I think it's pretty crazy and out there to have multiple event
loops in the same thread. That feels like an anti-pattern inspired by some
other event loop APIs (in other languages) that encourage this. But asyncio
was not designed for that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

> Thread may have multiple event loops, but only one, explicitly associated, is 
> default. And it's not necessary one which is currently running.

Ah, I got it.  You know what, this actually is starting to make sense.

Guido, what do you think about this?  Essentially, (Ilya, feel free to correct 
me if I'm wrong), we can implement the following:

1. Add a new thread-local storage, say "_running_loop"

2. Add another method "asyncio.get_running_loop()"

3. Update some places in asyncio where we currently use "get_event_loop()", 
such as Future constructor, Task.current_task, etc.

This might actually solve an awkwardness of get_event_loop(), which when 
called, instantiates an event loop if it doesn't find one, which makes it 
difficult to know if an event loop was running before the call.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

> In this case I'm not sure how this is different from the current 
> "get_event_loop"

Thread may have multiple event loops, but only one, explicitly associated, is 
default. And it's not necessary one which is currently running.

I think what I propose here can be expressed in Python terms as an implicit 
context manager that replaces Thread's default event loop once while it "runs" 
particular event loop then switches it back (possible to None) once loop is 
stopped.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

To clarify: the problem lies in "return currently running event loop when 
called from within a coroutine" -- what if there is no GIL and we have a few 
event loops running?  You'd need to use a threadlocal storage.  In this case 
I'm not sure how this is different from the current "get_event_loop".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

> Probably it would make sense to modify behavior of the default policy to make 
> get_event_loop to return Thread-local event loop when called from 
> out-of-event-loop and return currently running event loop when called from 
> within a coroutine.

The problem with this approach is that you're relying on the presence of GIL.  
If, let's say, PyPy releases tomorrow with 3.5 and STM, this won't work.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

> Why does it have to be a standard part of asyncio?

I've only seen few libraries that deal with asyncio so far (aiohttp, pyzmq), 
but my general impression is that this is a generic problem.

With asyncio code should be (ideally) written as a set of coroutines which 
schedule each other or are called in response to monitored events. That means 
(and asyncio very implementation shows that) loop has to be "self" for all 
coroutines it executes.

Thread-local default event loop is a good solution to get an entry point into 
the event loop from an out-of-event-loop execution location. But (ideally) 
there will be exactly one place in code where this behavior is convenient: when 
you "instantiate" and run event loop. After that the event loop becomes "self" 
for each coroutine it may run and therefore it's now convenient for 
get_event_loop to return currently running event loop.

Probably it would make sense to modify behavior of the default policy to make 
get_event_loop to return Thread-local event loop when called from 
out-of-event-loop and return currently running event loop when called from 
within a coroutine.

> Why don't you just use 'loop = asyncio.get_event_loop()' in places where you 
> need the loop?

As David pointed out, I'm not using thread-local event loop.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[R. David Murray]

R. David Murray added the comment:

Because, as indicated by the OP, I'm not using the default event loop of the 
thread.

Thinking about my code, I now remember that the reason I went down this path 
was because I wanted to make sure that my functions, like asyncio functions, 
accepted the optional loop parameter, and that it worked. For it to work my 
code pretty much has to pass loop everywhere, because that was a design 
decision I made early on.  So maybe there's nothing interesting for me here 
after all.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Yury Selivanov]

Yury Selivanov added the comment:

> Currently if one needs lazily resolve event loop depending on where awaitable 
> is being awaited have to pass loop everywhere explicitly. That quickly 
> becomes an unnecessary noise in interfaces of callables. 

Why don't you just use 'loop = asyncio.get_event_loop()' in places where you 
need the loop?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[R. David Murray]

R. David Murray added the comment:

Can we at least make it part of the documentation?  It's not obvious to me, at 
least, how to do it.  (Maybe I just haven't thought about it carefully enough.) 
 For that matter, it wasn't obvious to me it could even be done, so I've been 
passing loop everywhere in my code rather than thinking about how to avoid it; 
which is another argument for documenting it at least.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Guido van Rossum]

Guido van Rossum added the comment:

Can't you easily write such a policy yourself? Why does it have to be a
standard part of asyncio?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

New submission from Ilya Kulakov:

Currently if one needs lazily resolve event loop depending on where awaitable 
is being awaited have to pass loop everywhere explicitly. That quickly becomes 
an unnecessary noise in interfaces of callables. 

Consider an example where a coroutine which constructs other awaitables can be 
executed in arbitrary loop:

import asyncio
import datetime

async def foo_coro(loop):
await some_other_coro(loop)

async def bar_coro(loop):
await asyncio.ensure_future(..., loop=loop)

async def baz_coro(loop):
await asyncio.gather(foo_coro(loop), bar_coro(loop), loop=loop)

loop = asyncio.get_event_loop()
loop.run_until_complete(multiple_coros(loop))
loop.close()

It would be nice, if all functions that belong to an event loop instance as 
well as asyncio helpers that accept a loop would set default event loop to one 
that was passed to these functions. So that the example above could be 
rewritten as:

import asyncio
import datetime

async def foo_coro():
await some_other_coro()

async def bar_coro():
await asyncio.ensure_future(...)

async def baz_coro():
await asyncio.gather(foo_coro(), bar_coro())

loop = asyncio.get_event_loop()
loop.run_until_complete(multiple_coros())
loop.close()

--
components: asyncio
messages: 264941
nosy: Ilya.Kulakov, gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: ascynio should provide a policy to address pass-loop-everywhere problem
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com