Thank you! It was never the code I posted, but some very old leftover __del__ method. This way "disonnect" would be called twice and the second (implict, __del__) one caused the error message. After deleting the __del__ part entirely everything works like a charm. Damn implicit code :)
Thank you very much for your help, it would have taken me ages to consider the __del__ method. On Friday, March 28, 2014 1:25:10 AM UTC+1, Guido van Rossum wrote: > > I'm out of ideas, sorry. Perhaps the code in the __del__ method is wrong > and the generator can be in the state that triggers the error message after > it has run. Perhaps one of your asserts triggers but it itself is silenced > by a similar issue. (Hm, I would definitely check that hypothesis -- use an > explicit print or logging call on the condition the assert checks, > especially the "assert not pending" one.) > > > On Thu, Mar 27, 2014 at 4:43 PM, Florian Rüchel > <[email protected]<javascript:> > > wrote: > >> I tried it and it yields the same result: The assertion passes, the >> exception is None and the error is still being logged. >> >> def done(fut): >> print("Done") >> assert fut.exception() is None >> >> The "Done" part is printed as before. >> >> >> On Friday, March 28, 2014 12:22:00 AM UTC+1, Guido van Rossum wrote: >> >>> Try calling fut.exception() inside the done() callback. >>> >>> >>> On Thu, Mar 27, 2014 at 4:18 PM, Florian Rüchel <[email protected]>wrote: >>> >>>> Oh sorry... No it didn't solve my problem. The assertions all pass and >>>> the error messages remain. I also added logging as you recommended, but it >>>> only shows that the exception is None. >>>> >>>> >>>> On Friday, March 28, 2014 12:16:26 AM UTC+1, Guido van Rossum wrote: >>>> >>>>> So does this solve your problem or not? You didn't say... >>>>> >>>>> >>>>> On Thu, Mar 27, 2014 at 4:15 PM, Florian Rüchel >>>>> <[email protected]>wrote: >>>>> >>>>>> I modified the code a little, so I make sure it is done and has no >>>>>> exception: >>>>>> >>>>>> def done(fut): >>>>>> print("Future is done") >>>>>> for task in conn_tasks: >>>>>> task.add_done_callback(done) >>>>>> done, pending = yield from asyncio.wait(conn_tasks, timeout=5) >>>>>> assert not pending >>>>>> for task in done: >>>>>> assert task.exception() is None >>>>>> assert task.result() is None >>>>>> >>>>>> This executes cleanly, without throwing an exception and prints >>>>>> "Future is done" for each connection. >>>>>> >>>>>> On Friday, March 28, 2014 12:07:41 AM UTC+1, Guido van Rossum wrote: >>>>>> >>>>>>> Looks to me like you're never asking for the result of those Futures >>>>>>> you're passing to asyncio.wait(). It waits until the Futures are >>>>>>> complete, >>>>>>> but it doesn't inspect whether they have errors. You need something like >>>>>>> >>>>>>> for task in done: >>>>>>> if task.exception(): ...log it... >>>>>>> >>>>>>> >>>>>>> On Thu, Mar 27, 2014 at 3:19 PM, Florian Rüchel < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> I can post the code, I just thought it might be too much. However, >>>>>>>> I'll just post the relevant section. Contrary to multithreaded >>>>>>>> programming >>>>>>>> here most of the times the local section is enough (I love asyncio :)) >>>>>>>> >>>>>>>> Anyways here is the main function: >>>>>>>> >>>>>>>> try: >>>>>>>> self.event_loop.run_forever() >>>>>>>> except KeyboardInterrupt: >>>>>>>> exit_main_loop() >>>>>>>> finally: >>>>>>>> print("Running shutdown cleanup") >>>>>>>> collector.cancel() >>>>>>>> >>>>>>>> @asyncio.coroutine >>>>>>>> def shutdown(): >>>>>>>> yield from self.connection_pool.close() >>>>>>>> self.event_loop.run_until_complete(shutdown()) >>>>>>>> self.event_loop.close() >>>>>>>> >>>>>>>> And the close method: >>>>>>>> >>>>>>>> @asyncio.coroutine >>>>>>>> def close(self): >>>>>>>> try: >>>>>>>> if self.returned_connections: >>>>>>>> msg = ("Not all connections were returned: %s" >>>>>>>> % self.returned_connections) >>>>>>>> log.error(msg) >>>>>>>> raise ValueError(msg) >>>>>>>> finally: >>>>>>>> conn_tasks = [asyncio.Task(conn.disconnect()) >>>>>>>> for conn in self.all_connections] >>>>>>>> done, pending = yield from asyncio.wait(conn_tasks, >>>>>>>> timeout=5) >>>>>>>> assert not pending >>>>>>>> >>>>>>>> Relevant is the finally part: I wrap each connection in a task and >>>>>>>> wait for them together. Removing the timeout did not help (I didn't >>>>>>>> expect >>>>>>>> it would, no exception is raised). >>>>>>>> >>>>>>>> Per existing connection (up to 10) a message is produced: >>>>>>>> >>>>>>>> 2014-03-27 23:11:58,143 ERROR[asyncio] Coroutine 'disconnect' >>>>>>>> defined at /path/to/library/connection.py:109 was never yielded >>>>>>>> from >>>>>>>> >>>>>>>> I have made sure that the amount of logging messages matches >>>>>>>> exactly the number of connections created (by counting the logging >>>>>>>> messages >>>>>>>> inside connection's __init__ function). In a very simple case there >>>>>>>> was >>>>>>>> only one connection, so only one error entry and only one log entry >>>>>>>> for a >>>>>>>> created connection. >>>>>>>> >>>>>>>> Hope this additional information helps uncover the issue. Thanks >>>>>>>> for the response. >>>>>>>> >>>>>>>> On Thursday, March 27, 2014 10:23:10 AM UTC+1, Florian Rüchel wrote: >>>>>>>>> >>>>>>>>> Hi everyone, >>>>>>>>> >>>>>>>>> I currently have a problem I cannot figure out. My code is running >>>>>>>>> perfectly fine, but when using "PYTHONASYNCIODEBUG=1" it reports that >>>>>>>>> a >>>>>>>>> function was never yielded from per instance of that class I have. >>>>>>>>> The >>>>>>>>> problem is I cannot create an example that replicates the issue and >>>>>>>>> the >>>>>>>>> application is fairly large so maybe someone can give a hint on where >>>>>>>>> I >>>>>>>>> might go looking. The function in question is a method of a >>>>>>>>> connection >>>>>>>>> class. The class manages a TCP connection that has to send an init >>>>>>>>> and a >>>>>>>>> disconnect request. It is the disconnect method that is reported as >>>>>>>>> never >>>>>>>>> yielded from. I have log entries that prove that this function was >>>>>>>>> actually >>>>>>>>> run and I have tried wrapping it in asyncio.Task and asyncio.async >>>>>>>>> and I >>>>>>>>> also had a wrapping coroutine that *was *called and an explicit >>>>>>>>> "yield from connection.disconnect()" statement. >>>>>>>>> >>>>>>>>> In all three cases the proper log entries were written and the >>>>>>>>> server got the disconnect request, so I know it was properly >>>>>>>>> executed. Now >>>>>>>>> I am wondering if I have a bug in my application that is hidden >>>>>>>>> somewhere >>>>>>>>> between the layers that is repsonsible for this message. Or is it >>>>>>>>> possible >>>>>>>>> to trigger this error message even though the function was properly >>>>>>>>> called? >>>>>>>>> >>>>>>>>> I'd also like to note that this is *the only* method where this >>>>>>>>> happens, all other methods on all other classes are never reported as >>>>>>>>> problematic.I have checked the method against others but I cannot >>>>>>>>> find any >>>>>>>>> difference at all: They are called with yield from or as tasks, they >>>>>>>>> all >>>>>>>>> use other coroutines (the connect and disconnect method actually both >>>>>>>>> use *yield >>>>>>>>> from self.talk(...)*) and they are all properly decorated. >>>>>>>>> >>>>>>>>> I'm sorry I cannot provide any code examples, but since I am >>>>>>>>> unable to replicate the issue in a small sample, I fear I need >>>>>>>>> further >>>>>>>>> guidance on where to look for the issue. >>>>>>>>> >>>>>>>>> Many thanks in advance and Regards, >>>>>>>>> Florian >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> --Guido van Rossum (python.org/~guido) >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> --Guido van Rossum (python.org/~guido) >>>>> >>>> >>> >>> >>> -- >>> --Guido van Rossum (python.org/~guido) >>> >> > > > -- > --Guido van Rossum (python.org/~guido) >
