[issue25074] Bind logger and waninigs modules for asyncio __del__ methods

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution:  -> wont fix
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25074] Bind logger and waninigs modules for asyncio __del__ methods

[Andrew Svetlov]

Andrew Svetlov added the comment:

Yes, the approach use hackery shadows for binding global variables on shutdown 
stage.
The same I've found in subprocess module: see 
https://github.com/python/cpython/blob/master/Lib/subprocess.py#L1544

The problem is: I want to inform users about unclosed resources.
Printouts like
`TypeError: 'NoneType' object is not callable`
or 
`AttributeError: 'NoneType' object has no attribute 'warn'` are too confusing 
for casual developer.

On shutdown logger and warnings modules may be destroyed at the moment of 
__del__ call, so `warnings.warn`, `loop.call_exception_handler` etc. are 
forbidden.

As an option we can add atexit handler to store shutdown flag into loop object 
(I don't know other way to get info is interpreter in shutdown or not).

According to the flag we can eihter print some short text about non-closed 
resources or raise proper ResourceWarning along with comprehensive logging with 
stacktrace if PYTHONASYNCIO is on.

Maybe I've missed something and you can suggest better solution.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25074] Bind logger and waninigs modules for asyncio __del__ methods

[Andrew Svetlov]

Andrew Svetlov added the comment:

> You should try to implement something in aiohttp or even in the application 
> to cleanup objects at exit. For example, it's probably wrong if you still 
> have tasks when the event loop is closed. Especially if tasks are still 
> pending.

The problem is for client API. For server I have more control and  implemented 
checks like you suggest.

For for client lazy user writes floppy code like:

client = aiohttp.ClientSession()
resp = yield from client.get(url)
text = yield from resp.text()

Client is closed by GC, at the moment is not determined is loop has been closed 
or not (GC releases object in non-determenistic way).

So without changes I just cannot use `loop.call_exception_handler()` in 
`__del__` methods.

It's not only aiohttp issue but we should get rid of `call_exception_handler()` 
reports in asyncio itself for the same reason. 

We have `logger` and `warnings` modules usage e.g. in asyncio transports, it's 
not safe if transport is not closed properly before interpreter shutdown.

`subprecess` module has tricks for proper destruction like I've used in my 
patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25074] Bind logger and waninigs modules for asyncio __del__ methods

[STINNER Victor]

STINNER Victor added the comment:

I don't like such "hacks". IMHO It's too late to try to log errors, to execute 
code to cleanup objects, etc. 

You should try to implement something in aiohttp or even in the application to 
cleanup objects at exit. For example, it's probably wrong if you still have 
tasks when the event loop is closed. Especially if tasks are still pending. See 
this part of the doc which lists "pending tasks at exit":
https://docs.python.org/dev/library/asyncio-dev.html#chain-coroutines-correctly

For "exceptions never consumed", I proposed a different enhancement in asyncio 
directly:
https://bugs.python.org/issue24598

What do you think?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25074] Bind logger and waninigs modules for asyncio __del__ methods

[Andrew Svetlov]

New submission from Andrew Svetlov:

See https://github.com/KeepSafe/aiohttp/issues/497 for the reason.

Desctructors and descendant code (`loop.call_exception_handler()`) can be 
called on interpreter shutdown stage, which leads to printouts like 'None 
object has no attribute "errror"'.

--
assignee: asvetlov
components: asyncio
files: asyncio_bind_modules.patch
keywords: patch
messages: 250490
nosy: asvetlov, gvanrossum, haypo, yselivanov
priority: normal
severity: normal
stage: patch review
status: open
title: Bind logger and waninigs modules for asyncio __del__ methods
type: enhancement
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file40442/asyncio_bind_modules.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com