[issue37006] Add top level await statement support for doctest

[Austin "Paco" Rainwater]

Change by Austin "Paco" Rainwater :


--
nosy: +pacorain

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
nosy: +BTaskaya
versions: +Python 3.9 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[jack1142]

Change by jack1142 :


--
nosy: +jack1142

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[Matthias Bussonnier]

Matthias Bussonnier  added the comment:

As a reference, PR from Yuri for an asyncREPL  `python -m asyncio` which I 
believe he want in 3.8:

https://github.com/python/cpython/pull/13472

I'm also likely to align IPython behavior on whatever core python decides.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Agree with Nathaniel.
There is no need to rush now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[Nathaniel Smith]

Nathaniel Smith  added the comment:

As far as things like run/run_until_complete/etc go, I think doctests should 
have the same semantics as async REPLs, whatever those end up being.

Given that we don't actually have mature async REPLs, that the core feature to 
enable them only landed a few days ago, and the 3.8 freeze is almost here, I 
think we should defer the doctest discussion for 3.9. Hopefully by then, things 
will have settled down more in async REPL land, and it'll be more obvious what 
doctest should do.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[Matthias Bussonnier]

Matthias Bussonnier  added the comment:

The other thing to think about is `ensure_future` and `create_task`, they may 
not move forward until a foreground task is running. 

You can keep a loop running between lines or code-chunks, but then doctest 
cannot contain `asyncio.run()`.

I'm leaning toward conservatisme, and make things better but not perfect for 
3.8; potentially improving, in minor release of 3.9

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I tried using AsyncioDocTestRunner that inherits from DocTestRunner and most of 
the current DocTestRunner is synchronous and the execution happens in __run 
that seems to cause problem due to name mangling inheriting and changing it. 
Also python -m doctest by default uses testmod/testfile that use DocTestRunner 
so I thought to change DocTestRunner would be simpler and existing code can use 
added doctest flag without changing runner. 

To be little more clear by each line I meant each example. So in below "async 
with session.get(url) as resp:\ntext = await resp.text()" counts as a 
single example whose code object is evaluated in asyncio.run which I said as 
per line by mistake.

>>> async with session.get(url) as resp:
... text = await resp.text()


Your concerns are reasonable about asyncio.run per example seem to be over kill 
and might not work in few cases. I also didn't think about trio. I will look 
into those. Thanks for the pointers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Please keep in mind: not only asyncio can be used to execute async/await code.

For example, trio has completely different implementation but utilizes async 
functions as well.

Probably we need to customize it, maybe by inheriting AsyncioDocTestRunner from 
DocTestRunner base class.

Also, using asyncio.run() for every line is not a good idea.
asyncio.run() creates an new loop instance every time.
Thus, the following doesn't work:
>>> session = aiohttp.ClientSession()
>>> async with session.get(url) as resp:
... text = await resp.text()

The session is bound to loop, but the loop is changed by the next line.
loop.run_until_complete() is not ideal too: it creates a new task every time.
In AsyncioTestCase (https://github.com/python/cpython/pull/13386) I'm avoiding 
this problem by keeping a long-running task for saving the execution context 
between setup/teardown and test itself.

--
nosy: +njs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[Karthikeyan Singaravelan]

New submission from Karthikeyan Singaravelan :

Since issue34616 is merged that allows using compile flags to support top level 
await statements I think it would be good to add support for top level await in 
doctest. This would help in concise examples in docs where await statements 
need to be wrapped in async def wrapper functions currently. This can be done 
using a doctest flag like ALLOW_TOP_LEVEL_AWAIT so that places where top level 
await is needed it can be explicitly marked as such so that when users copy 
paste code they are aware that it requires top level await statement.

I have implemented a simple patch where ALLOW_TOP_LEVEL_AWAIT flag (not to be 
confused with ast module flag) is added to doctest and if the doctest line has 
the flag then the ast flag is added th compileflags and then await 
eval(code_object) is used and then the awaitabe is executed with asyncio.run. 
Synchronous code has usual exec(code_object). I don't see any doctest failures 
with this patch against our Doc folder and test_doctest.

Few downsides is that it requires ast import for the flag value which could be 
little heavy but inspect module is already imported and I think it's an okay 
tradeoff for doctest. I have used asyncio.run and I am not sure if there is an 
efficient way to run awaitables. Feedback welcome.

Patch : 
https://github.com/python/cpython/compare/master...tirkarthi:asyncio-await-doctest

# await_flag_doctest.rst
>>> import asyncio
>>> await asyncio.sleep(1.0)  # doctest: +ALLOW_TOP_LEVEL_AWAIT

cpython git:(asyncio-await-doctest)   time ./python.exe -m doctest 
await_flag_doctest.rst
./python.exe -m doctest await_flag_doctest.rst  0.31s user 0.02s system 24% cpu 
1.343 total

# await_no_flag_doctest.rst that will fail
>>> import asyncio
>>> await asyncio.sleep(1.0)

cpython git:(asyncio-await-doctest)   time ./python.exe -m doctest 
await_no_flag_doctest.rst
**
File "await_no_flag_doctest.rst", line 2, in await_no_flag_doctest.rst
Failed example:
await asyncio.sleep(1.0)
Exception raised:
Traceback (most recent call last):
  File 
"/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/doctest.py", line 
1338, in __run
code = compile(example.source, filename, "single",
  File "", line 1
SyntaxError: 'await' outside function
**
1 items had failures:
   1 of   2 in await_no_flag_doctest.rst
***Test Failed*** 1 failures.
./python.exe -m doctest await_no_flag_doctest.rst  0.35s user 0.03s system 94% 
cpu 0.393 total

--
components: asyncio
messages: 343160
nosy: asvetlov, mbussonn, xtreak, yselivanov
priority: normal
severity: normal
status: open
title: Add top level await statement support for doctest
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com