Re: [Async-sig] async testing question

[Martin Richard]

Hi,

asynctest provides a asynctest.TestCase class, inheriting
unittest.TestCase. It supports coroutines as test cases and adds a few
other useful features like checking that there are no scheduled callbacks
left(see
http://asynctest.readthedocs.io/en/latest/asynctest.case.html#asynctest.case.asynctest.fail_on)
or ClockedTestCase which allows to to control time in the test.

Sorry for the bugs left in asynctest, I try to add as many tests as
possible to covers all cases, but I'm having a hard time keeping the
library compatible with unittest. A few remarks though:
* keeping the behavior of asynctest in sync with unittest sometimes leads
to unexpected behaviors making asynctest look more buggy than it is (at
least I hope so...),
* some libraries are hard to mock correctly because they use advanced
features, for instance, aiohttp uses its own coroutine type, I still don't
know what can be done with those cases,
* I'm also thinking about removing support of python 3.4 (@coroutine
decorator, etc) as it's a lot of work.

Unfortunately, I only have a few hours here and there to work on my free
time, as I'm not sponsored by my employer anymore. And, as many other
open-source libraries out there, I don't receive a lot of feedback nor help
:)

Thanks for using (or at least trying to use) asynctest!

Martin

2017-07-04 9:38 GMT+02:00 Alex Grönholm :

> For asyncio, you can write your test functions as coroutines if you use
> pytest-asyncio. You can even write test fixtures using coroutines. Mocking
> coroutine functions can be done using asynctest, although I've found that
> library a bit buggy.
>
>
>
> Chris Jerdonek kirjoitti 02.07.2017 klo 00:00:
>
>> On Sat, Jul 1, 2017 at 1:42 PM, Nathaniel Smith  wrote:
>>
>>> On Jul 1, 2017 3:11 AM, "Chris Jerdonek" 
>>> wrote:
>>> Is there a way to write a test case to check that task.cancel() would
>>> behave correctly if, say, do_things() is waiting at the line
>>> do_more()?
>>>
>>> One possibility for handling this case with a minimum of mocking would
>>> be to
>>> hook do_more so that it calls task.cancel and then calls the regular
>>> do_more.
>>>
>>> Beyond that it depends on what the actual functions are, I guess. If
>>> do_more
>>> naturally blocks under some conditions then you might be able to set up
>>> those conditions and then call cancel. Or you could try experimenting
>>> with
>>> tests that call sleep(0) a fixed number of times before issuing the
>>> cancel,
>>> and repeat with different iteration counts to find different cancel
>>> points.
>>>
>> Thanks, Nathaniel. The following would be overkill in my case, but
>> your suggestion makes me wonder if it would make sense for there to be
>> testing tools that have functions to do things like "run the event
>> loop until  is at ." Do such things
>> exist? This is a little bit related to what Dima was saying about
>> tools.
>>
>> --Chris
>> ___
>> Async-sig mailing list
>> Async-sig@python.org
>> https://mail.python.org/mailman/listinfo/async-sig
>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>
>
> ___
> Async-sig mailing list
> Async-sig@python.org
> https://mail.python.org/mailman/listinfo/async-sig
> Code of Conduct: https://www.python.org/psf/codeofconduct/
>



-- 
Martin  Richard
www.martiusweb.net
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

Re: [Async-sig] async documentation methods

[Dima Tisnek]

That's good start, looks like it would satisfy asyncio-only code :)

I haven't noticed that earlier.

On 4 July 2017 at 16:40, Andrew Svetlov  wrote:
> Did you look on
> https://github.com/python/cpython/blob/master/Lib/asyncio/test_utils.py#L265
> ?
>
> On Tue, Jul 4, 2017 at 1:04 PM Dima Tisnek  wrote:
>>
>> Come to think of it, what sane tests need is a custom event loop or clever
>> mocks around asyncio.sleep, asyncio.Condition.wait, etc. So that code under
>> test never sleeps.
>>
>> In simple cases actual delay in the event loop would raise an exception.
>>
>> A full solution would synchronise asyncio.sleep and friends with
>> time.time, time.monotonic and friends, so that a if the loop were to delay,
>> it would advance global/virtual time instead. I think I saw such library for
>> synchronous code, probably with limitations...
>>
>>
>> In any case you should not have to add delays in your mocks or fixtures to
>> hack specific order of task execution by the event loop.
>>
>> My 2c,
>> D.
>>
>> On Jul 4, 2017 9:34 AM, "Alex Grönholm"  wrote:
>>>
>>> Yeah, but that doesn't answer my question :)
>>>
>>>
>>> Chris Jerdonek kirjoitti 04.07.2017 klo 10:02:

 On Mon, Jul 3, 2017 at 11:49 PM, Alex Grönholm
  wrote:
>
> The real question is: why doesn't vanilla Sphinx have any kind of
> support
> for async functions which have been part of the language for quite a
> while?

 It looks like this is the issue (which Brett filed in Nov. 2015):
 https://github.com/sphinx-doc/sphinx/issues/2105

 --Chris

>
>
> Nathaniel Smith kirjoitti 01.07.2017 klo 13:35:
>>
>> If we're citing curio and sphinxcontrib-asyncio I guess I'll also
>> mention sphinxcontrib-trio [1], which was inspired by both of them
>> (and isn't in any way specific to trio). I don't know if the python
>> docs can use third-party sphinx extensions, though, and it is a bit
>> opinionated (in particular it calls async functions async functions
>> instead of coroutines).
>>
>> For the original text, I'd probably write something like::
>>
>>  You acquire a lock by calling ``await lock.acquire()``, and
>> release
>> it with ``lock.release()``.
>>
>> -n
>>
>> [1] https://sphinxcontrib-trio.readthedocs.io/en/latest/
>>
>> On Fri, Jun 30, 2017 at 8:31 AM, Brett Cannon 
>> wrote:
>>>
>>> Curio uses `.. asyncfunction:: acquire` and it renders as `await
>>> acquire()`
>>> at least in the function definition.
>>>
>>> On Fri, 30 Jun 2017 at 03:36 Andrew Svetlov
>>> 
>>> wrote:

 I like "two methods, `async acquire()` and `release()`"

 Regarding to extra markups -- I created sphinxcontrib-asyncio [1]
 library
 for it. Hmm, README is pretty empty but we do use the library for
 documenting aio-libs and aiohttp [2] itself

 We use ".. comethod:: connect(request)" for method and "cofunction"
 for
 top level functions.

 Additional markup for methods that could be used as async context
 managers:

  .. comethod:: delete(url, **kwargs)
 :async-with:
 :coroutine:

 and `:async-for:` for async iterators.


 1. https://github.com/aio-libs/sphinxcontrib-asyncio
 2. https://github.com/aio-libs/aiohttp

 On Fri, Jun 30, 2017 at 1:11 PM Dima Tisnek 
 wrote:
>
> Hi all,
>
> I'm working to improve async docs, and I wonder if/how async
> methods
> ought to be marked in the documentation, for example
> library/async-sync.rst:
>
> """ ... It [lock] has two basic methods, `acquire()` and
> `release()`.
> ...
> """
>
> In fact, these methods are not symmetric, the earlier is
> asynchronous
> and the latter synchronous:
>
> Definitions are `async def acquire()` and `def release()`.
> Likewise user is expected to call `await .acquire()` and
> `.release()`.
>
> This is user-facing documentation, IMO it should be clearer.
> Although there are examples for this specific case, I'm concerned
> with
> general documentation best practice.
>
> Should this example read, e.g.:
> * two methods, `async acquire()` and `release()`
> or perhaps
> * two methods, used `await x.acquire()` and `x.release()`
> or something else?
>
> If there's a good example already Python docs or in some 3rd party
> docs, please tell.
>
> Likewise, should there be 

Re: [Async-sig] async documentation methods

[Dima Tisnek]

Come to think of it, what sane tests need is a custom event loop or clever
mocks around asyncio.sleep, asyncio.Condition.wait, etc. So that code under
test never sleeps.

In simple cases actual delay in the event loop would raise an exception.

A full solution would synchronise asyncio.sleep and friends with time.time,
time.monotonic and friends, so that a if the loop were to delay, it would
advance global/virtual time instead. I think I saw such library for
synchronous code, probably with limitations...


In any case you should not have to add delays in your mocks or fixtures to
hack specific order of task execution by the event loop.

My 2c,
D.

On Jul 4, 2017 9:34 AM, "Alex Grönholm"  wrote:

> Yeah, but that doesn't answer my question :)
>
>
> Chris Jerdonek kirjoitti 04.07.2017 klo 10:02:
>
>> On Mon, Jul 3, 2017 at 11:49 PM, Alex Grönholm 
>> wrote:
>>
>>> The real question is: why doesn't vanilla Sphinx have any kind of support
>>> for async functions which have been part of the language for quite a
>>> while?
>>>
>> It looks like this is the issue (which Brett filed in Nov. 2015):
>> https://github.com/sphinx-doc/sphinx/issues/2105
>>
>> --Chris
>>
>>
>>>
>>> Nathaniel Smith kirjoitti 01.07.2017 klo 13:35:
>>>
 If we're citing curio and sphinxcontrib-asyncio I guess I'll also
 mention sphinxcontrib-trio [1], which was inspired by both of them
 (and isn't in any way specific to trio). I don't know if the python
 docs can use third-party sphinx extensions, though, and it is a bit
 opinionated (in particular it calls async functions async functions
 instead of coroutines).

 For the original text, I'd probably write something like::

  You acquire a lock by calling ``await lock.acquire()``, and release
 it with ``lock.release()``.

 -n

 [1] https://sphinxcontrib-trio.readthedocs.io/en/latest/

 On Fri, Jun 30, 2017 at 8:31 AM, Brett Cannon  wrote:

> Curio uses `.. asyncfunction:: acquire` and it renders as `await
> acquire()`
> at least in the function definition.
>
> On Fri, 30 Jun 2017 at 03:36 Andrew Svetlov 
> wrote:
>
>> I like "two methods, `async acquire()` and `release()`"
>>
>> Regarding to extra markups -- I created sphinxcontrib-asyncio [1]
>> library
>> for it. Hmm, README is pretty empty but we do use the library for
>> documenting aio-libs and aiohttp [2] itself
>>
>> We use ".. comethod:: connect(request)" for method and "cofunction"
>> for
>> top level functions.
>>
>> Additional markup for methods that could be used as async context
>> managers:
>>
>>  .. comethod:: delete(url, **kwargs)
>> :async-with:
>> :coroutine:
>>
>> and `:async-for:` for async iterators.
>>
>>
>> 1. https://github.com/aio-libs/sphinxcontrib-asyncio
>> 2. https://github.com/aio-libs/aiohttp
>>
>> On Fri, Jun 30, 2017 at 1:11 PM Dima Tisnek  wrote:
>>
>>> Hi all,
>>>
>>> I'm working to improve async docs, and I wonder if/how async methods
>>> ought to be marked in the documentation, for example
>>> library/async-sync.rst:
>>>
>>> """ ... It [lock] has two basic methods, `acquire()` and `release()`.
>>> ...
>>> """
>>>
>>> In fact, these methods are not symmetric, the earlier is asynchronous
>>> and the latter synchronous:
>>>
>>> Definitions are `async def acquire()` and `def release()`.
>>> Likewise user is expected to call `await .acquire()` and
>>> `.release()`.
>>>
>>> This is user-facing documentation, IMO it should be clearer.
>>> Although there are examples for this specific case, I'm concerned
>>> with
>>> general documentation best practice.
>>>
>>> Should this example read, e.g.:
>>> * two methods, `async acquire()` and `release()`
>>> or perhaps
>>> * two methods, used `await x.acquire()` and `x.release()`
>>> or something else?
>>>
>>> If there's a good example already Python docs or in some 3rd party
>>> docs, please tell.
>>>
>>> Likewise, should there be marks on iterators? async generators?
>>> things
>>> that ought to be used as context managers?
>>>
>>> Cheers,
>>> d.
>>> ___
>>> Async-sig mailing list
>>> Async-sig@python.org
>>> https://mail.python.org/mailman/listinfo/async-sig
>>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>>
>> --
>> Thanks,
>> Andrew Svetlov
>> ___
>> Async-sig mailing list
>> Async-sig@python.org
>> https://mail.python.org/mailman/listinfo/async-sig
>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>
>
> 

Re: [Async-sig] async documentation methods

[Alex Grönholm]

I'm somewhat reluctant to send them any PRs anymore since I sent them a 
couple of one liner fixes (with tests) which took around 5 months to get 
merged in spite of me repeatedly reminding them on the Google group.



Nathaniel Smith kirjoitti 04.07.2017 klo 10:55:

On Mon, Jul 3, 2017 at 11:49 PM, Alex Grönholm  wrote:

The real question is: why doesn't vanilla Sphinx have any kind of support
for async functions which have been part of the language for quite a while?

Because no-one's sent them a PR, I assume. They're pretty swamped AFAICT.

One of the maintainers has at least expressed interest in integrating
something like sphinxcontrib-trio if someone does the work:
https://github.com/sphinx-doc/sphinx/issues/3743

-n



___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

Re: [Async-sig] async documentation methods

[Nathaniel Smith]

On Mon, Jul 3, 2017 at 11:49 PM, Alex Grönholm  wrote:
> The real question is: why doesn't vanilla Sphinx have any kind of support
> for async functions which have been part of the language for quite a while?

Because no-one's sent them a PR, I assume. They're pretty swamped AFAICT.

One of the maintainers has at least expressed interest in integrating
something like sphinxcontrib-trio if someone does the work:
https://github.com/sphinx-doc/sphinx/issues/3743

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

Re: [Async-sig] async testing question

[Alex Grönholm]

For asyncio, you can write your test functions as coroutines if you use 
pytest-asyncio. You can even write test fixtures using coroutines. 
Mocking coroutine functions can be done using asynctest, although I've 
found that library a bit buggy.



Chris Jerdonek kirjoitti 02.07.2017 klo 00:00:

On Sat, Jul 1, 2017 at 1:42 PM, Nathaniel Smith  wrote:

On Jul 1, 2017 3:11 AM, "Chris Jerdonek"  wrote:
Is there a way to write a test case to check that task.cancel() would
behave correctly if, say, do_things() is waiting at the line
do_more()?

One possibility for handling this case with a minimum of mocking would be to
hook do_more so that it calls task.cancel and then calls the regular
do_more.

Beyond that it depends on what the actual functions are, I guess. If do_more
naturally blocks under some conditions then you might be able to set up
those conditions and then call cancel. Or you could try experimenting with
tests that call sleep(0) a fixed number of times before issuing the cancel,
and repeat with different iteration counts to find different cancel points.

Thanks, Nathaniel. The following would be overkill in my case, but
your suggestion makes me wonder if it would make sense for there to be
testing tools that have functions to do things like "run the event
loop until  is at ." Do such things
exist? This is a little bit related to what Dima was saying about
tools.

--Chris
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/


___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

Re: [Async-sig] async documentation methods

[Alex Grönholm]

Yeah, but that doesn't answer my question :)


Chris Jerdonek kirjoitti 04.07.2017 klo 10:02:

On Mon, Jul 3, 2017 at 11:49 PM, Alex Grönholm  wrote:

The real question is: why doesn't vanilla Sphinx have any kind of support
for async functions which have been part of the language for quite a while?

It looks like this is the issue (which Brett filed in Nov. 2015):
https://github.com/sphinx-doc/sphinx/issues/2105

--Chris




Nathaniel Smith kirjoitti 01.07.2017 klo 13:35:

If we're citing curio and sphinxcontrib-asyncio I guess I'll also
mention sphinxcontrib-trio [1], which was inspired by both of them
(and isn't in any way specific to trio). I don't know if the python
docs can use third-party sphinx extensions, though, and it is a bit
opinionated (in particular it calls async functions async functions
instead of coroutines).

For the original text, I'd probably write something like::

 You acquire a lock by calling ``await lock.acquire()``, and release
it with ``lock.release()``.

-n

[1] https://sphinxcontrib-trio.readthedocs.io/en/latest/

On Fri, Jun 30, 2017 at 8:31 AM, Brett Cannon  wrote:

Curio uses `.. asyncfunction:: acquire` and it renders as `await
acquire()`
at least in the function definition.

On Fri, 30 Jun 2017 at 03:36 Andrew Svetlov 
wrote:

I like "two methods, `async acquire()` and `release()`"

Regarding to extra markups -- I created sphinxcontrib-asyncio [1]
library
for it. Hmm, README is pretty empty but we do use the library for
documenting aio-libs and aiohttp [2] itself

We use ".. comethod:: connect(request)" for method and "cofunction" for
top level functions.

Additional markup for methods that could be used as async context
managers:

 .. comethod:: delete(url, **kwargs)
:async-with:
:coroutine:

and `:async-for:` for async iterators.


1. https://github.com/aio-libs/sphinxcontrib-asyncio
2. https://github.com/aio-libs/aiohttp

On Fri, Jun 30, 2017 at 1:11 PM Dima Tisnek  wrote:

Hi all,

I'm working to improve async docs, and I wonder if/how async methods
ought to be marked in the documentation, for example
library/async-sync.rst:

""" ... It [lock] has two basic methods, `acquire()` and `release()`.
...
"""

In fact, these methods are not symmetric, the earlier is asynchronous
and the latter synchronous:

Definitions are `async def acquire()` and `def release()`.
Likewise user is expected to call `await .acquire()` and `.release()`.

This is user-facing documentation, IMO it should be clearer.
Although there are examples for this specific case, I'm concerned with
general documentation best practice.

Should this example read, e.g.:
* two methods, `async acquire()` and `release()`
or perhaps
* two methods, used `await x.acquire()` and `x.release()`
or something else?

If there's a good example already Python docs or in some 3rd party
docs, please tell.

Likewise, should there be marks on iterators? async generators? things
that ought to be used as context managers?

Cheers,
d.
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

--
Thanks,
Andrew Svetlov
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/


___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/




___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/


___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/