Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under
On Sat, Aug 18, 2018 at 2:44 PM, Chris Jerdonek wrote: > The kind of alternative I had in mind for a neutral location is > setting an attribute with an agreed upon name on a module in the > standard lib, perhaps something like > `contextvars.current_async_library_cvar` to use your naming. This is > analogous to agreeing on a file name to store information of a certain > kind in a repository root, like .travis.yml, package.json, or > pyproject.toml. It's light-weight and doesn't require any > infrastructure or tying to a particular package on PyPI. Yeah, it'd be possible. I guess it just didn't seem worth the extra complication. -n -- Nathaniel J. Smith -- https://vorpus.org ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under
On Sat, Aug 18, 2018 at 2:13 PM, Nathaniel Smith wrote: > On Sat, Aug 18, 2018, 01:22 Chris Jerdonek wrote: >> >> Also, just to be clear, I think the idea of a library to sniff this >> information is great. >> >> It's just the location of where this information is being stored that >> I'm focusing in on and asking about. It seems like it should be in a >> "neutral" location, and in particular different from / decoupled from >> the library above. >> >> One advantage of decoupling the sniffing library from the registry >> location is that it would allow for someone to write an improved >> library in the future (say "detectio") without forever locking in / >> requiring the old library to be installed. > > > Yeah, we discussed this some when initially designing it, because we were > worried about the lock in issue too. Obviously if you want to do anything > you need to make some decisions, but to reduce the risk here we > intentionally kept sniffio as minimal and unopinionated as possible: > https://github.com/python-trio/sniffio/issues/1#issuecomment-408812146 > > Have you seen the code? It's not *entirely* trivial – certainly big enough > to contain a bug or two – but it's tiny and only contains what it absolutely > needs to work: a contextvar (native on 3.7, or via Yury's backport library > on earlier pythons), and a fallback for detecting asyncio (since as a stdlib > module it can't really work any other way). > > https://github.com/python-trio/sniffio/blob/master/sniffio/_impl.py Yes, I had looked at the code a few times and agree it's simple and useful. (But it could always grow.) I looked in the linked issue and didn't see any discussion of the registry location though, which is the part I was interested in. The kind of alternative I had in mind for a neutral location is setting an attribute with an agreed upon name on a module in the standard lib, perhaps something like `contextvars.current_async_library_cvar` to use your naming. This is analogous to agreeing on a file name to store information of a certain kind in a repository root, like .travis.yml, package.json, or pyproject.toml. It's light-weight and doesn't require any infrastructure or tying to a particular package on PyPI. --Chris ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under
On Fri, Aug 17, 2018 at 11:44 PM, Chris Jerdonek wrote: > On Fri, Aug 17, 2018 at 12:50 PM, Nathaniel Smith wrote: >> On Fri, Aug 17, 2018, 12:12 Chris Jerdonek wrote: >>> >>> Did you also think about whether it would be possible for a library to >>> advertise itself without having to depend on a third-party library >>> (e.g. using some sort of convention)? That would permit a less >>> "centralized" approach. >> >> >> What kind of convention do you have in mind? > > Good question. I don't claim to know the answer which is why I asked > if you had thought about it. The *kind* of thing I had in mind was to > set a variable with an agreed-upon name and value on an agreed-upon > module in the standard library -- though I agree that seems hacky as > stated. > > It does seem to me like something that should (already?) have a > general solution. What other ways does Python let things register or > "announce" themselves? Well, you could register an entry in an some global dict under an agreed-on key, like, say, sys.modules["sniffio"]. Of course, whenever you're mutating a global object like this you should worry about name collisions, but fortunately that particular dict has a good convention for reserving names. In fact there's a whole web service called "PyPI" devoted to managing those registrations! And then you might as well upload the code for accessing that variable to the web service, so everyone doesn't have to copy/paste it into their programs... ;-) Now that packaging works reliably, it's a pretty good solution for this kind of thing IMHO. -n -- Nathaniel J. Smith -- https://vorpus.org ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under
On Fri, Aug 17, 2018 at 12:50 PM, Nathaniel Smith wrote: > On Fri, Aug 17, 2018, 12:12 Chris Jerdonek wrote: >> >> Did you also think about whether it would be possible for a library to >> advertise itself without having to depend on a third-party library >> (e.g. using some sort of convention)? That would permit a less >> "centralized" approach. > > > What kind of convention do you have in mind? Good question. I don't claim to know the answer which is why I asked if you had thought about it. The *kind* of thing I had in mind was to set a variable with an agreed-upon name and value on an agreed-upon module in the standard library -- though I agree that seems hacky as stated. It does seem to me like something that should (already?) have a general solution. What other ways does Python let things register or "announce" themselves? --Chris > > The problem with a convention AFAICT is that you need some shared agreement > about where to rendezvous. That's basically all the sniffio library is: a > shared, neutral place for libraries to advertise themselves. (Plus a > fallback for detecting asyncio, because stdlib libraries have special > constraints.) > > -n ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under
On Fri, Aug 17, 2018, 09:09 Alex Grönholm wrote:
> This was my approach:
>
> def _detect_running_asynclib() -> str:
> if 'trio' in sys.modules:
> from trio.hazmat import current_trio_token
> try:
> current_trio_token()
> except RuntimeError:
> pass
> else:
> return 'trio'
>
> if 'curio' in sys.modules:
> from curio.meta import curio_running
> if curio_running():
> return 'curio'
>
> if 'asyncio' in sys.modules:
> from .backends.asyncio import get_running_loop
> if get_running_loop() is not None:
> return 'asyncio'
>
> raise LookupError('Cannot find any running async event loop')
>
>
> Is there something wrong with this?
>
If you're using trio-asyncio, then you can have both trio-flavored
coroutines and asyncio-flavored coroutines running in the same thread. And
in particular, the trio and asyncio tests you do above will both return
true at the same time, even though at any given moment only you can only
'await' one kind of async function or the other.
Twisted running on the asyncio reactor has a similar situation.
-n
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under
If I'm reading the docs correctly, it looks like an async library has to depend on sniffio in order to be detected by sniffio: https://sniffio.readthedocs.io/en/latest/#adding-support-to-a-new-async-library Did you also think about whether it would be possible for a library to advertise itself without having to depend on a third-party library (e.g. using some sort of convention)? That would permit a less "centralized" approach. --Chris On Thu, Aug 16, 2018 at 12:01 AM, Nathaniel Smith wrote: > Hi all, > > A number of people are working on packages that support multiple async > backends (e.g., asyncio + trio, or trio + curio, or trio + twisted, > ...). So then the question arises... how can I figure out which async > library my user is actually using? > > Answer: install sniffio, and then call > sniffio.current_async_library(), and it tells you. > > Well, right now it only works for trio and asyncio, but if you > maintain an async library and you want to make it easier for packages > to detect you, then it's easy to add support – see the manual. We > considered various clever things, but ultimately decided that the best > approach was to use a ContextVar and make it the coroutine runner's > responsibility to advertise which async flavor it uses. In particular, > this approach works even for hybrid programs that are using multiple > coroutine runners in the same loop, like a Twisted program with > asyncio-flavored and twisted-flavored coroutines in the same thread, > or a Trio program using trio-asyncio to run both asyncio-flavored and > trio-flavored coroutines in the same thread. > > Github: https://github.com/python-trio/sniffio > Manual: https://sniffio.readthedocs.io/ > PyPI: https://pypi.org/p/sniffio > > -n > > -- > Nathaniel J. Smith -- https://vorpus.org > ___ > Async-sig mailing list > Async-sig@python.org > https://mail.python.org/mailman/listinfo/async-sig > Code of Conduct: https://www.python.org/psf/codeofconduct/ ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under
If you look at the code more carefully, you'll see that I'm not merely
checking what's been imported. In each case I'm asking the relevant
framework if they're running an event loop *in the current thread*.
pe, 2018-08-17 kello 09:26 -0700, Brett Cannon kirjoitti:
> Importation does not equate to execution. I.e. since I could have
> multiple event loops running at once that means what's in sys.modules
> can't tell me what event loop I'm currently interacting with.
>
> On Fri, 17 Aug 2018 at 09:09 Alex Grönholm
> wrote:
> > This was my approach:
> >
> > def _detect_running_asynclib() -> str:
> > if 'trio' in sys.modules:
> > from trio.hazmat import current_trio_token
> > try:
> > current_trio_token()
> > except RuntimeError:
> > pass
> > else:
> > return 'trio'
> >
> > if 'curio' in sys.modules:
> > from curio.meta import curio_running
> > if curio_running():
> > return 'curio'
> >
> > if 'asyncio' in sys.modules:
> > from .backends.asyncio import get_running_loop
> > if get_running_loop() is not None:
> > return 'asyncio'
> >
> > raise LookupError('Cannot find any running async event loop')
> >
> > Is there something wrong with this?
> > to, 2018-08-16 kello 00:01 -0700, Nathaniel Smith kirjoitti:
> > > Hi all,
> > > A number of people are working on packages that support multiple
> > > asyncbackends (e.g., asyncio + trio, or trio + curio, or trio +
> > > twisted,...). So then the question arises... how can I figure out
> > > which asynclibrary my user is actually using?
> > > Answer: install sniffio, and then
> > > callsniffio.current_async_library(), and it tells you.
> > > Well, right now it only works for trio and asyncio, but if
> > > youmaintain an async library and you want to make it easier for
> > > packagesto detect you, then it's easy to add support – see the
> > > manual. Weconsidered various clever things, but ultimately
> > > decided that the bestapproach was to use a ContextVar and make it
> > > the coroutine runner'sresponsibility to advertise which async
> > > flavor it uses. In particular,this approach works even for hybrid
> > > programs that are using multiplecoroutine runners in the same
> > > loop, like a Twisted program withasyncio-flavored and twisted-
> > > flavored coroutines in the same thread,or a Trio program using
> > > trio-asyncio to run both asyncio-flavored andtrio-flavored
> > > coroutines in the same thread.
> > > Github: https://github.com/python-trio/sniffioManual:
> > > https://sniffio.readthedocs.io/PyPI: https://pypi.org/p/sniffio
> > > -n
> > >
> >
> > ___
> >
> > Async-sig mailing list
> >
> > Async-sig@python.org
> >
> > https://mail.python.org/mailman/listinfo/async-sig
> >
> > Code of Conduct: https://www.python.org/psf/codeofconduct/
> >
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under
Importation does not equate to execution. I.e. since I could have multiple
event loops running at once that means what's in sys.modules can't tell me
what event loop I'm currently interacting with.
On Fri, 17 Aug 2018 at 09:09 Alex Grönholm wrote:
> This was my approach:
>
> def _detect_running_asynclib() -> str:
> if 'trio' in sys.modules:
> from trio.hazmat import current_trio_token
> try:
> current_trio_token()
> except RuntimeError:
> pass
> else:
> return 'trio'
>
> if 'curio' in sys.modules:
> from curio.meta import curio_running
> if curio_running():
> return 'curio'
>
> if 'asyncio' in sys.modules:
> from .backends.asyncio import get_running_loop
> if get_running_loop() is not None:
> return 'asyncio'
>
> raise LookupError('Cannot find any running async event loop')
>
>
> Is there something wrong with this?
>
> to, 2018-08-16 kello 00:01 -0700, Nathaniel Smith kirjoitti:
>
> Hi all,
>
>
> A number of people are working on packages that support multiple async
>
> backends (e.g., asyncio + trio, or trio + curio, or trio + twisted,
>
> ...). So then the question arises... how can I figure out which async
>
> library my user is actually using?
>
>
> Answer: install sniffio, and then call
>
> sniffio.current_async_library(), and it tells you.
>
>
> Well, right now it only works for trio and asyncio, but if you
>
> maintain an async library and you want to make it easier for packages
>
> to detect you, then it's easy to add support – see the manual. We
>
> considered various clever things, but ultimately decided that the best
>
> approach was to use a ContextVar and make it the coroutine runner's
>
> responsibility to advertise which async flavor it uses. In particular,
>
> this approach works even for hybrid programs that are using multiple
>
> coroutine runners in the same loop, like a Twisted program with
>
> asyncio-flavored and twisted-flavored coroutines in the same thread,
>
> or a Trio program using trio-asyncio to run both asyncio-flavored and
>
> trio-flavored coroutines in the same thread.
>
>
> Github: https://github.com/python-trio/sniffio
>
> Manual: https://sniffio.readthedocs.io/
>
> PyPI: https://pypi.org/p/sniffio
>
>
> -n
>
>
>
> ___
> Async-sig mailing list
> Async-sig@python.org
> https://mail.python.org/mailman/listinfo/async-sig
> Code of Conduct: https://www.python.org/psf/codeofconduct/
>
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under
This was my approach:
def _detect_running_asynclib() -> str:
if 'trio' in sys.modules:
from trio.hazmat import current_trio_token
try:
current_trio_token()
except RuntimeError:
pass
else:
return 'trio'
if 'curio' in sys.modules:
from curio.meta import curio_running
if curio_running():
return 'curio'
if 'asyncio' in sys.modules:
from .backends.asyncio import get_running_loop
if get_running_loop() is not None:
return 'asyncio'
raise LookupError('Cannot find any running async event loop')
Is there something wrong with this?
to, 2018-08-16 kello 00:01 -0700, Nathaniel Smith kirjoitti:
> Hi all,
> A number of people are working on packages that support multiple
> asyncbackends (e.g., asyncio + trio, or trio + curio, or trio +
> twisted,...). So then the question arises... how can I figure out
> which asynclibrary my user is actually using?
> Answer: install sniffio, and then
> callsniffio.current_async_library(), and it tells you.
> Well, right now it only works for trio and asyncio, but if
> youmaintain an async library and you want to make it easier for
> packagesto detect you, then it's easy to add support – see the
> manual. Weconsidered various clever things, but ultimately decided
> that the bestapproach was to use a ContextVar and make it the
> coroutine runner'sresponsibility to advertise which async flavor it
> uses. In particular,this approach works even for hybrid programs that
> are using multiplecoroutine runners in the same loop, like a Twisted
> program withasyncio-flavored and twisted-flavored coroutines in the
> same thread,or a Trio program using trio-asyncio to run both asyncio-
> flavored andtrio-flavored coroutines in the same thread.
> Github: https://github.com/python-trio/sniffioManual:
> https://sniffio.readthedocs.io/PyPI: https://pypi.org/p/sniffio
> -n
>
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under
Neat! On Thu, Aug 16, 2018 at 9:02 AM Nathaniel Smith wrote: > Hi all, > > A number of people are working on packages that support multiple async > backends (e.g., asyncio + trio, or trio + curio, or trio + twisted, > ...). So then the question arises... how can I figure out which async > library my user is actually using? > > Answer: install sniffio, and then call > sniffio.current_async_library(), and it tells you. > > Well, right now it only works for trio and asyncio, but if you > maintain an async library and you want to make it easier for packages > to detect you, then it's easy to add support – see the manual. We > considered various clever things, but ultimately decided that the best > approach was to use a ContextVar and make it the coroutine runner's > responsibility to advertise which async flavor it uses. In particular, > this approach works even for hybrid programs that are using multiple > coroutine runners in the same loop, like a Twisted program with > asyncio-flavored and twisted-flavored coroutines in the same thread, > or a Trio program using trio-asyncio to run both asyncio-flavored and > trio-flavored coroutines in the same thread. > > Github: https://github.com/python-trio/sniffio > Manual: https://sniffio.readthedocs.io/ > PyPI: https://pypi.org/p/sniffio > > -n > > -- > Nathaniel J. Smith -- https://vorpus.org > ___ > Async-sig mailing list > Async-sig@python.org > https://mail.python.org/mailman/listinfo/async-sig > Code of Conduct: https://www.python.org/psf/codeofconduct/ > -- --Guido (mobile) ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
[Async-sig] new library: sniffio – Sniff out which async library your code is running under
Hi all, A number of people are working on packages that support multiple async backends (e.g., asyncio + trio, or trio + curio, or trio + twisted, ...). So then the question arises... how can I figure out which async library my user is actually using? Answer: install sniffio, and then call sniffio.current_async_library(), and it tells you. Well, right now it only works for trio and asyncio, but if you maintain an async library and you want to make it easier for packages to detect you, then it's easy to add support – see the manual. We considered various clever things, but ultimately decided that the best approach was to use a ContextVar and make it the coroutine runner's responsibility to advertise which async flavor it uses. In particular, this approach works even for hybrid programs that are using multiple coroutine runners in the same loop, like a Twisted program with asyncio-flavored and twisted-flavored coroutines in the same thread, or a Trio program using trio-asyncio to run both asyncio-flavored and trio-flavored coroutines in the same thread. Github: https://github.com/python-trio/sniffio Manual: https://sniffio.readthedocs.io/ PyPI: https://pypi.org/p/sniffio -n -- Nathaniel J. Smith -- https://vorpus.org ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
