Re: [Async-sig] async/sync library reusage

[Alex Grönholm]

Yarko Tymciurak kirjoitti 09.06.2017 klo 11:49:


On Fri, Jun 9, 2017 at 3:05 AM Alex Grönholm > wrote:


Yarko Tymciurak kirjoitti 09.06.2017 klo 09:19:


On Fri, Jun 9, 2017 at 12:48 AM Nathaniel Smith > wrote:

On Thu, Jun 8, 2017 at 3:32 PM, manuel miranda
> wrote:
> Hello everyone,
>
> After using asyncio for a while, I'm struggling to find
information about
> how to support both synchronous and asynchronous use cases
for the same
> library.
>
> I.e. imagine you have a package for http requests and you
want to give the
> user the choice to use a synchronous or an asynchronous
interface. Right now
> the approach the community is following is creating
separate libraries one
> for each version. This is far from ideal for several
reasons, some I can
> think of:
>
> - Code duplication, most of the functionality is the same
in both libraries,
> only difference is the sync/async behaviors
> - Some new async libraries lack functionality compared to
their sync
> siblings. Others will introduce bugs that the sync version
already solved
> long ago, etc.
> - Different interfaces for the user for the same exact
functionality.
>
> In summary, in some cases it looks like reinventing the
wheel. So now comes
> the question, is there any documentation, guide on what
would be best
> practice supporting this kind of duality?

I would say that this is something that we as a community are
still
figuring out. I really like the Sans-IO approach, and it's a
really
valuable piece of the solution, but it doesn't solve the
whole problem
by itself - you still need to actually do I/O, and this means
things
like error handling and timeouts that aren't obviously a
natural fit
to the Sans-IO approach, and this means you may still have
some tricky
code that can end up duplicated. (Or maybe the Sans-IO
approach can be
extended to handle these things too?) There are active
discussions
happening in projects like urllib3 [1] and packaging [2]
about what
the best strategy to take is. And the options vary a lot
depending on
whether you need to support python 2 etc.

If you figure out a good approach I think everyone would be
interested
to hear it :-)


Just to leave this breadcrumb here - I've said this before, but
not thought in depth about it a lot, but pretty sure that in
something like Python4, async needs to become "first class
citizen," that is from the inside out, right in the bowels of the
repl loop.


Python 4 will be nothing more than the next minor release after
3.9. Because Guido hates double digit minor versions :)


If async is the default, and synchronous calls just a special
case (e.g. single-task async), then I'd expect two things (at
least): developers would have an easier time, make fewer mistakes
in async programming (the language would handle more), and
libraries would be unified as async & sync would be the same.

Are you suggesting the removal of the "await", "async with" and
"async for" structures? Those were added deliberately so
developers can spot the yield points in a coroutine function. Not
having them would give us something like gevent where you can
never tell when your task is going to be adjourned in favor of
another.


actually I was bot thinking of that...  but I was thinking of 
processing in the language, rather than a library...


In any case, I don't have answers, only a vision which keeps coming 
up.  My interest is not in providing "a solution", rather generating a 
reasoned discussion...
Then explain what you mean by making async a first class citizen in 
Python. In my mind it already is, by courtesy of having the "async def", 
"await" et al added to the language syntax itself and the inclusion of 
the asyncio module in the standard library. The only other thing that 
could've been done is to tie the language syntax to a single event loop 
implementation but that was deliberately left out.






Maybe there's something that would make this not make sense, but
I'd be really surprised. Larry's gil removal work intuitively
seems an enabler for this kind of (potential) work...

-y



-n

[1]
https://github.com/shazow/urllib3/pull/1068#issuecomment-294422348

[2] Here's the same API implemented three different ways:
Using deferreds: 

Re: [Async-sig] async/sync library reusage

[Yarko Tymciurak]

On Fri, Jun 9, 2017 at 3:05 AM Alex Grönholm 
wrote:

> Yarko Tymciurak kirjoitti 09.06.2017 klo 09:19:
>
>
> On Fri, Jun 9, 2017 at 12:48 AM Nathaniel Smith  wrote:
>
>> On Thu, Jun 8, 2017 at 3:32 PM, manuel miranda 
>> wrote:
>> > Hello everyone,
>> >
>> > After using asyncio for a while, I'm struggling to find information
>> about
>> > how to support both synchronous and asynchronous use cases for the same
>> > library.
>> >
>> > I.e. imagine you have a package for http requests and you want to give
>> the
>> > user the choice to use a synchronous or an asynchronous interface.
>> Right now
>> > the approach the community is following is creating separate libraries
>> one
>> > for each version. This is far from ideal for several reasons, some I can
>> > think of:
>> >
>> > - Code duplication, most of the functionality is the same in both
>> libraries,
>> > only difference is the sync/async behaviors
>> > - Some new async libraries lack functionality compared to their sync
>> > siblings. Others will introduce bugs that the sync version already
>> solved
>> > long ago, etc.
>> > - Different interfaces for the user for the same exact functionality.
>> >
>> > In summary, in some cases it looks like reinventing the wheel. So now
>> comes
>> > the question, is there any documentation, guide on what would be best
>> > practice supporting this kind of duality?
>>
>> I would say that this is something that we as a community are still
>> figuring out. I really like the Sans-IO approach, and it's a really
>> valuable piece of the solution, but it doesn't solve the whole problem
>> by itself - you still need to actually do I/O, and this means things
>> like error handling and timeouts that aren't obviously a natural fit
>> to the Sans-IO approach, and this means you may still have some tricky
>> code that can end up duplicated. (Or maybe the Sans-IO approach can be
>> extended to handle these things too?) There are active discussions
>> happening in projects like urllib3 [1] and packaging [2] about what
>> the best strategy to take is. And the options vary a lot depending on
>> whether you need to support python 2 etc.
>>
>> If you figure out a good approach I think everyone would be interested
>> to hear it :-)
>>
>
> Just to leave this breadcrumb here - I've said this before, but not
> thought in depth about it a lot, but pretty sure that in something like
> Python4, async needs to become "first class citizen," that is from the
> inside out, right in the bowels of the repl loop.
>
> Python 4 will be nothing more than the next minor release after 3.9.
> Because Guido hates double digit minor versions :)
>
> If async is the default, and synchronous calls just a special case (e.g.
> single-task async), then I'd expect two things (at least): developers would
> have an easier time, make fewer mistakes in async programming (the language
> would handle more), and libraries would be unified as async & sync would be
> the same.
>
> Are you suggesting the removal of the "await", "async with" and "async
> for" structures? Those were added deliberately so developers can spot the
> yield points in a coroutine function. Not having them would give us
> something like gevent where you can never tell when your task is going to
> be adjourned in favor of another.
>

actually I was bot thinking of that...  but I was thinking of processing in
the language, rather than a library...

In any case, I don't have answers, only a vision which keeps coming up.  My
interest is not in providing "a solution", rather generating a reasoned
discussion...



>
> Maybe there's something that would make this not make sense, but I'd be
> really surprised.  Larry's gil removal work intuitively seems an enabler
> for this kind of (potential) work...
>
> -y
>
>
>
>> -n
>>
>> [1] https://github.com/shazow/urllib3/pull/1068#issuecomment-294422348
>>
>> [2] Here's the same API implemented three different ways:
>> Using deferreds: https://github.com/pypa/packaging/pull/87
>> "traditional" sans-IO: https://github.com/pypa/packaging/pull/88
>> Using the "effect" library: https://github.com/dstufft/packaging/pull/1
>>
>> --
>> Nathaniel J. Smith -- https://vorpus.org
>> ___
>> Async-sig mailing list
>> Async-sig@python.org
>> https://mail.python.org/mailman/listinfo/async-sig
>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>
>
>
> ___
> Async-sig mailing 
> listAsync-sig@python.orghttps://mail.python.org/mailman/listinfo/async-sig
> Code of Conduct: https://www.python.org/psf/codeofconduct/
>
>
> ___
> Async-sig mailing list
> Async-sig@python.org
> https://mail.python.org/mailman/listinfo/async-sig
> Code of Conduct: https://www.python.org/psf/codeofconduct/
>
___
Async-sig mailing list

Re: [Async-sig] async/sync library reusage

[Nathaniel Smith]

On Thu, Jun 8, 2017 at 3:32 PM, manuel miranda  wrote:
> Hello everyone,
>
> After using asyncio for a while, I'm struggling to find information about
> how to support both synchronous and asynchronous use cases for the same
> library.
>
> I.e. imagine you have a package for http requests and you want to give the
> user the choice to use a synchronous or an asynchronous interface. Right now
> the approach the community is following is creating separate libraries one
> for each version. This is far from ideal for several reasons, some I can
> think of:
>
> - Code duplication, most of the functionality is the same in both libraries,
> only difference is the sync/async behaviors
> - Some new async libraries lack functionality compared to their sync
> siblings. Others will introduce bugs that the sync version already solved
> long ago, etc.
> - Different interfaces for the user for the same exact functionality.
>
> In summary, in some cases it looks like reinventing the wheel. So now comes
> the question, is there any documentation, guide on what would be best
> practice supporting this kind of duality?

I would say that this is something that we as a community are still
figuring out. I really like the Sans-IO approach, and it's a really
valuable piece of the solution, but it doesn't solve the whole problem
by itself - you still need to actually do I/O, and this means things
like error handling and timeouts that aren't obviously a natural fit
to the Sans-IO approach, and this means you may still have some tricky
code that can end up duplicated. (Or maybe the Sans-IO approach can be
extended to handle these things too?) There are active discussions
happening in projects like urllib3 [1] and packaging [2] about what
the best strategy to take is. And the options vary a lot depending on
whether you need to support python 2 etc.

If you figure out a good approach I think everyone would be interested
to hear it :-)

-n

[1] https://github.com/shazow/urllib3/pull/1068#issuecomment-294422348

[2] Here's the same API implemented three different ways:
Using deferreds: https://github.com/pypa/packaging/pull/87
"traditional" sans-IO: https://github.com/pypa/packaging/pull/88
Using the "effect" library: https://github.com/dstufft/packaging/pull/1

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/