Oh, I apologize for not following the links in the original question. Indeed in this case StreamReader is the solution.
I was answering a different question that gets asked a lot too (though perhaps the underlying use case is the same -- apparently StreamReader is not advertised well enough): "Why can't data_received() etc. be coroutines so they can use 'yield from'?" IRC you (Glyph) specifically didn't want that, because it would increase the burden for alternate event loop implementations. And your observation about multiple tasks is correct too, of course. But allowing data_received() to be a coroutine doesn't solve this by itself. I don't want the transport to pause or start buffering until that coroutine completes. So I think we should just note that protocols are a lower-level concept than StreamReader. (And more versatile -- that's usually how lower-level concepts work. :-) --Guido On Thu, Jan 23, 2014 at 12:28 AM, Glyph <[email protected]> wrote: > > On Jan 22, 2014, at 7:33 AM, Guido van Rossum <[email protected]> wrote: > > The solution is simple: write that logic as a separate method marked > with @coroutine, and fire it off in data_received() using async() (== > Task(), in this case). E.g. > > > Wouldn't that process each subsequent TCP segment in a parallel coroutine, > thereby with multiple segments of data stomping on each other if they're > yielding during processing? > > It seems to me that the real solution to do this (IntNReceiver) in the > coroutine style would be to use StreamReader, and then do something like > this: > > @coroutine > def read_one_string(): > prefix_length = struct.calcsize(prefix_format) > prefix = yield from reader.readexactly(prefix_length) > size = struct.unpack(prefix_format, prefix)[0] > return (yield from reader.readexactly(size)) > > If you don't want to use StreamReader, then you'll need to implement > something like what it does with feed_data, unblocking futures that are > waiting for data_received to be called: > > <https://code.google.com/p/tulip/source/browse/asyncio/streams.py#261>. > > -glyph > -- --Guido van Rossum (python.org/~guido)
