#33735: Add asynchronous responses for use with an ASGI server
--------------------------------+--------------------------------------
Reporter: florianvazelle | Owner: nobody
Type: Uncategorized | Status: new
Component: HTTP handling | Version: 4.0
Severity: Normal | Resolution:
Keywords: ASGI async | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+--------------------------------------
Description changed by florianvazelle:
Old description:
> This ticket follows the one I opened on
> [https://github.com/django/daphne/issues/413 django/daphne].
>
> ==== Initial Issue
>
> Using Daphne as a server for a Django application seems to cause a
> malfunction in streaming responses.
> When you pass an iterator into a response, the contents of the response
> are not retrieved chunk by chunk, but from a single block when the
> iterator has finished iterating.
> I have a minimal project that can reproduce the behavior, at
> [https://github.com/florianvazelle/minimal-daphne-stream florianvazelle
> /minimal-daphne-stream].
>
> ==== Solution
>
> After some research, in Daphne, we use Twisted with asyncioreactor. I
> managed to reproduce the bug in a minimal way:
> [https://gist.github.com/florianvazelle/c7d8c49fb005a72ed5962ee55a83b000
> Minimal example of an "ASGI-like" streaming server with Twisted].
> The streaming issue occurs when we call a blocking method in an iterator
> (io, sleep, request ...), which blocks the reactor. That's why the result
> is not done progressively.
> The reactor does not get control of execution back until the end of the
> iteration.
> To correct this behavior we need to add an asynchronous layer and use
> async / non-blocking alternatives.
>
> ==== Proposition
>
> In my minimal project, the view will become:
>
> {{{#!python
> import asyncio
>
> from django.http.response import StreamingHttpResponse
>
> async def iterable_content():
> for _ in range(5):
> await asyncio.sleep(1)
> print('Returning chunk')
> yield b'a' * 10000
>
> def test_stream_view(request):
> return StreamingHttpResponse(iterable_content())
> }}}
>
> But django does not handle asynchronous generators.
> Some works are already did in this tickets
> [https://code.djangoproject.com/ticket/32798 #32798
> (StreamingHttpResponse Raises SynchronousOnlyOperation in ASGI Server) –
> Django], but considering the performance, we could manage it differently
> here.
>
> I propose to add asynchronous responses, to be used in an async context,
> so with an ASGI server.
> I make a PoC, in a very basic way:
> [https://github.com/florianvazelle/django/commit/4a52f975fa39ae1f52e015a973f9d146fc1cb561
> florianvazelle/django].
New description:
This ticket follows the one I opened on
[https://github.com/django/daphne/issues/413 django/daphne].
==== Initial Issue
Using Daphne as a server for a Django application seems to cause a
malfunction in streaming responses.
When you pass an iterator into a response, the contents of the response
are not retrieved chunk by chunk, but from a single block when the
iterator has finished iterating.
I have a minimal project that can reproduce the behavior, at
[https://github.com/florianvazelle/minimal-daphne-stream florianvazelle
/minimal-daphne-stream].
==== Solution
After some research, in Daphne, we use Twisted with asyncioreactor. I
managed to reproduce the bug in a minimal way:
[https://gist.github.com/florianvazelle/c7d8c49fb005a72ed5962ee55a83b000
Minimal example of an "ASGI-like" streaming server with Twisted].
The streaming issue occurs when we call a blocking method in an iterator
(io, sleep, request ...), which blocks the reactor. That's why the result
is not done progressively.
The reactor does not get control of execution back until the end of the
iteration.
To correct this behavior we need to add an asynchronous layer and use
async / non-blocking alternatives.
==== Proposition
In my minimal project, the view will become:
{{{#!python
import asyncio
from django.http.response import StreamingHttpResponse
async def iterable_content():
for _ in range(5):
await asyncio.sleep(1)
print('Returning chunk')
yield b'a' * 10000
def test_stream_view(request):
return StreamingHttpResponse(iterable_content())
}}}
But django does not handle asynchronous generators.
Some works are already did in this tickets
[https://code.djangoproject.com/ticket/32798 #32798 (StreamingHttpResponse
Raises SynchronousOnlyOperation in ASGI Server) – Django], but considering
the performance, we could manage it differently here.
I propose to add asynchronous responses, to be used in an async context,
so with an ASGI server.
I make a PoC, in a very basic way:
[https://github.com/django/django/pull/15727 PoC Fixed #33735 -- Handle
async streaming response].
--
--
Ticket URL: <https://code.djangoproject.com/ticket/33735#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/01070180f0590f9b-d633afb7-06ee-4539-9d41-e4223349cb78-000000%40eu-central-1.amazonses.com.
