Thank you, Mark, for starting this discussion. I, too, found myself simply accepting that channels was the right way to go, despite having the same questions you do. I realize this shouldn't be, so I've chimed in on some of your comments.
> On May 5, 2016, at 2:34 PM, Mark Lavin <markdla...@gmail.com> wrote: > > [snip] > > The Channel API is built more around a simple queue/list rather than a full > messaging layer. [snip] Kombu supports [snip]. The API was purposefully limited, because channels shouldn't need all those capabilities. All this is spelled out in the documentation, which I know you already understand because you've mentioned it elsewhere. I think that the choice to use a more limited API makes sense, though that doesn't necessarily mean that it is the right choice. > > [snip description of architecture] First off, the concerns you mention make a lot of sense to me, and I've been thinking along the same lines. I've been considering if having an alternative to Daphne that only used channels for websockets, but used WSGI for everything else. Or some alternative split where some requests would be ASGI and some WSGI. I've tested a bit the latency overhead that using channels adds (on my local machine even), and it's not insignificant. I agree that finding a solution that doesn't so drastically slow down the requests that we've already worked hard to optimize is important. I'm not yet sure the right way to do that. As far as scaling, it is apparent to me that it will be very important to have the workers split out, in a similar way to how we have different celery instances processing different queues. This allows us to scale those queues separately. While it doesn't appear to exist in the current implementation, the channel names are obviously suited to such a split, and I'd expect channels to grow the feature of selecting which channels a worker should be processing (forgive me if I've just missed this capability, Andrew). > > [[ comments on how this makes deployment harder ]] ASGI is definitely more complex that WSGI. It's this complexity that gives it power. However, to the best of my knowledge, there's not a push to be dropping WSGI. If you're doing a simple request/response site, then you don't need the complexity, and you probably should be using WSGI. However, if you need it, having ASGI standardized in Django will help the community build on the power that it brings. > > Channels claims to have a better zero-downtime deployment story. However, in > practice I’m not convinced that will be true. [snip] I've been concerned about this as well. On Heroku my web dynos don't go down, because the new ones are booted up while the old ones are running, and then a switch is flipped to have the router use the new dynos. Worker dynos, however, do get shut down. Daphne won't be enough to keep my site functioning. This is another reason I was thinking of a hybrid WSGI/ASGI server. > > There is an idea floating around of using Channels for background jobs/Celery > replacement. It is not/should not be. [snip reasons] It's not a Celery replacement. However, this simple interface may be good enough for many things. Anything that doesn't use celery's `acks_late` is a candidate, because in those cases even Celery doesn't guarantee delivery, and ASGI is a simpler interface than the powerful, glorious behemoth that is Celery. There's an idea that something like Celery could be built on top of it. That may or may not be a good idea, since Celery uses native protocol features of AMQP to make things work well, and those may not be available or easy to replicate accurately with ASGI. I'll be sticking with Celery for all of those workloads, personally, at least for the foreseeable future. > > [snip] locks everyone into using Redis. Thankfully, I know you're wrong about this. Channel layers can be built for other things, but Redis is a natural fit, so that's what he's written. I expect we'll see other channel layers for queues like AMQP before too long. > > I see literally no advantage to pushing all HTTP requests and responses > through Redis. It seems like a bad idea to push _all_ HTTP requests through Redis given the latency it adds, but long-running requests can still be a good idea for this case, because it separates the HTTP interface from the long-running code. This can be good, if used carefully. > What this does enable is that you can continue to write synchronous code. To > me that’s based around some idea that async code is too hard for the average > Django dev to write or understand. Or that nothing can be done to make parts > of Django play nicer with existing async frameworks which I also don’t > believe is true. Python 3.4 makes writing async Python pretty elegant and > async/await in 3.5 makes that even better. Async code is annoying, at best. It can be done, and it's getting much more approachable with async/await, etc. But even when you've done all that, there's stuff that, for many reasons, either cannot be written async (using a non-async library), or isn't IO-bound and async could actually _hurt_ the performance. ASGI doesn't force you to write anything synchronous _or_ asynchronous, and that's part of the beauty: it doesn't care. > > Thanks for taking the time to read this all the way through and I welcome any > feedback. I hope my comments have been a help for this discussion. I'll be giving a talk on Channels at the local Python user group later this month, so this is of particular timely importance to me. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/88339232-C9C6-4909-BBDC-60A3AFDB59EB%40ryanhiebert.com. For more options, visit https://groups.google.com/d/optout.
