I'm not convinced either way about whether putting this in core will help mature it and fix bugs more quickly or not. I don't have any sense of how the code might change after we merge it, but things get more complicated if we start selectively backporting somes fixes for Django's monthly bug fix releases. If channels continues to live externally, I assume there won't be the complication of a master vs. stable branch and that releases could happen more often. If this feature is exciting enough, whether or not its in Django itself shouldn't make too much of a difference, should it? Also, if we do include this in 1.10 and then start paying people using the Mozilla money, that might result in master changing even more aggressively compared to the stable branch at which point having a 1.10 release that's X months old in terms of the latest features might not be all that valuable.
Tell me if I'm wrong, but I get the sense that you are somewhat burned out maintaining this project on your own and you feel that merging this to Django will help offload your burden and attract other contributors. If that's the case, there's a possible solution in using some of the Mozilla money to try to get help in more of the code review/maintenance tasks. If we have an accepted DEP and plans to merge it to core in 1.11, I would try to get more involved with those projects too. I don't see a problem in continuing to refine this externally even if that means it lands in an LTS. I sure would feel less overwhelmed if I didn't have to review the big patch up against the alpha deadline and/or if this threatens to delay the 1.10 release. On Friday, May 6, 2016 at 1:16:20 AM UTC-4, Andrew Godwin wrote: > > > > On Thu, May 5, 2016 at 9:28 PM, Anssi Kääriäinen <akaa...@gmail.com > <javascript:>> wrote: > >> On Thursday, May 5, 2016, Andrew Godwin <and...@aeracode.org >> <javascript:>> wrote: >>> >>> Do you have a link to the presentation about them removing it? >>> >> >> https://youtu.be/839rskyJaro around 34 minutes and onwards, another >> version is https://youtu.be/2dG5HeM7MvA at 24 minutes. >> > > Summarising their issues here for future people to save on video watching: > > - Large response bodies/streamed/chunked bodies are hard to serialise into > Redis > - Working out the order to restart things in is tricker than standard HTTP > loadbalancing > - Healthchecking the Rails process was difficult as they didn't listen on > HTTP (just to Redis) > - Hard to do rate-limiting and monitoring as it lacked context > - HAProxy got a lot more featureful and grew the features they needed > (hot-swapping of backends), and was a lot more battle-tested > > In general, it seems they were mainly using it to achieve HA, and it > didn't quite pull through for them on that front. Some of these issues also > apply to Channels, some less so as it's not designed as a HA solution and > just being used for that. > > In particular, they were doing a lot more to keep requests around, whereas > Channels will drop them if it's unsure what's going on, which gives it a > lot more leeway (but makes it less "perfect" at HA). > > >> >> They were tackling a bit different problem, so their lessons might not >> apply. Most importantly they weren't aiming for websockets at all, just for >> high availability and throughput for normal HTTP traffic. On the other >> hand, the architecture of broxy itself is pretty much the same as Daphne, >> and they felt they had problems with it. >> > > I need to start writing up my preferred large-scale deployment > architecture for Channels more, which is "small clusters of interfaces and > workers with an external loadbalancer across all clusters"; all of the > stories I've heard, this included, the common theme is trying to push all > of the site traffic through just the one bus and having great issues when > it keels over under unexpected load or machine failure. > > I'm also starting to think we should have a local shared-memory channel > layer that only works between processes on the one machine, so one could in > theory just run clusters like this on decently sized multicore boxes; that > would make a lot of people happier who don't want to have to run Redis in > production but still want the multi-process behaviour that Channels gives. > Running Channels would then just mean two processes, Daphne and workers. > > >> >> The way we have approached recent feature additions is that we let them >> prove themselves outside of core. I think the crucial question is why >> Channels can't do this, that is why can't we wait some time and let actual >> production deployments prove the design? >> >> I know South is an example of why some features have a hard time living >> outside core. But DRF is an example that a feature crucial to modern web >> development can work very well outside of core. >> >> > Channels sits somewhere between these two; it's not quite as deeply > involved in hacking around core parts of Django as South was, but it's a > bit deeper than DRF gets since it technically inserts itself under the > entire view/URL routing system and sits alongside the WSGI handling code. > > My reason for pushing to get it in is that a lot of the utility of > Channels is not in the code itself but what it enables; most of the things > I would like to build can sit on top as third-party packages and get pulled > in later if they make sense (e.g. a nice set of generic websocket > consumers, or a high-level model-changes streaming solution). > > If it lives as some third-party package for another 8 months, it's a lot > harder to sound that rally cry to build up the necessary community of > layers on top of it that complete the ecosystem; plus, there's the obvious > human factor that it will likely place the same workload on a much smaller > set of main contributors. > > The situation is helped by the fact that the code that will likely need > most polish and iteration - Daphne and asgi_redis - will still live outside > Django's core codebase, giving us some more flexibility in how we handle > changes (though I would like to still provide similar levels of stability > and security assurances on them) - and that we'll be labelling it as > "provisional" in 1.10 (similar to how PEP 411 describes it for Python), > meaning we get one last chance to tweak things in 1.11 before we lock the > API in for good. > > Andrew > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ee9ed1e8-4030-4696-ac0f-536d2d4cb097%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
