Thank you Pierre for writing this. I wanted to say this for a while, but I am too busy these days to do it myself.
Guys, please don’t miss what he has to say: > On Aug 31, 2017, at 5:49 PM, Pierre Habouzit via swift-evolution > <swift-evolution@swift.org> wrote: > > On real systems going wide concurrently (and I will not call this parallelism > on purpose, parallelism is about doing the same compute on different CPUs) is > not often a win provided the thing you're doing is complex enough. > The reason for it is that most of these subsystems, and loadWebResource is a > perfect example of this, use constrained resources that use locks and the > like. > > In practice, you would really want all resources that you load from the > internet to be handled from the same execution context (whether it's a > dispatch queue, a thread or a CFRunloop is absolutely not relevant), because > it's very likely that you'll end up contending concurrent executions of these > loads through locks and the various shared resources in use (like your NIC > and the networking stack). > > We've learned over the last 10 years of working on a massively concurrent > system because of libdispatch making it too easy (some will say thanks to, > but these aren't the one who need to cope with that it does to the OS :P), is > that going wide by default is a design mistake and is very difficult to > optimize when it goes bad. > > The right design is to have silos of execution contexts that are very few, as > few as you can, and parallelize (and here I mean parallelize) the operations > that benefit from using several compute units when you have proved that you > do scale linearly (and scaling linearly is VERY hard, even for low values of > NCPU as soon as what you do is complex enough) > > To me (but I'm catching up with the thread and maybe there have been > refinements in that regard already), the loseness of the "execution context" > that will run your completion handler here is a liability for the > implementation of the runtime. Go for example is not doing that well, for the > longest time the environment they recommended had a very small amount of > physical threads because of scalability concerns around their goroutines > scheduler. I'm mentioning go as an example of a language which is > implementing CSP as a core concept, and has had (I'm not quite sure where > they stand these days) quite a lot of trouble scaling the implementation. > > > IMSHO dispatch_get_global_queue() is in practice on of the worst thing that > the dispatch API provides, because despite all the best efforts of the > runtime, there aren't enough information at runtime about your > operations/actors/... to understand what your intent is and optimize for it. > Which means that the language should make sure that (1) the anti-pattern is > not the default behavior and (2) the interface provides a way to give and > propagate the hints (dependency relationships, ordering, execution context, > priorities, ...) the runtime will potentially need upfront.
_______________________________________________ swift-evolution mailing list swift-evolution@swift.org https://lists.swift.org/mailman/listinfo/swift-evolution
