Thanks Andrew! I usually ignore the AGI list these days, but today I happened to tune in, and happened upon this post of your regarding the relation btw immutability and massive parallelism, from which I actually learned something ;-)
-- Ben G On Fri, Aug 14, 2015 at 4:17 AM, J. Andrew Rogers <[email protected]> wrote: > >> On Aug 13, 2015, at 12:04 PM, Juan Carlos Kuri Pinto <[email protected]> >> wrote: >> >> Regarding massive parallelism, Haskell is naturally parallelizable because >> it is 100% pure. That is, Haskell doesn't have mutating state. You never >> modify memory locations. You only create new constants and let the garbage >> collector remove them when they are out of scope. > > > This is an incorrect conception of massive parallelism. The above model > creates a vast amount of shared, mutable state that is being ignored. It is > the reason languages like Haskell are poor for massive parallelism in > practice. > > What you describe above is focused on data access parallelism. Unfortunately, > topological parallelism dominates scalability in massively parallel systems. > Computing models that rely on immutable memory are generally gaining that > data access parallelism by reducing the efficiency of topological parallelism. > > Memory locations are not the only mutable state in real computing systems. > The wires that move data *between* memory locations are also a dynamic shared > writable resource, and a relatively scarce one at that. Computing models that > effect mutability by copying state greatly increase the contention for this > resource and exhibit strong sublinearity as the parallelism increases. > > > The most efficiently parallelizable computing models use non-shared mutable > state. You basically move functions (which are naturally immutable and > compact) to the process that owns the state you want to mutate. This allows > very efficient topological parallelism, and since the state is almost never > shared between processes, most of the problems immutable memory is designed > to solve do not apply. It is actually an elegant if somewhat unintuitive > model in implementation. > > Even on single machines, this model typically offers an integer factor > throughput improvement over lock-free multithreading models of parallelism > and concurrency (or functional models). > > Cheers, > Andrew > > ------------------------------------------- > AGI > Archives: https://www.listbox.com/member/archive/303/=now > RSS Feed: https://www.listbox.com/member/archive/rss/303/212726-deec6279 > Modify Your Subscription: https://www.listbox.com/member/?&; > Powered by Listbox: http://www.listbox.com -- Ben Goertzel, PhD http://goertzel.org "The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." -- George Bernard Shaw ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
