How is this a different problem than the cache coherency problem? https://en.wikipedia.org/wiki/Cache_coherence
Perhaps that's an unhelpful abstraction? This hasn't gone anywhere: https://en.wikipedia.org/wiki/Distributed_shared_memory#Directory_memory_coherence Here's a great comparison chart for message passing vs distributed shared memory: https://en.wikipedia.org/wiki/Distributed_shared_memory#Message_Passing_vs._DSM Could there be a multiprocessing.MemoryPool that tracks allocations, refcounts, and also locks? A combined approach might have an IPC channel/stream/source/sinks for messages that instruct workers to invalidate/re-fetch object_id/references, but consistency and ordering result in the same issues encountered with the cache coherence problem. Then, what is the best way to enqueue changes to shared global state (in shared memory on one node, in this instance)? (... "Ask HN: Learning about distributed systems?" https://news.ycombinator.com/item?id=23931730 ) A solution for this could help accelerate dask and dask.distributed (which already address many parallel issues in multiprocess and distributed systems in pure python) "Accelerate intra-node IPC with shared memory" https://github.com/dask/dask/issues/6267 On Sun, Aug 2, 2020, 3:11 PM Eric V. Smith <e...@trueblade.com> wrote: > > On 8/2/2020 12:20 PM, Eric V. Smith wrote: > > > On Sat, 1 Aug 2020 at 22:42, Eric V. Smith <e...@trueblade.com> wrote: > >> While they're immutable at the Python level, strings (and all other >> objects) are mutated at the C level, due to reference count updates. You >> need to consider this if you're sharing objects without locking or other >> synchronization. >> >> > This is interesting. What if you want to have a language that uses only > immutable objects and garbage collection? Could smart pointers address this > problem? > > Yes, garbage collection changes the picture entirely, with or without > immutable objects. But the original topic was cross-processs shared memory, > and I don't know of any cross-process aware garbage collectors that support > shared memory. Although such a thing could easily exist without my > knowledge. > > Note that I'm talking about putting Python objects into this shared > memory. If that's not what people are contemplating, then my observations > don't apply. > > Eric > _______________________________________________ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/KT7URLYEMMHTSB7NIBLF3667OYOCF67I/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/AAQ5PIGMKSUAGRDVIZ5H2ABGFXGHQ6O6/ Code of Conduct: http://python.org/psf/codeofconduct/