How is this a different problem than the cache coherency problem?
https://en.wikipedia.org/wiki/Cache_coherence

Perhaps that's an unhelpful abstraction?

This hasn't gone anywhere:
https://en.wikipedia.org/wiki/Distributed_shared_memory#Directory_memory_coherence

Here's a great comparison chart for message passing vs distributed shared
memory:
https://en.wikipedia.org/wiki/Distributed_shared_memory#Message_Passing_vs._DSM

Could there be a multiprocessing.MemoryPool that tracks allocations,
refcounts, and also locks?

A combined approach might have an IPC channel/stream/source/sinks for
messages that instruct workers to invalidate/re-fetch object_id/references,
but consistency and ordering result in the same issues encountered with the
cache coherence problem.

Then, what is the best way to enqueue changes to shared global state (in
shared memory on one node, in this instance)?

(... "Ask HN: Learning about distributed systems?"
https://news.ycombinator.com/item?id=23931730 )

A solution for this could help accelerate dask and dask.distributed (which
already address many parallel issues in multiprocess and distributed
systems in pure python)
"Accelerate intra-node IPC with shared memory"
https://github.com/dask/dask/issues/6267



On Sun, Aug 2, 2020, 3:11 PM Eric V. Smith <e...@trueblade.com> wrote:

>
> On 8/2/2020 12:20 PM, Eric V. Smith wrote:
>
>
> On Sat, 1 Aug 2020 at 22:42, Eric V. Smith <e...@trueblade.com> wrote:
>
>> While they're immutable at the Python level, strings (and all other
>> objects) are mutated at the C level, due to reference count updates. You
>> need to consider this if you're sharing objects without locking or other
>> synchronization.
>>
>>
> This is interesting. What if you want to have a language that uses only
> immutable objects and garbage collection? Could smart pointers address this
> problem?
>
> Yes, garbage collection changes the picture entirely, with or without
> immutable objects. But the original topic was cross-processs shared memory,
> and I don't know of any cross-process aware garbage collectors that support
> shared memory. Although such a thing could easily exist without my
> knowledge.
>
> Note that I'm talking about putting Python objects into this shared
> memory. If that's not what people are contemplating, then my observations
> don't apply.
>
> Eric
> _______________________________________________
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/KT7URLYEMMHTSB7NIBLF3667OYOCF67I/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AAQ5PIGMKSUAGRDVIZ5H2ABGFXGHQ6O6/
Code of Conduct: http://python.org/psf/codeofconduct/

Reply via email to