On Thu, Aug 31, 2023 at 3:25 PM Dom Grigonis <dom.grigo...@gmail.com> wrote:
> Hi everyone, > > I am working with shared arrays and their slices. And trying to subclass > ndarray in such way so that they remap to memory on unpickling. I am using > package SharedArray, which doesn’t micro-manage memory locations, but > rather stores the whole map via shm using unique name. One issue I ran into > is that when I pickle & unpickle slice of memory mapped array, I need to > store the spec of the subset so that I can retrieve the same portion. > > The issue I am working on has been briefly discussed on stack: > https://stackoverflow.com/questions/12421770/find-location-of-slice-in-numpy-array. > Although I am not sure if it is the exact same one. > > So, in short, is there a way to determine location of a slice in original > array. Ideally, it would be indexing which works for any non-copy > slice/subset possible. > If you chase the `.base` chain all the way down, you get to a `SharedArray.map_owner` object with its `.addr` attribute giving the memory address that's assigned to it in the current process. This will be the same address that's in the `'data'` key of that first `ndarray` returned from `SharedArray.create()` that you are making your views from. In your view `ndarray` (which may be a view of views, with slices, transposes, reshapes, and arbitrary `stride_tricks` manipulations in between). If you store the difference between the view `ndarray`'s `'data'` address from the `map_owner.addr` address, I think that's all you need to recreate the array in a different process which gets assigned a different memory address for the same `shm` name. Just add that offset to the `map_owner.addr` and restore the rest of the information in `__array_interface__`, and I think you should be good to go. I don't think you'll need to infer or represent the precise path of Python-level operations (slices, transposes, reshapes, etc.) to which it got to that point. -- Robert Kern
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
