Arrow Buffers, memoryview, array.array Apache Arrow Buffers support zero-copy slicing:
> arrow::Buffer can be zero-copy sliced to permit Buffers to cheaply reference other Buffers, while preserving memory lifetime and clean parent-child relationships. > > There are many implementations of arrow::Buffer, but they all provide a standard interface: a data pointer and length. This is similar to Python’s built-in buffer protocol and memoryview objects. > > A Buffer can be created from any Python object implementing the buffer protocol by calling the py_buffer() function. https://arrow.apache.org/docs/python/memory.html#pyarrow-buffer https://docs.python.org/3/library/stdtypes.html#memoryview : > A memoryview has the notion of an element, which is the atomic memory unit handled by the originating object obj. For many simple types such as bytes and bytearray, an element is a single byte, but other types such as array.array may have bigger elements. > > len(view) is equal to the length of tolist. If view.ndim = 0, the length is 1. If view.ndim = 1, the length is equal to the number of elements in the view. For higher dimensions, the length is equal to the length of the nested list representation of the view. The itemsize attribute will give you the number of bytes in a single element. > > A memoryview supports slicing and indexing to expose its data. One-dimensional slicing will result in a subview: https://docs.python.org/3/c-api/memoryview.html https://docs.python.org/3/library/array.html#module-array On Tue, Oct 6, 2020, 1:56 PM Wes Turner <wes.tur...@gmail.com> wrote: > So, Sequence views that do direct addressing with doubly-linked lists? > > > https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence > > > https://docs.python.org/3/library/stdtypes.html#dict-views : > > > The objects returned by dict.keys(), dict.values() and dict.items() are > view objects. They provide a dynamic view on the dictionary’s entries, > which means that when the dictionary changes, the view reflects these > changes. > > You may be looking for (directly-addressable) NumPy arrays? > https://numpy.org/doc/stable/reference/generated/numpy.array.html > > https://numpy.org/doc/stable/reference/generated/numpy.ndarray.view.html : > > > a.view(ndarray_subclass) or a.view(type=ndarray_subclass) just returns > an instance of ndarray_subclass that looks at the same array (same shape, > dtype, etc.) This does not cause a reinterpretation of the memory. > > > > On Tue, Oct 6, 2020, 1:35 PM Alex Hall <alex.moj...@gmail.com> wrote: > >> On Tue, Oct 6, 2020 at 7:21 PM Christopher Barker <python...@gmail.com> >> wrote: >> >>> >>> >>> On Tue, Oct 6, 2020 at 10:14 AM Marco Sulla < >>> marco.sulla.pyt...@gmail.com> wrote: >>> >>>> What I do not understand is why you need to use the iterator instead >>>> of using the iterable itself. This way you can jump to whatever >>>> position without slicing. >>>> >>> >>> if you want the Nth item, that's easy, yes. >>> >>> if you want to iterate through items N to the end, then how do you do >>> that without either iterating through the first N and throwing them away, >>> or making a slice, which copies the rest of the sequence? >>> >> >> ```python >> for i in range(start, stop): >> x = lst[i] >> process(x) >> ``` >> >> The only problem is that there's slightly more execution in Python-land >> than in C-land, but that only matters if `process(x)` does very little and >> you're really concerned about performance. I can see how the proposal could >> be useful but only in very limited use cases. >> _______________________________________________ >> Python-ideas mailing list -- python-ideas@python.org >> To unsubscribe send an email to python-ideas-le...@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-ideas@python.org/message/UQ327GHUIHT46AZU73KM562NFEGNGYUQ/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/344SN3RN7AU26RNPAVCVSAUODUFDQPNC/ Code of Conduct: http://python.org/psf/codeofconduct/
