kylebarron commented on issue #43410: URL: https://github.com/apache/arrow/issues/43410#issuecomment-2299307622
That sounds awesome. For reference in my own experiments in https://github.com/kylebarron/arro3, I created an [ArrayReader class](https://kylebarron.dev/arro3/latest/api/core/array-reader/), essentially just a `RecordBatchReader` but generalized to yield generic `Array`s. Then for example [`cast`](https://kylebarron.dev/arro3/latest/api/compute/#arro3.compute.cast) is overloaded. So if it sees an object with `__arrow_c_array__` it will immediately return an `arro3.Array` with the result. If it sees an object with `__arrow_c_stream__` it will create a new `ArrayReader` holding an iterator with the compute function. So it will lazily yield casted chunks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
