tustvold commented on code in PR #6527: URL: https://github.com/apache/arrow-rs/pull/6527#discussion_r1793006879
########## arrow-array/src/lib.rs: ########## @@ -161,7 +161,55 @@ //! array.as_primitive::<Float32Type>().values() //! } //! ``` +//! # Alternatives to ChunkedArray Support //! +//! The Rust implementation does not provide the ChunkedArray abstraction implemented by the Python +//! and C++ Arrow implementations. The recommended alternative is to use one of the following: +//! - `Vec<ArrayRef>` a simple, eager version of a `ChunkedArray` +//! - `impl Iterator<Item=ArrayRef>` a lazy version of a `ChunkedArray` +//! - `impl Stream<Item=ArrayRef>` a lazy async version of a `ChunkedArray` +//! +//! Similar patterns can be applied at the `RecordBatch` level. For example, [DataFusion] makes +//! extensive use of [RecordBatchStream]. +//! +//! This approach integrates well into the Rust ecosystem, simplifies the implementation and +//! encourages the use of performant lazy and async patterns. +//! +//! The iterator API makes it ergonomic to work with these patterns and since most kernels accept +//! `&dyn Array` many use-cases do not even require downcasting. Review Comment: ```suggestion ``` I think this is probably redundant with the section on downcasting above, but I don't fell strongly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
