alamb commented on a change in pull request #8473:
URL: https://github.com/apache/arrow/pull/8473#discussion_r507632793
##########
File path: rust/datafusion/src/physical_plan/common.rs
##########
@@ -31,53 +32,58 @@ use array::{
};
use arrow::datatypes::{DataType, SchemaRef};
use arrow::error::Result as ArrowResult;
-use arrow::record_batch::{RecordBatch, RecordBatchReader};
+use arrow::record_batch::RecordBatch;
use arrow::{
array::{self, ArrayRef},
datatypes::Schema,
};
+use futures::{Stream, TryStreamExt};
-/// Iterator over a vector of record batches
-pub struct RecordBatchIterator {
+/// Stream of record batches
+pub struct SizedRecordBatchStream {
Review comment:
Yeah, I was thinking it would be nice to imply in this struct's name
somehow that it should be avoided if possible, because, as you say, it requires
buffering / collecting up the `RecordBatch`es before producing any results
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
