alamb commented on code in PR #8262:
URL: https://github.com/apache/arrow-rs/pull/8262#discussion_r2323545563
##########
parquet/src/arrow/async_writer/mod.rs:
##########
@@ -332,6 +350,35 @@ mod tests {
assert_eq!(to_write, read);
}
+ #[tokio::test]
+ async fn test_async_arrow_group_writer() {
+ let col = Arc::new(Int64Array::from_iter_values([1, 2, 3])) as
ArrayRef;
+ let to_write = RecordBatch::try_from_iter([("col", col)]).unwrap();
+
+ let mut buffer = Vec::new();
+ let mut writer = AsyncArrowWriter::try_new(&mut buffer,
to_write.schema(), None).unwrap();
+ let mut writers = writer.get_column_writers().await.unwrap();
+
+ for (field, column) in
to_write.schema().fields().iter().zip(to_write.columns()) {
+ for leaf in compute_leaves(field.as_ref(), column).unwrap() {
+ writers[0].write(&leaf).unwrap();
+ }
+ }
+
+ let columns: Vec<_> = writers.into_iter().map(|w|
w.close().unwrap()).collect();
+ writer.append_row_group(columns).await.unwrap();
Review Comment:
this test only writes a single row group, but there is code above that
checks for buffered row groups.
Can you please add a test for the other buffering? It looks like that would
be something like append some rows normally, and then call `get_column_writers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
