Dandandan commented on code in PR #3207:
URL: https://github.com/apache/arrow-rs/pull/3207#discussion_r1044652796
##########
arrow-flight/src/utils.rs:
##########
@@ -111,3 +139,25 @@ pub fn ipc_message_from_arrow_schema(
let IpcMessage(vals) = message;
Ok(vals)
}
+
+/// Convert `RecordBatch`es to wire protocol `FlightData`s
+pub fn batches_to_flight_data(
+ schema: Schema,
+ batches: Vec<RecordBatch>,
+) -> Result<Vec<FlightData>, ArrowError> {
+ let options = IpcWriteOptions::default();
+ let schema_flight_data: FlightData = SchemaAsIpc::new(&schema,
&options).into();
+ let mut dictionaries = vec![];
+ let mut flight_data = vec![];
+ for batch in batches.iter() {
+ let (flight_dictionaries, flight_datum) =
+ flight_data_from_arrow_batch(batch, &options);
Review Comment:
See https://github.com/apache/arrow-rs/issues/3312 - for dictionary data
this is inefficient (as repeated use of the dictionary will have new
`FlightData` instances to send over the wire.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
