alamb commented on issue #34485: URL: https://github.com/apache/arrow/issues/34485#issuecomment-1739684988
We hit gRPC message size limits (the aforementioned 4MB limit in the default golang one) frequently The rust flight API has a workaround for this limit that does the RecordBatch slicing mentioned above for users. See https://docs.rs/arrow-flight/47.0.0/arrow_flight/encode/struct.FlightDataEncoderBuilder.html#method.with_max_flight_data_size This works pretty well for us, but it does not handle the cases that @lidavidm mentions (single large row / string value) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
