DevMattG commented on PR #11048: URL: https://github.com/apache/arrow-rs/pull/11048#issuecomment-5661548742
@Rich-T-kid, while in #10383 you focus on changing the heuristic, this PR introduces a second step after it, that directly evaluates the actual encoded size of the flight batch. If it still exceeds `max_flight_data_size` a re-split is performed. I think this could solve the problem of the inflated sizes after IPC encoding you were addressing here https://github.com/apache/arrow-rs/pull/10383#issuecomment-5018890719. Maybe combining this PR with yours could be the best solution so far? (at least I could think of) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
