[
https://issues.apache.org/jira/browse/ARROW-10351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332487#comment-17332487
]
Yibo Cai edited comment on ARROW-10351 at 4/27/21, 4:24 AM:
------------------------------------------------------------
Regarding random test data, I do see the compressed data is actually larger
than original uncompressed data.
Shall we replace random test data with some real world data? Otherwise, the
compression test in current benchmark will always be a pure loss.
was (Author: yibocai):
Regarding random test data, I do see the compressed data is actually larger
than original uncompressed data.
> [C++][Flight] See if reading/writing to gRPC get/put streams asynchronously
> helps performance
> ---------------------------------------------------------------------------------------------
>
> Key: ARROW-10351
> URL: https://issues.apache.org/jira/browse/ARROW-10351
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, FlightRPC
> Reporter: Wes McKinney
> Priority: Major
>
> We don't use any asynchronous concepts in the way that Flight is implemented
> now, i.e. IPC deconstruction/reconstruction (which may include compression!)
> is not performed concurrent with moving FlightData objects through the gRPC
> machinery, which may yield suboptimal performance.
> It might be better to apply an actor-type approach where a dedicated thread
> retrieves and prepares the next raw IPC message (within a Future) while the
> current IPC message is being processed -- that way reading/writing to/from
> the gRPC stream is not blocked on the IPC code doing its thing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
