Tim Armstrong created IMPALA-9134:
-------------------------------------
Summary: Parallelise flush in data stream sender
Key: IMPALA-9134
URL: https://issues.apache.org/jira/browse/IMPALA-9134
Project: IMPALA
Issue Type: Improvement
Components: Distributed Exec
Reporter: Tim Armstrong
The data stream sender currently does a synchronous RPC to close each channel
https://github.com/apache/impala/blob/d4648e8/be/src/runtime/krpc-data-stream-sender.cc#L565.
This is suboptimal because it serializes the network round-trips and takes
sum(RTT) over all the destinations in the best case, where no data needs to be
flushed or 2 * sum(RTT) in the worst case if all channels need to flush data.
If the RPCs were done asynchronously and overlapped with each other, we could
get this down to 2 * max(RTT).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
