Michael Ho created IMPALA-5859:
----------------------------------
Summary: Better flow-control for TransmitData() with KRPC
Key: IMPALA-5859
URL: https://issues.apache.org/jira/browse/IMPALA-5859
Project: IMPALA
Issue Type: Sub-task
Components: Distributed Exec
Reporter: Michael Ho
Each KRPC service by default has a fixed length queue which limits the number
of incoming call requests to be processed. If the queue fills up, any incoming
calls will be dropped and the RPC layer will return an error message:
{noformat}
// Queue message on service queue
boost::optional<InboundCall*> evicted;
auto queue_status = service_queue_.Put(c, &evicted);
if (queue_status == QUEUE_FULL) {
RejectTooBusy(c);
return Status::OK();
}
{noformat}
The current implementation of TransmitData() with KRPC will sleep for a
predefined period of time before retrying the RPC. However, it may be wasteful
of network bandwidth to resend the entire row batch if the remote server's
service queue is still filled up.
Instead, when the client hits the capacity limit above, it should fall back to
a periodic ping RPC to check if the remote server has any space available and
only when space is available in the remote service should the client send the
entire row batch again.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
