[
https://issues.apache.org/jira/browse/IMPALA-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Ho updated IMPALA-5859:
-------------------------------
Parent Issue: IMPALA-7300 (was: IMPALA-5865)
> Better flow-control for TransmitData() with KRPC
> ------------------------------------------------
>
> Key: IMPALA-5859
> URL: https://issues.apache.org/jira/browse/IMPALA-5859
> Project: IMPALA
> Issue Type: Sub-task
> Components: Distributed Exec
> Reporter: Michael Ho
> Priority: Major
>
> Each KRPC service by default has a fixed length queue which limits the number
> of incoming call requests to be processed. If the queue fills up, any
> incoming calls will be dropped and the RPC layer will return an error message:
> {noformat}
> // Queue message on service queue
> boost::optional<InboundCall*> evicted;
> auto queue_status = service_queue_.Put(c, &evicted);
> if (queue_status == QUEUE_FULL) {
> RejectTooBusy(c);
> return Status::OK();
> }
> {noformat}
> The current implementation of TransmitData() with KRPC will sleep for a
> predefined period of time before retrying the RPC. However, it may be
> wasteful of network bandwidth to resend the entire row batch if the remote
> server's service queue is still filled up.
> Instead, when the client hits the capacity limit above, it should fall back
> to a periodic ping RPC to check if the remote server has any space available
> and only when space is available in the remote service should the client send
> the entire row batch again.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
