[
https://issues.apache.org/jira/browse/KAFKA-14025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mao updated KAFKA-14025:
------------------------------
Description:
In the NetworkClient we measure request timeouts from when a request is first
queued in
{{{}InflightRequests{}}}. A Kafka broker currently only processes at most one
in-flight request at a time per connection. This disconnect between client-side
request timeout enforcement and broker-side request processing means that an
aggressive request timeout can result in spurious timeouts when a connection
has multiple requests in-flight.
Example:
broker 0 has avg 25 ms latency, producer 1 has a request timeout of 50 ms.
t=0 p1 sends r1, b0 starts processing r1
t=5 p1 sends r2
t=10 p1 sends r3
t=25 b0 sends response for r1, starts processing r2
t=50 b0 sends response for r2, starts processing r3
t=60 client times out r3, despite the broker only having 10 ms or so to process
the request.
Instead, we should be measuring request timeouts only once previous requests
have been responded to.
was:
In the NetworkClient we measure request timeouts from when a request is first
queued in
{{{}InflightRequests{}}}. A Kafka broker currently only processes at most one
in-flight request at a time per connection. This disconnect between client-side
request timeout enforcement and broker-side request processing means that an
aggressive request timeout can result in spurious timeouts when a connection
has multiple requests in-flight.
Example:
broker 0 has avg 25 ms latency, producer 1 has a request timeout of 50 ms.
t=0 p1 sends r1, b0 starts processing r1
t=5 p1 sends r2
t=10 p1 sends r3
t=25 b0 sends response for r1, starts processing r2
t=50 b0 sends response for r2, starts processing r3
t=60 client times out r3
Instead, we should be measuring request timeouts only once previous requests
have been responded to.
> Request timeout includes time queued per connection
> ---------------------------------------------------
>
> Key: KAFKA-14025
> URL: https://issues.apache.org/jira/browse/KAFKA-14025
> Project: Kafka
> Issue Type: Bug
> Reporter: David Mao
> Priority: Major
>
> In the NetworkClient we measure request timeouts from when a request is first
> queued in
> {{{}InflightRequests{}}}. A Kafka broker currently only processes at most one
> in-flight request at a time per connection. This disconnect between
> client-side request timeout enforcement and broker-side request processing
> means that an aggressive request timeout can result in spurious timeouts when
> a connection has multiple requests in-flight.
> Example:
> broker 0 has avg 25 ms latency, producer 1 has a request timeout of 50 ms.
> t=0 p1 sends r1, b0 starts processing r1
> t=5 p1 sends r2
> t=10 p1 sends r3
> t=25 b0 sends response for r1, starts processing r2
> t=50 b0 sends response for r2, starts processing r3
> t=60 client times out r3, despite the broker only having 10 ms or so to
> process the request.
>
> Instead, we should be measuring request timeouts only once previous requests
> have been responded to.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
