Vadym Zhytkevych created KAFKA-19889:
----------------------------------------
Summary: max.connections.per.ip.overrides does not apply overrides
to all IPs for multi-IP hostnames
Key: KAFKA-19889
URL: https://issues.apache.org/jira/browse/KAFKA-19889
Project: Kafka
Issue Type: Bug
Affects Versions: 3.8.0
Reporter: Vadym Zhytkevych
Assignee: Vadym Zhytkevych
{{max.connections.per.ip}} can be overridden per host using the
{{max.connections.per.ip.overrides}} configuration.
The override value may be specified using either an IP address or a hostname.
When a hostname is used, and that hostname resolves to multiple IP addresses
(e.g., behind a load balancer or auto-scaling group), {*}only the first
resolved IP address receives the override{*}, while all other resolved
addresses fallback to the default limit.
*Example:*
{code:java}
max.connections.per.ip=100
max.connections.per.ip.overrides=kafka.connect.test.com:500{code}
hostname resolution:
{code:java}
kafka.connect.test.com => [127.0.0.1, 127.0.0.2, 127.0.0.3] {code}
actual applied values:
{code:java}
127.0.0.1 => 500
127.0.0.2 => 100
127.0.0.3 => 100 {code}
*Expected behavior*
All IPs resolved from the hostname should receive the override:
{code:java}
127.0.0.1 => 500
127.0.0.2 => 500
127.0.0.3 => 500 {code}
*Use Case*
We configure a global connection limit of 100 and want to increase the limit
for Kafka Connect workers behind a load-balanced and auto-scaling hostname.
Using the hostname in {{max.connections.per.ip.overrides}} should apply the
override to _all_ backend IPs.
*Cause*
Current code uses:
{code:java}
@volatile private var maxConnectionsPerIpOverrides =
config.maxConnectionsPerIpOverrides.map { case (host, count) =>
(InetAddress.getByName(host), count) } {code}
where _InetAddress.getByName(host)_ returns only the first resolved IP, as it
internally uses:
{code:java}
public static InetAddress getByName(String host)
throws UnknownHostException {
return InetAddress.getAllByName(host)[0];
} {code}
*Proposed Fix*
Resolve all IPs using {{InetAddress.getAllByName(host)}} and apply the override
value to each returned address.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
