Hi Team,
We are using Apache Kafka 3.3.1 in our application.
Scenario - 1 --> We have created an Kafka admin client from our java
application and have not configured the property "connections.max.idle.ms" so
its default value which is 5 minutes is used.
In the above scenario we see a port being established every 5 mins towards
Kafka and an old port being dropped.
9:46 :30 --> process starts.. using default property
Every 1.0s: netstat -plant | grep 9092 | grep 3455069
seliiuvd07797: Thu Dec 7 09:47:16 2023
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp6 0 0 127.0.0.1:46182 127.0.0.1:9092 ESTABLISHED
3455069/java
9:51:00 --------------------------------->
Every 1.0s: netstat -plant | grep 9092 | grep 3455069
seliiuvd07797: Thu Dec 7 09:51:38 2023
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp6 0 0 127.0.0.1:36866 127.0.0.1:9092 ESTABLISHED
3455069/java
9:59:00 --------------------------------->
Every 1.0s: netstat -plant | grep 9092 | grep 3455069
seliiuvd07797: Thu Dec 7 09:59:05 2023
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp6 0 0 127.0.0.1:51096 127.0.0.1:9092 ESTABLISHED
3455069/java
10:01:00 ---------------------------------------->
Every 1.0s: netstat -plant | grep 9092 | grep 3455069
seliiuvd07797: Thu Dec 7 10:01:40 2023
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp6 0 0 127.0.0.1:38202 127.0.0.1:9092 ESTABLISHED
3455069/java
10:06:32 -------------------------------------------->
Every 1.0s: netstat -plant | grep 9092 | grep 3455069
seliiuvd07797: Thu Dec 7 10:06:32 2023
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp6 0 0 127.0.0.1:50450 127.0.0.1:9092 ESTABLISHED
3455069/java
Scenario - 2 --> However, if we create a Apache Kafka admin client using the
property "connections.max.idle.ms" to a value greater than 5 mins. for eg. 10
mins.
Initially when our process starts (Process started at 18:19:10) we have one
connection established with the Apache Kafka broker as shown below.
Time : 18:19:10
Every 1.0s: netstat -plant | grep 9092 | grep 2860979
seliiuvd07797: Wed Dec 6 18:19:35 2023
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp6 0 0 127.0.0.1:44280 127.0.0.1:9092 ESTABLISHED
2860979/java
After 5 minutes, we observed one more connection was established ( though we
only created one single admin client from the code. so this connection is being
established internally by Apache Kafka)
Time : 18:24:10
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp6 0 0 127.0.0.1:48526 127.0.0.1:9092 ESTABLISHED
2860979/java
tcp6 0 0 127.0.0.1:44280 127.0.0.1:9092 ESTABLISHED
2860979/java
After further 5 minutes, i.e. 10 minutes since the process started ( 18:29
Hrs.) , we observed that connection from the old port was dropped.
Time : 18:29:10
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp6 0 0 127.0.0.1:48526 127.0.0.1:9092 ESTABLISHED
2860979/java
After this there were no further disconnections or new connections being
established.
So the question basically is why Kafka admin client not getting disconnected
every 10 minutes which was observed when the default value of
"connections.max.idle.ms = 300000" property is being used for creating admin
client.?
Secondly why a new connection in Scenario 2 was established at the end of 5th
minutes as shown above??
Regards,
Ankit Nigam
