[
https://issues.apache.org/jira/browse/KAFKA-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118317#comment-14118317
]
Jun Rao commented on KAFKA-1282:
--------------------------------
Thanks for the latest patch. I was trying to do some local testing. The
following are my observations.
1. I first started a local ZK and broker (setting connections.max.idle.ms
10secs). I then started a console-producer and a console-consumer. Then, I
typed in sth in console-producer every 15 secs. However, I don't see the
producer connection gets killed. I added sth instrumentation. It doesn't seem
that removeEldestEntry() is called on every fetch request.
2. As I was debugging this, I realized that it's kind of weird to kill idle
connections only when there is another non-idle connection. This makes
debugging harder since one can't just test this out with a single connection.
It's much simpler to understand if the idle connection can just be killed after
the connection idle time, independent of other connections to the broker. To
address the concern of closing many sockets in one iteration of the selector,
we can calculate the time that a socket entry is expected to be killed (this is
the access time of the oldest entry + maxIdleTime, or maxIdleTime if no entry
exists). When that time comes during the iteration of the selector, we can just
check the oldest entry and see if it needs to be closed.
3. It would be good to check if our clients (especially the producer, both old
and new) can handle a closed idle connection properly. For example, when
detecting an already closed socket, the producer should be able to resend the
message and therefore we shouldn't see any data loss.
> Disconnect idle socket connection in Selector
> ---------------------------------------------
>
> Key: KAFKA-1282
> URL: https://issues.apache.org/jira/browse/KAFKA-1282
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 0.8.2
> Reporter: Jun Rao
> Assignee: Neha Narkhede
> Labels: newbie++
> Fix For: 0.9.0
>
> Attachments:
> KAFKA-1282_Disconnect_idle_socket_connection_in_Selector.patch,
> idleDisconnect.patch
>
>
> To reduce # socket connections, it would be useful for the new producer to
> close socket connections that are idle. We can introduce a new producer
> config for the idle time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
