[
https://issues.apache.org/jira/browse/KAFKA-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swapnil Ghike updated KAFKA-1017:
---------------------------------
Description:
Reported by Jun Rao:
For over-partitioned topics, each broker could be the leader for at least 1
partition. In the producer, we randomly select a partition to send the data.
Pretty soon, each producer will establish a connection to each of the n
brokers. Effectively, we increased the # of socket connections by a factor of
n, compared to 0.7.
The increased number of socket connections increases the number of open file
handles, this could come pretty close to the OS limit.
was:
Reported by Jun Rao:
For over-partitioned topics, each broker could be the leader for at least 1
partition. In the producer, we randomly select a partition to send the data.
Pretty soon, each producer will establish a connection to each of the n
brokers. Effectively, we increased the # of socket connections by a factor of
n, compared to 0.7.
The increased number of socket connections increases the number of open file
handles, this could come pretty close to the OS limit if left unnoticed.
> High number of open file handles in 0.8 producer
> ------------------------------------------------
>
> Key: KAFKA-1017
> URL: https://issues.apache.org/jira/browse/KAFKA-1017
> Project: Kafka
> Issue Type: Bug
> Reporter: Swapnil Ghike
> Assignee: Swapnil Ghike
>
> Reported by Jun Rao:
> For over-partitioned topics, each broker could be the leader for at least 1
> partition. In the producer, we randomly select a partition to send the data.
> Pretty soon, each producer will establish a connection to each of the n
> brokers. Effectively, we increased the # of socket connections by a factor of
> n, compared to 0.7.
> The increased number of socket connections increases the number of open file
> handles, this could come pretty close to the OS limit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
