[
https://issues.apache.org/jira/browse/STORM-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
P. Taylor Goetz resolved STORM-1674.
------------------------------------
Resolution: Fixed
Fix Version/s: 1.1.0
1.0.2
2.0.0
> Idle KafkaSpout consumes more bandwidth than needed
> ---------------------------------------------------
>
> Key: STORM-1674
> URL: https://issues.apache.org/jira/browse/STORM-1674
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka
> Affects Versions: 0.9.3
> Reporter: Robert Hastings
> Fix For: 2.0.0, 1.0.2, 1.1.0
>
>
> Discovered 30 megabits of traffic flowing between a set of KafkaSpouts
> and our kafka servers even though no Kafka messages were moving.
> Using the wireshark kafka dissector, we were able to see that
> each FetchRequest had maxWait set to 10000
> and minBytes set to 0. When binBytes is set to 0 the kafka server
> responds immediately when there are no messages. In turn the KafkaSpout
> polls without any delay causing a constant stream of FetchRequest/
> FetchResponse messages. Using a non-KafkaSpout client had a similar
> traffic pattern with two key differences
> 1) minBytes was 1
> 2) maxWait was 100
> With these FetchRequest parameters and no messages flowing,
> the kafka server delays the FetchResponse by 100 ms. This reduces
> the network traffic from megabits to the low kilobits. It also
> reduced the CPU utilization of our kafka server from 140% to 2%.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
