[
https://issues.apache.org/jira/browse/STORM-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim resolved STORM-2557.
---------------------------------
Resolution: Fixed
Assignee: tangkailin
Fix Version/s: 1.1.1
2.0.0
Thanks [~wendyshusband], I merged into master and 1.x branch.
> A bug in DisruptorQueue causing severe underestimation of queue arrival rates
> -----------------------------------------------------------------------------
>
> Key: STORM-2557
> URL: https://issues.apache.org/jira/browse/STORM-2557
> Project: Apache Storm
> Issue Type: Bug
> Reporter: tangkailin
> Assignee: tangkailin
> Fix For: 2.0.0, 1.1.1
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Recently, we are tuning the performance of our topology and deploying some
> theoretical performance models that heavily rely on the metrics of query
> arrival rates. We found a bug in DisruptorQueue that leads to severe
> underestimation to the queue arrival rates. After further investigation, we
> finally found that in the current implementation of DisruptorQueue, the
> arrival rates are actually measured as the number of batches of tuples rather
> than the actual number of tuples, resulting in significant underestimation of
> the arrival rates.
> To be more specific, in DisruptorQueue.publishDirectSingle() and
> DisruptorQueue.publishDirect() functions, objects containing tuples are
> published to the buffer and the metrics are notified by calling
> _metric.notifyArrivals(1). This works fine when the object is simply a
> wrapper of a single tuple. However, the object could also be an instance of
> ArrayList<AddressedTuple> or HashMap<Integer, ArrayList<TaskMessage>>. In
> such case, we should get the actual number of tuples in the object and notify
> the _metrics with the right value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
