[
https://issues.apache.org/jira/browse/STORM-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim updated STORM-2557:
--------------------------------
Fix Version/s: 1.2.0
> A bug in DisruptorQueue causing severe underestimation of queue arrival rates
> -----------------------------------------------------------------------------
>
> Key: STORM-2557
> URL: https://issues.apache.org/jira/browse/STORM-2557
> Project: Apache Storm
> Issue Type: Bug
> Reporter: tangkailin
> Assignee: tangkailin
> Fix For: 2.0.0, 1.1.1, 1.2.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Recently, we are tuning the performance of our topology and deploying some
> theoretical performance models that heavily rely on the metrics of query
> arrival rates. We found a bug in DisruptorQueue that leads to severe
> underestimation to the queue arrival rates. After further investigation, we
> finally found that in the current implementation of DisruptorQueue, the
> arrival rates are actually measured as the number of batches of tuples rather
> than the actual number of tuples, resulting in significant underestimation of
> the arrival rates.
> To be more specific, in DisruptorQueue.publishDirectSingle() and
> DisruptorQueue.publishDirect() functions, objects containing tuples are
> published to the buffer and the metrics are notified by calling
> _metric.notifyArrivals(1). This works fine when the object is simply a
> wrapper of a single tuple. However, the object could also be an instance of
> ArrayList<AddressedTuple> or HashMap<Integer, ArrayList<TaskMessage>>. In
> such case, we should get the actual number of tuples in the object and notify
> the _metrics with the right value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
