[
https://issues.apache.org/jira/browse/STORM-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231704#comment-15231704
]
ASF GitHub Bot commented on STORM-1698:
---------------------------------------
GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/storm/pull/1323
STORM-1698 Asynchronous MetricsConsumerBolt (1.x)
* change MetricsConsumerBolt's behavior to asynchronus manner
* to avoid bad side effect of topology
* for details please refer JIRA issue:
https://issues.apache.org/jira/browse/STORM-1698
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/storm STORM-1698-1.x
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1323.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1323
----
commit 2dce735bacc5eb1ff7de5ca9f161373767e4f30d
Author: Jungtaek Lim <[email protected]>
Date: 2016-04-08T04:05:00Z
STORM-1698 Asynchronous MetricsConsumerBolt
* change MetricsConsumerBolt's behavior to asynchronus manner
* to avoid bad side effect of topology
* for details please refer JIRA issue:
https://issues.apache.org/jira/browse/STORM-1698
----
> Asynchronous MetricsConsumerBolt
> --------------------------------
>
> Key: STORM-1698
> URL: https://issues.apache.org/jira/browse/STORM-1698
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> Currently MetricsConsumerBolt is delegating MetricsConsumer to handle data
> points via synchronous manner.
> When MetricsConsumer cannot keep up, it will trigger backpressure when (queue
> size + overflow buffer size) reaches high watermark, which incurs slowing
> down the topology in result.
> Slowing down Itself is not a problem because that’s what backpressure is for.
> The actual problem is that backpressure only throttles spout, not metrics. If
> MetricsConsumerBolt cannot keep up with incoming tuples, backpressure never
> ends and topology just hangs. If we turn off backpressure, we have unbounded
> queue and worker could throw OOME eventually.
> Making MetricsConsumerBolt asynchronous can resolve this issue. One downside
> of making it async is that it's hard to see that MetricsConsumerBolt is
> keeping up now. (capacity will be always around 0)
> I don't have an idea for now but I think it's still better than current.
> Before making consensus about huge change of metrics, I'd love to improve
> current metrics without breaking backward compatible manner. It could be
> applied to 1.x-branch, and even 0.10.x-branch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
