[
https://issues.apache.org/jira/browse/STORM-599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277273#comment-14277273
]
ASF GitHub Bot commented on STORM-599:
--------------------------------------
GitHub user d2r opened a pull request:
https://github.com/apache/storm/pull/381
[Storm 599] Use use nimbus's cached heartbeats rather than fetching again
from ZK, take two
This is a retry of the reverted pull request #356.
In #356, a merge error (we think?) on the branch used for the pull request
caused part of the change on one line to go missing.
```Diff
- beats (.executor-beats storm-cluster-state storm-id (:executor->node+port
assignment))
+ beats (map-val :heartbeat @(:heartbeats-cache nimbus))
```
The new code should have been:
```Diff
+ beats (map-val :heartbeat (get @(:heartbeats-cache nimbus) storm-id))
```
This caused incorrect retrieval of heartbeats and therefore metrics were
not populated as noted by @harshach
[here](https://issues.apache.org/jira/browse/STORM-599?focusedCommentId=14275512&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14275512).
Testing again with this branch in a clean environment shows that metrics
are updated as expected once per `nimbus.monitor.freq.secs`, which defaults to
10s.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/d2r/storm STORM-599-use-cached-hbs
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/381.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #381
----
commit 1d973c6838118e64024505269b82baffd0f41f8d
Author: Derek Dagit <[email protected]>
Date: 2015-01-14T17:12:30Z
use hb cache instead of fetching via zk
commit fc952d4c64f52e29cd162ffacee984ef36ab9447
Author: Derek Dagit <[email protected]>
Date: 2015-01-14T17:13:05Z
Remove unnecessary code
----
> UI is slow due to fetching heartbeats from ZK
> ---------------------------------------------
>
> Key: STORM-599
> URL: https://issues.apache.org/jira/browse/STORM-599
> Project: Apache Storm
> Issue Type: Improvement
> Affects Versions: 0.9.3
> Reporter: Derek Dagit
> Assignee: Derek Dagit
> Priority: Minor
>
> The method getTopologyInfo fetches every heartbeat from ZooKeeper in order to
> gather statistics. The UI calls this method via thrift to render the
> topology page.
> When topologies are launched with thousands of executors, this fetching from
> ZK dramatically slows things down.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
