[
https://issues.apache.org/jira/browse/STORM-636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287775#comment-14287775
]
ASF GitHub Bot commented on STORM-636:
--------------------------------------
GitHub user d2r opened a pull request:
https://github.com/apache/storm/pull/392
[STORM-636] Faster, optional retrieval of last component error
We want to speed up interactions with the topology that call
getTopologyInfo, including the Topology Page in the UI.
Before change:
- Errors written to /errors/topo-id/comp-name/eN, where N is a sequence
number.
- getTopologyInfo grabs all /errors/topo-id/*/* errors from ZK
After this change:
- Errors written to /errors/topo-id/comp-name/eN, where N is a sequence
number. (unchanged)
- Errors also written to /errors/topo-id/comp-name-last-error (this means
there is an extra ZK write per error)
- New Nimbus thrift interface method `getTopologyInfoWithOpts`
- Only option right now is how many errors to retrieve from ZK: 0, 1, or
all.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/d2r/storm storm-636-ui-errors
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/392.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #392
----
commit 1cfa190f2efb06f8798984b43dec801e5ff20ad5
Author: Derek Dagit <der...@yahoo-inc.com>
Date: 2015-01-22T16:46:03Z
Faster, optional retrieval of last component error
----
> UI/Monitor is slow for topologies with a large number of components
> -------------------------------------------------------------------
>
> Key: STORM-636
> URL: https://issues.apache.org/jira/browse/STORM-636
> Project: Apache Storm
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Derek Dagit
> Assignee: Derek Dagit
> Priority: Minor
>
> The getTopologyInfo method in nimbus fetches from ZK all errors reported by
> all components. This becomes too slow for topologies with a larger numbers
> of components (bolts/spouts).
> In one example, the UI consistently took over 5 minutes to load the topology
> page for a topology with nearly 500 components while ZK was under load.
> Errors are currently stored in ZooKeeper under individual znodes per
> component. This means that each call to getTopologyInfo needs to list
> children of each znode and then download the error znodes it finds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
