[
https://issues.apache.org/jira/browse/STORM-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated STORM-3093:
----------------------------------
Labels: pull-request-available (was: )
> Cache the storm id to executors mapping on master to avoid repeat computation
> -----------------------------------------------------------------------------
>
> Key: STORM-3093
> URL: https://issues.apache.org/jira/browse/STORM-3093
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-server
> Affects Versions: 2.0.0
> Reporter: Yuzhao Chen
> Assignee: Yuzhao Chen
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.0.0
>
>
> Now nimbus will collect all the topologies's conf/topology-ser/storm-base to
> compute in a scheduling round, which is a very heavy work. The scheduling
> will still take to minutes even we now change to PRC heartbeats and
> assignment distribution.
> So i decide to redesign the scheduler, so we can only schedule the topologies
> that need to: that have dead workers or not enough number workers.
> Here i checkout out the code and found that the id->executors mapping is
> computed every time for every topology, which is really a heavy computation
> and totally not that necessary, because this mapping is fixed invariable for
> a topology unless we rebalance or kill it.
> So i refactor the code a little here, and this is more powerful after the
> scheduler is resigned for delta-scheduling[ which is very lightweight even
> there are thousands of topologies on one cluster.]
> For now this is enough for us.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
