[
https://issues.apache.org/jira/browse/STORM-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046381#comment-14046381
]
ASF GitHub Bot commented on STORM-375:
--------------------------------------
Github user nathanmarz commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/167#discussion_r14314682
--- Diff: storm-core/src/clj/backtype/storm/daemon/supervisor.clj ---
@@ -297,7 +307,10 @@
^ISupervisor isupervisor (:isupervisor supervisor)
^LocalState local-state (:local-state supervisor)
sync-callback (fn [& ignored] (.add event-manager this))
- assignments-snapshot (assignments-snapshot storm-cluster-state
sync-callback)
+ assignment-versions (.get local-state LS-ASSIGNMENT-VERSIONS)
--- End diff --
This shouldn't be stored in the local state as there is no need to. Just
keep this information purely in memory. It's ok if the supervisor has to
re-download everything on death because deaths are rare. Persisted state should
be minimized as a general rule.
> Smarter downloading of assignments by supervisors and workers
> -------------------------------------------------------------
>
> Key: STORM-375
> URL: https://issues.apache.org/jira/browse/STORM-375
> Project: Apache Storm (Incubating)
> Issue Type: Improvement
> Reporter: Kyle Nusbaum
>
> Periodically the supervisors and workers each download their assignments from
> zookeeper, and the traffic begins to saturate the network and hammer
> zookeeper as we scale. Having these daemons only grab the assignments when
> the zookeeper path's version number changes reduces this traffic
> considerably, along with STORM-376.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
