[
https://issues.apache.org/jira/browse/STORM-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007934#comment-15007934
]
ASF GitHub Bot commented on STORM-885:
--------------------------------------
Github user bastiliu commented on the pull request:
https://github.com/apache/storm/pull/838#issuecomment-157255819
@revans2 One of the reasons why we added topology master in JStorm is to
extend the hierarchy of the structrue of collection of HB & metrics to offload
Zookeepr & Nimbus(from "workers->zookeeper->nimbus" to "workers->topology
master->nimbus"). Besides it, the monitor mechanism of Nimbus is easily to
bring the dead topology master up without adding new complex recovery
functionality. Just went through these two solution again and thought about
merging. I think they are not conflicted for merging. As you mentioned above,
pacemaker is designed as a pluggable component used to extend the capablity of
nimbus for future. So, for merging, we can re-direct the reporting from
"topology master->nimbus" to "topology master->pacemaker". That will make the
scalability of processing of metrics better since the num of connections to
pacemaker are reduced significantly (one connection for each topology).
> Heartbeat Server (Pacemaker)
> ----------------------------
>
> Key: STORM-885
> URL: https://issues.apache.org/jira/browse/STORM-885
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Reporter: Robert Joseph Evans
> Assignee: Kyle Nusbaum
>
> Large highly connected topologies and large clusters write a lot of data into
> ZooKeeper. The heartbeats, that make up the majority of this data, do not
> need to be persisted to disk. Pacemaker is intended to be a secure
> replacement for storing the heartbeats without changing anything within the
> heartbeats. In the future as more metrics are added in, we may want to look
> into switching it over to look more like Heron, where a metrics server is
> running for each node/topology. And can be used to aggregate/per-aggregate
> them in a more scalable manor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
