Hi all, I'm studying the Storm 2.0 heartbeat/liveness paths and want to confirm my understanding of a design decision.
As I read the 2.0 code, there are two distinct worker-originated heartbeats: 1. *Liveness* — the worker writes an LSWorkerHeartbeat to local disk ( Worker.doHeartBeat); the supervisor reads those files and relays a batch to the leader Nimbus over *Thrift* (ReportWorkerHeartbeats → Nimbus.sendSupervisorWorkerHeartbeats → HeartbeatCache), governed by nimbus.task.timeout.secs. 2. *Stats* — Worker.doExecutorHeartbeats writes a heartbeat object ( time-secs + uptime + executor stats) to *ZooKeeper*, which the UI/metrics consume (and which HeartbeatCache.updateFromZkHeartbeat can still use for liveness on the ZK strategy). My understanding is that liveness was moved off ZooKeeper (the 1.x model, and later Pacemaker) because high-volume per-worker heartbeat writes made ZK a scaling bottleneck, and since heartbeats are ephemeral they don't need ZK's persistence/consistency — so the supervisor-relay-over-Thrift model removes those writes from ZK entirely. A few questions: - Is that the correct/primary motivation for the Thrift supervisor-relay path, or were there other drivers (connection count, watch load, Nimbus HA, recovery on leader change)? - Why do executor *stats* still go through ZooKeeper rather than riding the same Thrift path — is it purely that stats are lower-frequency and UI-oriented, or is there a stronger reason? - Is there a JIRA / design doc that captures this transition (beyond docs/Pacemaker.md) that I could read? Thanks for any pointers — trying to make sure I document this accurately.
