reiabreu opened a new pull request, #8984: URL: https://github.com/apache/storm/pull/8984
> **WIP / draft.** Opening early for visibility and discussion. No JIRA linked yet. ## What Corrects heartbeat documentation that still described the pre-2.0 model (workers/tasks heartbeating directly into ZooKeeper) and deprecates a config key that no code reads. Motivated by the user-list thread [_Design rationale: worker/executor liveness heartbeats over Thrift vs ZooKeeper_](https://lists.apache.org/thread/jrk810xtcpymtc6345j2yh0l9wmvr49b). ## Background (current behavior, verified against the code) - **Worker/executor liveness:** each worker writes `LSWorkerHeartbeat` to local disk; its supervisor's `ReportWorkerHeartbeats` relays them to Nimbus over Thrift (`sendSupervisorWorkerHeartbeats`), where they are held in an in-memory `HeartbeatCache`. Nimbus times workers out with `nimbus.task.timeout.secs`. This replaced the ZooKeeper-based path in STORM-2693. Worker heartbeats only reach a state store (`WORKERBEATS_SUBTREE`) under Pacemaker/legacy configuration. - **Supervisor liveness:** `SupervisorHeartbeat` writes `SupervisorInfo` as an **ephemeral** ZooKeeper node (`StormClusterStateImpl#supervisorHeartbeat`). Supervisor death is detected via ZooKeeper session expiry, not a Nimbus-side timeout. - Consequently `nimbus.supervisor.timeout.secs` (`DaemonConfig.NIMBUS_SUPERVISOR_TIMEOUT_SECS`) is **never read** in main source. ## Changes **Docs** - `Daemon-Fault-Tolerance.md` — describe the worker heartbeat Thrift-relay path and the ephemeral-znode supervisor liveness mechanism. - `Lifecycle-of-a-topology.md` — inline note that the 0.7.1 ZK-directory heartbeat model was replaced in 2.0 (the page is otherwise a period-accurate 0.7.1 walkthrough, left intact). - `Cluster-State-Serialization.md` — clarify worker heartbeats are not persisted in ZooKeeper by default. **Config** - `DaemonConfig.NIMBUS_SUPERVISOR_TIMEOUT_SECS` — marked `@Deprecated` with a javadoc explaining why it is unused (kept for backward compatibility). - `conf/defaults.yaml` — removed the misleading `nimbus.supervisor.timeout.secs: 60` default. - `NimbusClojurePortTest` — removed two inert references to the key. ## Verification - `grep` confirms only the deprecated constant remains (no `defaults.yaml`/test references). - No `defaults.yaml`-completeness or config-field-iteration test references the key. - `mvn -pl storm-server clean test-compile` → BUILD SUCCESS (231 main + 85 test sources). ## Open items (WIP) - File/link a JIRA (STORM-XXXX) and retitle accordingly. - Confirm the STORM-2693 rationale wording with committers who worked on it. - Decide whether to fully remove `NIMBUS_SUPERVISOR_TIMEOUT_SECS` in a later major release after the deprecation cycle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
