allthingssecurity opened a new pull request, #27311: URL: https://github.com/apache/camel/pull/27311
# Description [CAMEL-25286](https://issues.apache.org/jira/browse/CAMEL-25286) `ConsulClusterView` holds the leadership with a lock on `<rootPath>/<namespace>` owned by a Consul session, watches the key with a chain of blocking queries and renews the session with each query. Three paths ended the leader election for the node until the cluster service was restarted: - **A failed query** (the agent restarts, a network error, a 500 while the Consul cluster has no leader): `onFailure` released the lock of the *root* path (not the namespace path that the session holds, the mistake fixed in `getLeader()` by #4937), synchronously. While Consul could not be reached that call threw, so `setMaster(false)` and `watch()` never ran: no more queries and no more renewals. A leader kept its leadership (its clustered routes kept running) until its session expired, then another node took the lock and both ran the clustered routes. An exception while handling an answer ended the watch the same way, and `setMaster(false)` called Consul to fire the leadership event, which was lost while Consul was down. - **An invalidated session** (TTL expired during a partition, the local agent restarted or left, Consul lost its data) was never replaced: each renewal failed with `404 Session id '...' not found` (the stack trace in CAMEL-15423) and `acquireLock` returned false for ever. If all nodes lost their sessions no node was leader any more. - **A missing key** (Consul lost its data) was ignored by `onComplete`, so no node acquired the lock again. This change: - `onFailure` gives the leadership up, releases the lock of the namespace path (a failure is logged at debug) and queries again after `sessionRefreshInterval` (at least one second) on a single-thread scheduler of the view (created in `doStart`, shut down in `doStop`), so that an agent that cannot be reached is not queried in a loop. - The renewal replaces a session that Consul does not know any more (404, or no session returned) by a new one, under the session lock and only while the view runs, after giving the leadership up. Other renewal failures are logged at debug and retried with the next query. - `onComplete` handles a missing key like a free one (tries to acquire it), stores the index first, and gives the leadership up instead of ending the watch when handling the answer fails. - The leadership event is fired with no leader when the current leader cannot be read. Points a reviewer may question: - Releasing the right key changes what happens after one failed query while Consul is reachable: the lock is now really released, so another node can take the leadership (before, the release was a no-op and the node took its own lock back with the next answer). This is what the failure path was written to do; the node has already given the leadership up locally (main did that too whenever the wrong-key release did not throw). Consul applies `lock-delay` only when a session is invalidated, not on an explicit release, so the hand-over is immediate. While the Consul cluster has no leader (500) the release fails as well, so the node keeps its lock and takes the leadership back with the next answer. Other cluster services also give the leadership up on the first failure: `FileLockClusterView` releases its file lock when a heartbeat write fails, and Curator's `LeaderSelector` (ZooKeeper) cancels the leadership on `SUSPENDED`; the Kubernetes lease controller tolerates failures until `ren ewDeadline`. Since CAMEL-25062 `ClusteredRoutePolicy` starts and stops its routes on its own thread, so the leadership event fired from the renewal (under the session lock) does not stop routes there. - No upgrade note (no option or default changes; the view now does what it was meant to do). Happy to add one. - `mockito-core` is added as a test dependency, with exactly the same lines as #27304 (CAMEL-25275), so the two merge cleanly in either order. Tests: - `ConsulClusterViewRecoveryTest` (new): the real `ConsulClusterService` and view against mocked `SessionClient` / `KeyValueClient` that simulate Consul (sessions, lock holder, key, reachability; renewing an unknown session throws `ConsulException(call, response)` with code 404, as the kiwiproject client 1.12.1 does). Four cases: Consul cannot be reached, a failed query releases the namespace lock, an invalidated session, Consul lost its data. - Without the change all 4 fail (`expected: <false> but was: <true>` twice, `expected: <2> but was: <1>`, `Argument(s) are different! Wanted: releaseLock("/camel/my-ns", "session-1")`). - With the change all camel-consul unit tests pass: 8 tests, 0 failures (the cluster ITs need Docker). # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built and tested the affected module, including the formatter and import-sort plugins. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
