thinker0 opened a new issue, #3497:
URL: https://github.com/apache/kvrocks/issues/3497

   ## Summary
   
   When a cluster is created or updated through the controller, the controller
   pushes a `CLUSTERX SETNODES` payload that includes per-node master/slave
   role assignments. Each node accepts the command (`+OK`), **but the nodes
   marked as slave do not initiate replication** — `INFO replication` on every
   node returns `role:master` with `connected_slaves:0`.
   
   The replication link is never set up; every member behaves as a standalone
   master that just happens to know about other members.
   
   ## Observed
   
   Immediately after a fresh `POST /api/v1/namespaces/.../clusters` (54 nodes
   with `replicas=3`, expected 18 shards × 1 master + 2 slaves), we sampled
   each node's actual `INFO replication` `role:` and compared with the role
   recorded in the controller meta:
   
   ```
   Role alignment: match=26, mismatch=28, dead=0 (total 54)
   ```
   
   Roughly half the nodes (28/54) are not following the master the controller
   told them to follow. This shows up immediately after the API call — no
   failure injection, no restart needed.
   
   Once a node is in this "ghost master" state, no further controller action
   (`SETNODES` re-push, `DELETE`+recreate, manager restart) brings it back
   into a replica role.
   
   ## Expected
   
   When `CLUSTERX SETNODES` (or the controller's create/update API) marks a
   node as a slave of a particular master, the node should issue an internal
   `SLAVEOF <master-host> <master-port>` to start replication, OR the
   controller should explicitly issue `REPLICAOF` to each slave after the
   topology push.
   
   ## Versions
   
   - Apache Kvrocks 2.15.0
   - `kvctl-server` v1.15.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to