We have a 1.0.2 cluster with a node that's gone but still listed in member_status (as legacy):
riak-admin member_status Attempting to restart script through sudo -u riak ================================= Membership ================================== Status Ring Pending Node ------------------------------------------------------------------------------- (legacy) 7.8% -- '[email protected]' valid 7.8% -- '[email protected]' valid 7.8% -- '[email protected]' valid 7.8% -- '[email protected]' valid 7.8% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' ------------------------------------------------------------------------------- Valid:13 / Leaving:0 / Exiting:0 / Joining:0 / Down:0 riak-admin force-remove does not remove the node (we've tried a few times over the last 4 days). This node was down before we did the upgrade, but wasn't removing so we upgraded anyway. As you can see here, doing a force-remove reports successful, but the node is still listed: riak-admin force-remove '[email protected]' Attempting to restart script through sudo -u riak Success: "[email protected]" removed from the cluster riak-admin member_status Attempting to restart script through sudo -u riak ================================= Membership ================================== Status Ring Pending Node ------------------------------------------------------------------------------- (legacy) 7.8% -- '[email protected]' valid 7.8% -- '[email protected]' valid 7.8% -- '[email protected]' valid 7.8% -- '[email protected]' valid 7.8% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' valid 7.6% -- '[email protected]' ------------------------------------------------------------------------------- Valid:13 / Leaving:0 / Exiting:0 / Joining:0 / Down:0 After this, a small number of handoffs are seen in the logs: 17:13:29.514 [info] Handoff of partition riak_kv_vnode 1033327329519114953886199041881434478741108555776 from '[email protected]' to '[email protected]' completed: sent 2 objects in 0.04 seconds 17:13:39.470 [info] Starting handoff of partition riak_kv_vnode 176978713895539025251227460211737396911460581376 from '[email protected]' to '[email protected]' 17:13:39.480 [info] Starting handoff of partition riak_kv_vnode 1238850997268773176758592221482161778380224069632 from '[email protected]' to '[email protected]' 17:13:39.512 [info] Handoff of partition riak_kv_vnode 176978713895539025251227460211737396911460581376 from '[email protected]' to '[email protected]' completed: sent 1 objects in 0.04 seconds 17:13:39.525 [info] Handoff of partition riak_kv_vnode 1238850997268773176758592221482161778380224069632 from '[email protected]' to '[email protected]' completed: sent 3 objects in 0.04 seconds Here's the pending transfer list: riak-admin transfers Nodes ['[email protected]'] are currently down. '[email protected]' waiting to handoff 28 partitions '[email protected]' waiting to handoff 40 partitions '[email protected]' waiting to handoff 40 partitions '[email protected]' waiting to handoff 40 partitions '[email protected]' waiting to handoff 40 partitions '[email protected]' waiting to handoff 39 partitions '[email protected]' waiting to handoff 40 partitions '[email protected]' waiting to handoff 40 partitions '[email protected]' waiting to handoff 40 partitions '[email protected]' waiting to handoff 40 partitions '[email protected]' waiting to handoff 40 partitions '[email protected]' waiting to handoff 40 partitions Any ideas on how to get the cluster to remove this node? Thanks, Paul _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
