Hi All, The following is about leveldb, riak (1.1.1 2012-03-07) RedHat x86_64, and one riak newbie known as me!
I had this problem over the weekend whereby two nodes are leaving and they are both stuck trying to send transfers from one to the other. I had backed them up with tar, and after they became stuck, I tried launching new instances with those levedb tar file backups. But those new hosts, although listed in connected_nodes, are not in the ring_members. Are there any work arounds to resolving the stuck ownership handoff between the two leaving nodes? I tried different scenarios of marking them as down. That didn't seem to help. ring_status indicated it wanted them down, then it wanted them back online. etc. I don't really need either one. I'd like to eject them both from the cluster and have it rebalance onto the new nodes. Both these were asked to leave: Owner: [email protected]<mailto:[email protected]> Next Owner: [email protected]<mailto:[email protected]> ring_status output: [root@ip-10-0-0-171 riak]# riak-admin ring_status Attempting to restart script through sudo -u riak ================================== Claimant =================================== Claimant: '[email protected]<mailto:[email protected]>' Status: up Ring Ready: false ============================== Ownership Handoff ============================== Owner: [email protected]<mailto:[email protected]> Next Owner: [email protected]<mailto:[email protected]> Index: 727896323280039539339725844419242519555200778240 Waiting on: [riak_kv_vnode] Complete: [riak_pipe_vnode,riak_search_vnode] Index: 876330083321459366969787585241990013739006427136 Waiting on: [riak_kv_vnode] Complete: [riak_pipe_vnode,riak_search_vnode] ------------------------------------------------------------------------------- ============================== Unreachable Nodes ============================== All nodes are up and reachable And member_status output: [root@ip-10-0-0-168 ~]# riak-admin member_status Attempting to restart script through sudo -u riak ================================= Membership ================================== Status Ring Pending Node ------------------------------------------------------------------------------- down 16.6% 16.6% '[email protected]<mailto:[email protected]>' leaving 0.2% 0.0% '[email protected]<mailto:[email protected]>' leaving 15.6% 15.8% '[email protected]<mailto:[email protected]>' valid 0.0% 0.0% '[email protected]<mailto:[email protected]>' valid 0.0% 0.0% '[email protected]<mailto:[email protected]>' valid 18.0% 18.0% '[email protected]<mailto:[email protected]>' valid 17.4% 17.4% '[email protected]<mailto:[email protected]>' valid 15.4% 15.4% '[email protected]<mailto:[email protected]>' valid 16.8% 16.8% '[email protected]<mailto:[email protected]>' ------------------------------------------------------------------------------- Valid:6 / Leaving:1 / Exiting:0 / Joining:1 / Down:1 Thanks for your input! Austin
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
