Thomas,

I just replicated your setup (at least for the PR gets) and you can
indeed violate PR/PW when you pause a node on a VM. The reason this
happens is that riak's check for PR/PW simply looks at the ring's
preflist for a partition and checks that the required number of
partitions for that preflist are marked as primaries.

Now, when you pause a VM you interrupt any TCP connections that node has
open, just like if you unplugged the network cable, but not like if the
OS shut down or riak itself crashed. In those cases a FIN packet is sent
so that the other erlang nodes notice that their persistant connections
to that node have been reset, they will then reassign ownership of the
partitions owned by that downed node and PR/PW will start to fail.

However, since FIN packets are not generated when you pause the VM, it
takes a few moments for the erlang network heartbeat stuff to notice
that the node is down, so the preflists aren't recalculated. This is the
window where you see the mysterious behaviour.

Now, this is arguably a bug, although fixing it might be challenging.
I've filed https://issues.basho.com/show_bug.cgi?id=1318 to track this.

I don't have a workaround that I can think of offhand, unfortunately.

Andrew

_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Reply via email to