feedback on trunk code ... i'm using code from trunk, 4 node cluster, RF=3, W=Q, R=Q and did a nodeprobe loadbalance on the hot node. the cluster ran through the night and this morning i noticed a lot of pending hinted-handoff-pool via tpstats. looking thru the logs i notice that the nodes are very frequently "InetAddress /192.168.132.102 is now dead" then back UP .. flapping of the nodes.
is this triggering the hinted-handoff and is there a way to alleviate this (other than lowering the load)? it seems that the hinted handdoffs have slowed or stopped the loadbalance. i've been trying to load up the cluster with data, so i'm trying to do 2000+ puts a second with little or no reads. it seems like the node that is loadbalancing is bringing the performance of the cluster way down.
