The thing that seems odd to me is that the connectivity complaints are
out of the zk client, right?, why is it failing getting to member 14
and why not move to another ensemble member if issue w/ 14?, and if
there were a general connectivity issue, I'd think that the running
hbase cluster would be complaining at about the same time (its talking
to datanodes and masters at this time).
(Thanks for the input lads)
On Mon, Feb 22, 2010 at 11:26 AM, Mahadev Konar <maha...@yahoo-inc.com> wrote:
> I also looked at the logs. Ted might have a point. It does look like that
> zookeeper server's are doing fine (though as ted mentions the skew is a
> little concerning, though that might be due to very few packets served by
> the first server). Other than that the latencies of 300 ms at max should not
> cause any timeouts.
> Also, the number of packets received is pretty low - meaning that it wasn't
> serving huge traffic. Is there anyway we can check if the network connection
> from the client to the server is not flaky?
> On 2/22/10 10:40 AM, "Ted Dunning" <ted.dunn...@gmail.com> wrote:
>> Not sure this helps at all, but these times are remarkably asymmetrical. I
>> would expect members of a ZK cluster to have very comparable times.
>> Additionally, 345 ms is nowhere near large enough to cause a session to
>> expire. My take is that ZK doesn't think it caused the timeout.
>> On Mon, Feb 22, 2010 at 10:18 AM, Stack <st...@duboce.net> wrote:
>>> Latency min/avg/max: 2/125/345
>>> Latency min/avg/max: 0/7/81
>>> Latency min/avg/max: 1/1/1
>>> Thanks for any pointers on how to debug.