On Mon, Sep 12, 2011 at 11:27 PM, Geoff Hendrey <[email protected]> wrote: > Do you have any advice on what to look for (or how to sort it) when I do > lsof or netstat? A glance at it doesn't show any "standouts" but then > I'm not entirely sure what to look for. I see lots of connections to > various nodes in the cluster, from any given node, but I suppose that's > quite normal.
Yeah. On the slow RS, check who its talking too... take a look at a few of the nodes referenced. Check dmesg across your cluster see if any complaining. > Ganglia offers no clues either. It's pretty uniform for > all graphs across all servers. > No anomalies around datanodes? Spikes or troughs? Welcome to the joys of distributed computing. Once you figure whats going on, you'll be able to enable an alert for the future but meantime its no fun. St.Ack
