Also, it seems like Ganglia would be very well complemented by Nagios to allow you to monitor an overall health of your cluster. -- Take care, Konstantin (Cos) Boudnik 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
Disclaimer: Opinions expressed in this email are those of the author, and do not necessarily represent the views of any company the author might be affiliated with at the moment of writing. On Tue, May 17, 2011 at 15:15, Allen Wittenauer <a...@apache.org> wrote: > > On May 17, 2011, at 3:11 PM, Mark question wrote: > >> So what other memory consumption tools do you suggest? I don't want to do it >> manually and dump statistics into file because IO will affect performance >> too. > > We watch memory with Ganglia. We also tune our systems such that a > task will only take X amount. In other words, given an 8gb RAM: > > 1gb for the OS > 1gb for the TT and DN > 6gb for all tasks > > if we assume each task will take max 1gb, then we end up with 3 maps > and 3 reducers. > > Keep in mind that the mem consumed is more than just JVM heap size.
