> From: tim robertson
> So then I need to really look at what memory footprint the
> custom MR jobs I run need to determine the "jobs per node"
> right? E.g. with 7G per node, minus the 2G reserved, if I
> need jobs with -Xmx1G I can run max 5, but safely 4...
> sound reasonable?
Yes that sounds reasonable.
What you want to avoid is swapping. Both the Hadoop and HBase
daemons have an number of internal heartbeats and timers, and
swapping can interfere with the Java scheduler, leading to
excessively long sleeps, missed heartbeats, and knock-on
effects as timeouts cause recovery actions which further
increase load.
- Andy
