Author: todd
Date: Tue Mar 8 23:51:40 2011
New Revision: 1079606
URL: http://svn.apache.org/viewvc?rev=1079606&view=rev
Log:
HADOOP-7154. Should set MALLOC_ARENA_MAX in hadoop-env.sh. Contributed by Todd
Lipcon.
Modified:
hadoop/common/branches/branch-0.22/CHANGES.txt
hadoop/common/branches/branch-0.22/bin/hadoop-config.sh
Modified: hadoop/common/branches/branch-0.22/CHANGES.txt
URL:
http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.22/CHANGES.txt?rev=1079606&r1=1079605&r2=1079606&view=diff
==============================================================================
--- hadoop/common/branches/branch-0.22/CHANGES.txt (original)
+++ hadoop/common/branches/branch-0.22/CHANGES.txt Tue Mar 8 23:51:40 2011
@@ -221,6 +221,8 @@ Release 0.22.0 - Unreleased
HADOOP-6970. SecurityAuth.audit should be generated under /build (boryas)
+ HADOOP-7154. Should set MALLOC_ARENA_MAX in hadoop-env.sh (todd)
+
OPTIMIZATIONS
HADOOP-6884. Add LOG.isDebugEnabled() guard for each LOG.debug(..).
Modified: hadoop/common/branches/branch-0.22/bin/hadoop-config.sh
URL:
http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.22/bin/hadoop-config.sh?rev=1079606&r1=1079605&r2=1079606&view=diff
==============================================================================
--- hadoop/common/branches/branch-0.22/bin/hadoop-config.sh (original)
+++ hadoop/common/branches/branch-0.22/bin/hadoop-config.sh Tue Mar 8 23:51:40
2011
@@ -78,6 +78,11 @@ then
exit 1
fi
+# Newer versions of glibc use an arena memory allocator that causes virtual
+# memory usage to explode. This interacts badly with the many threads that
+# we use in Hadoop. Tune the variable down to prevent vmem explosion.
+export MALLOC_ARENA_MAX=${MALLOC_ARENA_MAX:-4}
+
# some Java parameters
if [ "$JAVA_HOME" != "" ]; then
#echo "run java in $JAVA_HOME"
