Hello,
Reading on the Internet best practices for selecting hardware for Hadoop
I noticed there are always many RAM memory. On my Hadoop environment I
have 16GB memory on all hardware, but I am worried about small
utilization of it:
$ free -m
total used free shared buffers cached
Mem: 15997 15907 90 0 287 15064
-/+ buffers/cache: 555 15442
Swap: 15258 150 15108
$ free -m
total used free shared buffers cached
Mem: 16029 15937 92 0 228 14320
-/+ buffers/cache: 1388 14641
Swap: 15258 1017 14240
As you see "buffers used" is below 10%. What options should I look
closer? I changed "Heap Size" of Cluster, but utilization doesn't grow
(Heap Size is 70.23 MB / 3.47 GB (1%)).
Current config which can impact on memory:
<name>fs.inmemory.size.mb</name>
<value>200</value>
<name>io.sort.mb</name>
<value>200</value>
<name>io.file.buffer.size</name>
<value>131072</value>
<name>dfs.block.size</name>
<value>134217728</value>
<name>mapred.child.java.opts</name>
<value>-Xmx1024M</value>
export HADOOP_HEAPSIZE=4000
Thanks for your reply,
K.R.
