Author: shv
Date: Fri Jan 28 22:42:37 2011
New Revision: 1064918
URL: http://svn.apache.org/viewvc?rev=1064918&view=rev
Log:
HADOOP-6812. Merge -r 1064916:1064917 from trunk to branch 0.22.
Modified:
hadoop/common/branches/branch-0.22/CHANGES.txt
hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/cluster_setup.xml
hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/hod_scheduler.xml
Modified: hadoop/common/branches/branch-0.22/CHANGES.txt
URL:
http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.22/CHANGES.txt?rev=1064918&r1=1064917&r2=1064918&view=diff
==============================================================================
--- hadoop/common/branches/branch-0.22/CHANGES.txt (original)
+++ hadoop/common/branches/branch-0.22/CHANGES.txt Fri Jan 28 22:42:37 2011
@@ -209,6 +209,11 @@ Release 0.22.0 - Unreleased
HADOOP-7110. Implement chmod with JNI. (todd)
+ HADOOP-6812. Change documentation for correct placement of configuration
+ variables: mapreduce.reduce.input.buffer.percent,
+ mapreduce.task.io.sort.factor, mapreduce.task.io.sort.mb
+ (Chris Douglas via shv)
+
OPTIMIZATIONS
HADOOP-6884. Add LOG.isDebugEnabled() guard for each LOG.debug(..).
Modified:
hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/cluster_setup.xml
URL:
http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/cluster_setup.xml?rev=1064918&r1=1064917&r2=1064918&view=diff
==============================================================================
---
hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/cluster_setup.xml
(original)
+++
hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/cluster_setup.xml
Fri Jan 28 22:42:37 2011
@@ -666,22 +666,33 @@
</td>
</tr>
<tr>
- <td>conf/core-site.xml</td>
- <td>fs.inmemory.size.mb</td>
- <td>200</td>
+ <td>conf/mapred-site.xml</td>
+ <td>mapreduce.reduce.shuffle.input.buffer.percent</td>
+ <td>0.80</td>
<td>
- Larger amount of memory allocated for the in-memory
- file-system used to merge map-outputs at the reduces.
+ Larger amount of memory allocated for merging map output
+ in memory during the shuffle. Expressed as a fraction of
+ the total heap.
</td>
</tr>
<tr>
- <td>conf/core-site.xml</td>
+ <td>conf/mapred-site.xml</td>
+ <td>mapreduce.reduce.input.buffer.percent</td>
+ <td>0.80</td>
+ <td>
+ Larger amount of memory allocated for retaining map
output
+ in memory during the reduce. Expressed as a fraction of
+ the total heap.
+ </td>
+ </tr>
+ <tr>
+ <td>conf/mapred-site.xml</td>
<td>mapreduce.task.io.sort.factor</td>
<td>100</td>
<td>More streams merged at once while sorting files.</td>
</tr>
<tr>
- <td>conf/core-site.xml</td>
+ <td>conf/mapred-site.xml</td>
<td>mapreduce.task.io.sort.mb</td>
<td>200</td>
<td>Higher memory-limit while sorting data.</td>
Modified:
hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/hod_scheduler.xml
URL:
http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/hod_scheduler.xml?rev=1064918&r1=1064917&r2=1064918&view=diff
==============================================================================
---
hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/hod_scheduler.xml
(original)
+++
hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/hod_scheduler.xml
Fri Jan 28 22:42:37 2011
@@ -250,7 +250,7 @@ info_port = Port number of the HDFS Name
<p>For example:</p>
<source>
server-params =
mapred.reduce.parallel.copies=20,io.sort.factor=100,io.sort.mb=128,io.file.buffer.size=131072
-final-server-params =
mapred.child.java.opts=-Xmx512m,dfs.block.size=134217728,fs.inmemory.size.mb=128
+final-server-params = mapred.child.java.opts=-Xmx512m,dfs.block.size=134217728
</source>
<p>In order to provide the options from command line, you can use the
following syntax:</p>
<p>For configuring the MapReduce daemons use:</p>
