Author: omalley
Date: Mon Aug 11 13:52:37 2008
New Revision: 684919
URL: http://svn.apache.org/viewvc?rev=684919&view=rev
Log:
HADOOP-3931. Fix corner case in the map-side sort that causes some values
to be counted as too large and cause pre-mature spills to disk. Some values
will also bypass the combiner incorrectly. (cdouglas via omalley)
Modified:
hadoop/core/branches/branch-0.17/CHANGES.txt
hadoop/core/branches/branch-0.17/src/java/org/apache/hadoop/mapred/MapTask.java
Modified: hadoop/core/branches/branch-0.17/CHANGES.txt
URL:
http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.17/CHANGES.txt?rev=684919&r1=684918&r2=684919&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.17/CHANGES.txt (original)
+++ hadoop/core/branches/branch-0.17/CHANGES.txt Mon Aug 11 13:52:37 2008
@@ -38,6 +38,10 @@
HADOOP-3859. Allow the maximum number of xceivers in the data node to
be configurable. (Johan Oskarsson via omalley)
+ HADOOP-3931. Fix corner case in the map-side sort that causes some values
+ to be counted as too large and cause pre-mature spills to disk. Some values
+ will also bypass the combiner incorrectly. (cdouglas via omalley)
+
Release 0.17.1 - 2008-06-23
INCOMPATIBLE CHANGES
Modified:
hadoop/core/branches/branch-0.17/src/java/org/apache/hadoop/mapred/MapTask.java
URL:
http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.17/src/java/org/apache/hadoop/mapred/MapTask.java?rev=684919&r1=684918&r2=684919&view=diff
==============================================================================
---
hadoop/core/branches/branch-0.17/src/java/org/apache/hadoop/mapred/MapTask.java
(original)
+++
hadoop/core/branches/branch-0.17/src/java/org/apache/hadoop/mapred/MapTask.java
Mon Aug 11 13:52:37 2008
@@ -435,7 +435,7 @@
try {
int keystart = bufindex;
keySerializer.serialize(key);
- if (bufindex < keystart || bufindex == bufvoid) {
+ if (bufindex < keystart) {
// wrapped the key; reset required
bb.reset();
keystart = 0;
