[ https://issues.apache.org/jira/browse/MAPREDUCE-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer updated MAPREDUCE-6063: ---------------------------------------- Fix Version/s: (was: 3.0.0) > In sortAndSpill of MapTask.java, size is calculated wrongly when bufend < > bufstart. > ----------------------------------------------------------------------------------- > > Key: MAPREDUCE-6063 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6063 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 > Reporter: zhihai xu > Assignee: zhihai xu > Fix For: 2.6.0 > > Attachments: MAPREDUCE-6063.000.patch, MAPREDUCE-6063.branch-1.patch > > > In sortAndSpill of MapTask.java, size is calculated wrongly when bufend < > bufstart. we should change (bufvoid - bufend) + bufstart to (bufvoid - > bufstart) + bufend. > Should change > {code} > long size = (bufend >= bufstart > ? bufend - bufstart > : (bufvoid - bufend) + bufstart) + > partitions * APPROX_HEADER_LENGTH; > {code} > to: > {code} > long size = (bufend >= bufstart > ? bufend - bufstart > : (bufvoid - bufstart) + bufend) + > partitions * APPROX_HEADER_LENGTH; > {code} > It is because when wraparound happen (bufend < bufstart) , the size should > bufvoid - bufstart (bigger one) + bufend(small one). > You can find similar code implementation in MapTask.java: > {code} > mapOutputByteCounter.increment(valend >= keystart > ? valend - keystart > : (bufvoid - keystart) + valend); > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)