[jira] [Updated] (MAPREDUCE-6063) In sortAndSpill of MapTask.java, size is calculated wrongly when bufend bufstart.

2014-11-18 Thread jiangyu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jiangyu updated MAPREDUCE-6063:
---
Description: 
In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
bufstart.  we should change (bufvoid - bufend) + bufstart to (bufvoid - 
bufstart) + bufend.
Should change
{code}
 long size = (bufend = bufstart
  ? bufend - bufstart
  : (bufvoid - bufend) + bufstart) +
  partitions * APPROX_HEADER_LENGTH;
{code}
to:
{code}
 long size = (bufend = bufstart
  ? bufend - bufstart
  : (bufvoid - bufstart) + bufend) +
  partitions * APPROX_HEADER_LENGTH;dd
{code}

It is because when wraparound happen (bufend  bufstart) ,  the size should 
bufvoid - bufstart (bigger one) + bufend(small one).
You can find similar code implementation in MapTask.java:
{code}
mapOutputByteCounter.increment(valend = keystart
? valend - keystart
: (bufvoid - keystart) + valend);
{code}


  was:
In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
bufstart.  we should change (bufvoid - bufend) + bufstart to (bufvoid - 
bufstart) + bufend.
Should change
{code}
 long size = (bufend = bufstart
  ? bufend - bufstart
  : (bufvoid - bufend) + bufstart) +
  partitions * APPROX_HEADER_LENGTH;
{code}
to:
{code}
 long size = (bufend = bufstart
  ? bufend - bufstart
  : (bufvoid - bufstart) + bufend) +
  partitions * APPROX_HEADER_LENGTH;
{code}

It is because when wraparound happen (bufend  bufstart) ,  the size should 
bufvoid - bufstart (bigger one) + bufend(small one).
You can find similar code implementation in MapTask.java:
{code}
mapOutputByteCounter.increment(valend = keystart
? valend - keystart
: (bufvoid - keystart) + valend);
{code}



 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.
 ---

 Key: MAPREDUCE-6063
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6063
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv1, mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Fix For: 2.6.0

 Attachments: MAPREDUCE-6063.000.patch, MAPREDUCE-6063.branch-1.patch


 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.  we should change (bufvoid - bufend) + bufstart to (bufvoid - 
 bufstart) + bufend.
 Should change
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufend) + bufstart) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 to:
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufstart) + bufend) +
   partitions * APPROX_HEADER_LENGTH;dd
 {code}
 It is because when wraparound happen (bufend  bufstart) ,  the size should 
 bufvoid - bufstart (bigger one) + bufend(small one).
 You can find similar code implementation in MapTask.java:
 {code}
 mapOutputByteCounter.increment(valend = keystart
 ? valend - keystart
 : (bufvoid - keystart) + valend);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6063) In sortAndSpill of MapTask.java, size is calculated wrongly when bufend bufstart.

2014-11-18 Thread jiangyu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jiangyu updated MAPREDUCE-6063:
---
Description: 
In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
bufstart.  we should change (bufvoid - bufend) + bufstart to (bufvoid - 
bufstart) + bufend.
Should change
{code}
 long size = (bufend = bufstart
  ? bufend - bufstart
  : (bufvoid - bufend) + bufstart) +
  partitions * APPROX_HEADER_LENGTH;
{code}
to:
{code}
 long size = (bufend = bufstart
  ? bufend - bufstart
  : (bufvoid - bufstart) + bufend) +
  partitions * APPROX_HEADER_LENGTH;
{code}

It is because when wraparound happen (bufend  bufstart) ,  the size should 
bufvoid - bufstart (bigger one) + bufend(small one).
You can find similar code implementation in MapTask.java:
{code}
mapOutputByteCounter.increment(valend = keystart
? valend - keystart
: (bufvoid - keystart) + valend);
{code}


  was:
In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
bufstart.  we should change (bufvoid - bufend) + bufstart to (bufvoid - 
bufstart) + bufend.
Should change
{code}
 long size = (bufend = bufstart
  ? bufend - bufstart
  : (bufvoid - bufend) + bufstart) +
  partitions * APPROX_HEADER_LENGTH;
{code}
to:
{code}
 long size = (bufend = bufstart
  ? bufend - bufstart
  : (bufvoid - bufstart) + bufend) +
  partitions * APPROX_HEADER_LENGTH;dd
{code}

It is because when wraparound happen (bufend  bufstart) ,  the size should 
bufvoid - bufstart (bigger one) + bufend(small one).
You can find similar code implementation in MapTask.java:
{code}
mapOutputByteCounter.increment(valend = keystart
? valend - keystart
: (bufvoid - keystart) + valend);
{code}



 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.
 ---

 Key: MAPREDUCE-6063
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6063
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv1, mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Fix For: 2.6.0

 Attachments: MAPREDUCE-6063.000.patch, MAPREDUCE-6063.branch-1.patch


 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.  we should change (bufvoid - bufend) + bufstart to (bufvoid - 
 bufstart) + bufend.
 Should change
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufend) + bufstart) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 to:
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufstart) + bufend) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 It is because when wraparound happen (bufend  bufstart) ,  the size should 
 bufvoid - bufstart (bigger one) + bufend(small one).
 You can find similar code implementation in MapTask.java:
 {code}
 mapOutputByteCounter.increment(valend = keystart
 ? valend - keystart
 : (bufvoid - keystart) + valend);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6063) In sortAndSpill of MapTask.java, size is calculated wrongly when bufend bufstart.

2014-09-09 Thread Allen Wittenauer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Allen Wittenauer updated MAPREDUCE-6063:

Fix Version/s: (was: 3.0.0)

 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.
 ---

 Key: MAPREDUCE-6063
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6063
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv1, mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Fix For: 2.6.0

 Attachments: MAPREDUCE-6063.000.patch, MAPREDUCE-6063.branch-1.patch


 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.  we should change (bufvoid - bufend) + bufstart to (bufvoid - 
 bufstart) + bufend.
 Should change
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufend) + bufstart) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 to:
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufstart) + bufend) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 It is because when wraparound happen (bufend  bufstart) ,  the size should 
 bufvoid - bufstart (bigger one) + bufend(small one).
 You can find similar code implementation in MapTask.java:
 {code}
 mapOutputByteCounter.increment(valend = keystart
 ? valend - keystart
 : (bufvoid - keystart) + valend);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6063) In sortAndSpill of MapTask.java, size is calculated wrongly when bufend bufstart.

2014-09-03 Thread Chris Douglas (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Douglas updated MAPREDUCE-6063:
-
   Resolution: Fixed
Fix Version/s: 2.6.0
   3.0.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

+1 Good catch

I committed this to trunk and branch-2

 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.
 ---

 Key: MAPREDUCE-6063
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6063
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv1, mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Fix For: 3.0.0, 2.6.0

 Attachments: MAPREDUCE-6063.000.patch


 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.  we should change (bufvoid - bufend) + bufstart to (bufvoid - 
 bufstart) + bufend.
 Should change
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufend) + bufstart) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 to:
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufstart) + bufend) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 It is because when wraparound happen (bufend  bufstart) ,  the size should 
 bufvoid - bufstart (bigger one) + bufend(small one).
 You can find similar code implementation in MapTask.java:
 {code}
 mapOutputByteCounter.increment(valend = keystart
 ? valend - keystart
 : (bufvoid - keystart) + valend);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6063) In sortAndSpill of MapTask.java, size is calculated wrongly when bufend bufstart.

2014-09-03 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6063:
-
Attachment: MAPREDUCE-6063.branch-1.patch

 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.
 ---

 Key: MAPREDUCE-6063
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6063
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv1, mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Fix For: 3.0.0, 2.6.0

 Attachments: MAPREDUCE-6063.000.patch, MAPREDUCE-6063.branch-1.patch


 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.  we should change (bufvoid - bufend) + bufstart to (bufvoid - 
 bufstart) + bufend.
 Should change
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufend) + bufstart) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 to:
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufstart) + bufend) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 It is because when wraparound happen (bufend  bufstart) ,  the size should 
 bufvoid - bufstart (bigger one) + bufend(small one).
 You can find similar code implementation in MapTask.java:
 {code}
 mapOutputByteCounter.increment(valend = keystart
 ? valend - keystart
 : (bufvoid - keystart) + valend);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6063) In sortAndSpill of MapTask.java, size is calculated wrongly when bufend bufstart.

2014-09-02 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6063:
-
Attachment: MAPREDUCE-6063.000.patch

 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.
 ---

 Key: MAPREDUCE-6063
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6063
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv1, mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6063.000.patch


 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.  we should change (bufvoid - bufend) + bufstart to (bufvoid - 
 bufstart) + bufend.
 Should change
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufend) + bufstart) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 to:
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufstart) + bufend) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 It is because when wraparound happen (bufend  bufstart) ,  the size should 
 bufvoid - bufstart (bigger one) + bufend(small one).
 You can find similar code implementation in MapTask.java:
 {code}
 mapOutputByteCounter.increment(valend = keystart
 ? valend - keystart
 : (bufvoid - keystart) + valend);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6063) In sortAndSpill of MapTask.java, size is calculated wrongly when bufend bufstart.

2014-09-02 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6063:
-
Status: Patch Available  (was: Open)

I attached a patch for review.

 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.
 ---

 Key: MAPREDUCE-6063
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6063
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv1, mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6063.000.patch


 In sortAndSpill of MapTask.java, size is calculated wrongly when bufend  
 bufstart.  we should change (bufvoid - bufend) + bufstart to (bufvoid - 
 bufstart) + bufend.
 Should change
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufend) + bufstart) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 to:
 {code}
  long size = (bufend = bufstart
   ? bufend - bufstart
   : (bufvoid - bufstart) + bufend) +
   partitions * APPROX_HEADER_LENGTH;
 {code}
 It is because when wraparound happen (bufend  bufstart) ,  the size should 
 bufvoid - bufstart (bigger one) + bufend(small one).
 You can find similar code implementation in MapTask.java:
 {code}
 mapOutputByteCounter.increment(valend = keystart
 ? valend - keystart
 : (bufvoid - keystart) + valend);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)