[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong "base" parameter to getMapOutputInfo if mapId is not in the cache.

2017-01-05 Thread Junping Du (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Junping Du updated MAPREDUCE-6425:
--
Fix Version/s: 2.8.0

> ShuffleHandler passes wrong "base" parameter to getMapOutputInfo if mapId is 
> not in the cache.
> --
>
> Key: MAPREDUCE-6425
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: mrv2, nodemanager
>Reporter: zhihai xu
>Assignee: zhihai xu
> Fix For: 2.8.0, 2.7.2, 3.0.0-alpha1
>
> Attachments: MAPREDUCE-6425.000.patch, MAPREDUCE-6425.001.patch
>
>
> ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
> mapId is not in the cache.
> {{getMapOutputInfo}} expected the {{base}} parameter is 
> {{getBaseLocation(jobId, user) + mapId}}
> When it is called inside populateHeaders, the {{base}} parameter is set 
> correctly
> {code}
> String base = outputBaseStr + mapId;
> MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
> user);
> {code}
> When  it is called outside populateHeaders, the {{base}} parameter is set 
> wrongly to outputBasePathStr after number of mapId cached exceeds 
> {{mapOutputMetaInfoCacheSize}}.
> {code}
>  String outputBasePathStr = getBaseLocation(jobId, user);
>   MapOutputInfo info = mapOutputInfoMap.get(mapId);
>   if (info == null) {
> info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-06 Thread Devaraj K (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj K updated MAPREDUCE-6425:
-
Hadoop Flags: Reviewed

Thanks [~zxu] for the updated patch.

+1, latest patch looks good to me, will commit it shortly.

 ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is 
 not in the cache.
 --

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6425.000.patch, MAPREDUCE-6425.001.patch


 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 {{getMapOutputInfo}} expected the {{base}} parameter is 
 {{getBaseLocation(jobId, user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-06 Thread Devaraj K (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj K updated MAPREDUCE-6425:
-
   Resolution: Fixed
Fix Version/s: 2.7.2
   Status: Resolved  (was: Patch Available)

Thanks [~zxu].

Committed  to trunk, branch-2 and branch-2.7.

 ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is 
 not in the cache.
 --

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu
 Fix For: 2.7.2

 Attachments: MAPREDUCE-6425.000.patch, MAPREDUCE-6425.001.patch


 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 {{getMapOutputInfo}} expected the {{base}} parameter is 
 {{getBaseLocation(jobId, user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-04 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6425:
-
Attachment: MAPREDUCE-6425.001.patch

 ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is 
 not in the cache.
 --

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6425.000.patch, MAPREDUCE-6425.001.patch


 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 {{getMapOutputInfo}} expected the {{base}} parameter is 
 {{getBaseLocation(jobId, user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-04 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6425:
-
Attachment: (was: MAPREDUCE-6425.001.patch)

 ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is 
 not in the cache.
 --

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6425.000.patch


 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 {{getMapOutputInfo}} expected the {{base}} parameter is 
 {{getBaseLocation(jobId, user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-03 Thread Devaraj K (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj K updated MAPREDUCE-6425:
-
Target Version/s: 2.7.2

 ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is 
 not in the cache.
 --

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6425.000.patch


 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 {{getMapOutputInfo}} expected the {{base}} parameter is 
 {{getBaseLocation(jobId, user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-03 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6425:
-
Attachment: MAPREDUCE-6425.001.patch

 ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is 
 not in the cache.
 --

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6425.000.patch, MAPREDUCE-6425.001.patch


 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 {{getMapOutputInfo}} expected the {{base}} parameter is 
 {{getBaseLocation(jobId, user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-02 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6425:
-
Summary: ShuffleHandler passes wrong base parameter to getMapOutputInfo 
if mapId is not in the cache.  (was: ShuffleHandler passes wrong {{base}} 
parameter to getMapOutputInfo if mapId is not in the cache.)

 ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is 
 not in the cache.
 --

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6425.000.patch


 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 {{getMapOutputInfo}} expected the {{base}} parameter is 
 {{getBaseLocation(jobId, user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-02 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6425:
-
Attachment: MAPREDUCE-6425.000.patch

 ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is 
 not in the cache.
 

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6425.000.patch


 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 getMapOutputInfo expected the {{base}} parameter is {{getBaseLocation(jobId, 
 user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong {{base}} parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-02 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6425:
-
Description: 
ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if mapId 
is not in the cache.
{{getMapOutputInfo}} expected the {{base}} parameter is 
{{getBaseLocation(jobId, user) + mapId}}
When it is called inside populateHeaders, the {{base}} parameter is set 
correctly
{code}
String base = outputBaseStr + mapId;
MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, user);
{code}
When  it is called outside populateHeaders, the {{base}} parameter is set 
wrongly to outputBasePathStr after number of mapId cached exceeds 
{{mapOutputMetaInfoCacheSize}}.
{code}
 String outputBasePathStr = getBaseLocation(jobId, user);
  MapOutputInfo info = mapOutputInfoMap.get(mapId);
  if (info == null) {
info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
  }
{code}

  was:
ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if mapId 
is not in the cache.
getMapOutputInfo expected the {{base}} parameter is {{getBaseLocation(jobId, 
user) + mapId}}
When it is called inside populateHeaders, the {{base}} parameter is set 
correctly
{code}
String base = outputBaseStr + mapId;
MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, user);
{code}
When  it is called outside populateHeaders, the {{base}} parameter is set 
wrongly to outputBasePathStr after number of mapId cached exceeds 
{{mapOutputMetaInfoCacheSize}}.
{code}
 String outputBasePathStr = getBaseLocation(jobId, user);
  MapOutputInfo info = mapOutputInfoMap.get(mapId);
  if (info == null) {
info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
  }
{code}

Summary: ShuffleHandler passes wrong {{base}} parameter to 
getMapOutputInfo if mapId is not in the cache.  (was: ShuffleHandler passes 
wrong base parameter to getMapOutputInfo if mapId is not in the cache.)

 ShuffleHandler passes wrong {{base}} parameter to getMapOutputInfo if mapId 
 is not in the cache.
 

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6425.000.patch


 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 {{getMapOutputInfo}} expected the {{base}} parameter is 
 {{getBaseLocation(jobId, user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-02 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6425:
-
Attachment: (was: MAPREDUCE-6425.000.patch)

 ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is 
 not in the cache.
 --

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu

 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 {{getMapOutputInfo}} expected the {{base}} parameter is 
 {{getBaseLocation(jobId, user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-02 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6425:
-
Status: Patch Available  (was: Open)

 ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is 
 not in the cache.
 

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6425.000.patch


 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 getMapOutputInfo expected the {{base}} parameter is {{getBaseLocation(jobId, 
 user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6425) ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.

2015-07-02 Thread zhihai xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6425:
-
Attachment: MAPREDUCE-6425.000.patch

 ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is 
 not in the cache.
 --

 Key: MAPREDUCE-6425
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6425
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2, nodemanager
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6425.000.patch


 ShuffleHandler passes wrong {{base}} parameter to {{getMapOutputInfo}} if 
 mapId is not in the cache.
 {{getMapOutputInfo}} expected the {{base}} parameter is 
 {{getBaseLocation(jobId, user) + mapId}}
 When it is called inside populateHeaders, the {{base}} parameter is set 
 correctly
 {code}
 String base = outputBaseStr + mapId;
 MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, 
 user);
 {code}
 When  it is called outside populateHeaders, the {{base}} parameter is set 
 wrongly to outputBasePathStr after number of mapId cached exceeds 
 {{mapOutputMetaInfoCacheSize}}.
 {code}
  String outputBasePathStr = getBaseLocation(jobId, user);
   MapOutputInfo info = mapOutputInfoMap.get(mapId);
   if (info == null) {
 info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)