[
https://issues.apache.org/jira/browse/MAPREDUCE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated MAPREDUCE-3230:
-----------------------------------------------
Description:
Size of cluster is 350 NMs. I have topology.node.switch.mapping.impl set to
enable rack-locality. Ran randomwriter/sort and scan jobs. Both jobs ran
completed successfully.
On Job client sort job says :
{code}
Launched map tasks=16800
Launched reduce tasks=700
Other local map tasks=10
Rack-local map tasks=16790
{code}
JobHistory files also don't have information about Data Local Maps.
There used to be information about data local maps before, till about a month
back, like so:
For sort with 349 NMs :
{code}
Launched map tasks=16754
Launched reduce tasks=700
Other local map tasks=48
Data-local map tasks=16459
Rack-local map tasks=248
{code}
was:
Size of cluster is 350 NMs.
I have topology.node.switch.mapping.impl is set to enable rack-locality.
Ran randomwriter/sort and scan jobs.
Both jobs ran
Launched map tasks=16800
On Job client sort says -:
Launched map tasks=16800
Launched reduce tasks=700
Other local map tasks=10
Rack-local map tasks=16790
On Job client scan says -:
Launched map tasks=16800
Other local map tasks=50
Rack-local map tasks=16750
JobHistory files also does having information of Data Local Maps
There is information data local maps, last month, when I checked out code
branch-0.23
data local maps was coming
e.g.
For sort it was with 349 NMs-:
Launched map tasks=16754
Launched reduce tasks=700
Other local map tasks=48
Data-local map tasks=16459
Rack-local map tasks=248
For scan it was with 349 NMs -:
Launched map tasks=16752
Other local map tasks=27
Data-local map tasks=16532
Rack-local map tasks=193
Summary: No information about Data Local maps from Job client CLI and
in JobHistory (was: Data Local maps information not coming on Job client CLI
or in JobHistory )
Thanks for filing this, Karam!
> No information about Data Local maps from Job client CLI and in JobHistory
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3230
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3230
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mrv2
> Affects Versions: 0.23.0
> Reporter: Karam Singh
> Fix For: 0.23.0
>
>
> Size of cluster is 350 NMs. I have topology.node.switch.mapping.impl set to
> enable rack-locality. Ran randomwriter/sort and scan jobs. Both jobs ran
> completed successfully.
> On Job client sort job says :
> {code}
> Launched map tasks=16800
> Launched reduce tasks=700
> Other local map tasks=10
> Rack-local map tasks=16790
> {code}
> JobHistory files also don't have information about Data Local Maps.
> There used to be information about data local maps before, till about a month
> back, like so:
> For sort with 349 NMs :
> {code}
> Launched map tasks=16754
> Launched reduce tasks=700
> Other local map tasks=48
> Data-local map tasks=16459
> Rack-local map tasks=248
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
