[
https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-3812:
-------------------------------
Attachment: MAPREDUCE-3812.patch
Here is a fresh rewrite patch as the last one available here was out of sync
and I figured its faster rewriting changes and gathering MAPREDUCE-4026 and
MAPREDUCE-4027 rather than trying to resolve the out-of-sync conflicts.
I've not touched some tests as I felt they were unnecessary. I ran 4 tests for
this change: TestFifoScheduler, TestRMWebServices, TestRM,
TestApplicationCleanup and all of them pass.
> Lower default allocation sizes, fix allocation configurations and document
> them
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3812
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, performance
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Harsh J
> Attachments: MAPREDUCE-3812-20120205.txt,
> MAPREDUCE-3812-20120206.1.txt, MAPREDUCE-3812-20120206.txt,
> MAPREDUCE-3812.patch, MAPREDUCE-3812.patch, MAPREDUCE-3812.patch
>
>
> After a few performance improvements tracked at MAPREDUCE-3561, like
> MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within
> 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work
> around the issues with AM heap. Now that those are fixed, we should go back
> to 1GB.
> This is just a configuration change.
> [P.s.]:
> - Currently min/max alloc is set at a per-scheduler config level, which makes
> no sense as there's no way to run multiple schedulers anyway. Switch configs
> to use a generic RM-config.
> - The min/max alloc configs aren't documented and we ought to document it
> (i.e. MAPREDUCE-4027)
> - 1 GB is perhaps too high for a slot's minimum. While job defaults can be
> left at such values, we should lower the minimum alloc to 128 MB to allow
> special requests of low memory out of the box itself. Shouldn't impact MR App
> in any way.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
