[
https://issues.apache.org/jira/browse/MAPREDUCE-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255396#comment-13255396
]
Patrick Wendell commented on MAPREDUCE-3451:
--------------------------------------------
Thanks for the feedback. About to attach a patch with the following changes:
- Updated the terminology from “pool” to “queue” to be consistent.
- I added annotations to all classes.
- Fixed the debug → log statement
- I modified the web UI to give a more precise error message in the logs, but
it still has a generic 500 in the web page. MRv2 uses a custom built MVC
framework based on Guice dependency injection. I’m embarrassed to say that I
spend 40 minutes trying to figure out how to make it render a 404 and couldn’t.
It seems like it’s not easy to trigger a 404 from within a view, rather than at
the URL router.
- Extra license header removed
- Noted spelling errors fixed.
- There is extensive test coverage in the MRS FS I’d like to get there
eventually but probably not with this first commit. I tried to included tests
of most of the core functionality. We can add this as a JIRA, though.
- I consolidated the configuration constants as you suggested. For now I’m
going to leave the queue manifest file as its own thing unless this is a
deal-breaker. There is a lot of logic in parsing that file which I’d prefer to
leave as-is since it’s very well tested. While the term names have changed it’s
essentially identical configuration to the MR1 FS, so leaving things as-is
should ease the transition for people running current clusters. It’s also a
fundamentally hierarchical type of configuration, and the apache conf is all
flat key/value space which will be much more clunky.
I haven’t done much testing other than on a single node. One reason I want to
get this in is that some are waiting on having something in trunk to start
testing. The build is stable and the unit tests pass, so I hope that is
sufficient to get this in there with the caveat that things may be brittle when
run on a real cluster.
I also added an updated version of the documentation (even the MR1 version
wasn’t totally up-to-date with changes that have been made in the configuration
options). This will need some more work going forward but it should give people
to the basic configuration/installation info they need to get started testing
this.
> Port Fair Scheduler to MR2
> --------------------------
>
> Key: MAPREDUCE-3451
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3451
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: mrv2, scheduler
> Reporter: Patrick Wendell
> Assignee: Patrick Wendell
> Attachments: MAPREDUCE-3451.v1.patch.txt, MAPREDUCE-3451.v2.patch.txt
>
>
> The Fair Scheduler is in widespread use today in MR1 clusters, but not yet
> ported to MR2. This is to track the porting of the Fair Scheduler to MR2 and
> will be updated to include design considerations and progress.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
