[
https://issues.apache.org/jira/browse/MAPREDUCE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13091476#comment-13091476
]
Matei Zaharia commented on MAPREDUCE-2836:
------------------------------------------
Hi Ahmed,
The patch looks like a good start, but there are a few issues with it:
* There's a typo in the name "mapred.fairscheuler.createpool" (missing D in
fairscheduler). Actually, I don't think the name "createpool" is great in
general; maybe something like mapred.fairscheduler.allowUndeclaredPools would
be more descriptive.
* Throwing an IllegalArgumentException in a running jobtracker would crash the
whole JobTracker, which is not what we want. There has to be some code, maybe
in the job initialization listener, that catches the exception and calls fail()
on the JobInProgress.
* It's a little weird to be throwing and catching IllegalArgumentException as
part of a normal code path. At the very least, create a new exception type
(e.g. UndeclaredPoolException), but in fact it would be better to refactor the
code so that it doesn't rely on throwing exceptions. For example, you could add
a PoolManager.isValidPool(String name) method and call it on new jobs.
> Provide option to fail jobs when submitted to non-existent pools.
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-2836
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2836
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/fair-share
> Reporter: Jeff Bean
> Assignee: Ahmed Radwan
> Priority: Minor
> Attachments: MAPREDUCE-2836.patch
>
>
> In some environments, it might be desirable to explicitly specify the fair
> scheduler pools and to explicitly fail jobs that are not submitted to any of
> the pools.
> Current behavior of the fair scheduler is to submit jobs to a default pool if
> a pool name isn't specified or to create a pool with the new name if the pool
> name doesn't already exist. There should be a configuration option for the
> fair scheduler that causes it to noisily fail the job if it's submitted to a
> pool that isn't pre-specified or if the specified pool doesn't exist.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
