[
https://issues.apache.org/jira/browse/MAPREDUCE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103232#comment-13103232
]
Todd Lipcon commented on MAPREDUCE-2836:
----------------------------------------
Hey Ahmed. Thanks for the revision. A few notes on the latest patch:
- you can make the {{allowUndeclaredPools}} member final, since it's set in the
constructor
- rather than making {{PoolManager.declaredPools}} public, add an accessor for
it
- the declaredPools set needs to be synchronized more carefully:
-- since you're exposing the Set<String> via a getter, it's probably better to
make a new instance every time you reload the pools. That way, the getter
returns just a snapshot that won't change. For example, you could do:
{code}
this.declaredPools = Collections.unmodifiableSet(new
TreeSet<String>(poolNamesInAllocFile));
{code}
and then make the getter synchronized. This makes it clear that the contents of
that set won't change, and you can expose it to other classes safely. The issue
with the current implementation is that a job may be submitted just as the set
is being modified, and you'd get a spurious job rejection.
- nit: consider using a TreeSet instead of HashSet, so that the pool names are
sorted in the error message
> Provide option to fail jobs when submitted to non-existent pools.
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-2836
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2836
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/fair-share
> Reporter: Jeff Bean
> Assignee: Ahmed Radwan
> Priority: Minor
> Attachments: MAPREDUCE-2836.patch, MAPREDUCE-2836_rev2.patch,
> MAPREDUCE-2836_rev3.patch, MAPREDUCE-2836_rev4.patch,
> MAPREDUCE-2836_rev5.patch, MAPREDUCE-2836_trunk.patch
>
>
> In some environments, it might be desirable to explicitly specify the fair
> scheduler pools and to explicitly fail jobs that are not submitted to any of
> the pools.
> Current behavior of the fair scheduler is to submit jobs to a default pool if
> a pool name isn't specified or to create a pool with the new name if the pool
> name doesn't already exist. There should be a configuration option for the
> fair scheduler that causes it to noisily fail the job if it's submitted to a
> pool that isn't pre-specified or if the specified pool doesn't exist.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
