[
https://issues.apache.org/jira/browse/MAPREDUCE-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773645#action_12773645
]
Matei Zaharia commented on MAPREDUCE-707:
-----------------------------------------
Here are some comments on the patch:
# Instead of using the string "mapred.fairscheduler.pool" in multiple places in
PoolManager, make it a constant at the top of the file (something like
EXPLICIT_POOL_PROPERTY).
# Add a comment to PoolManager.getPoolName to explain the logic (first look for
the explicit pool property, then for the property named by poolNameProperty,
and finally default to DEFAULT_POOL_NAME).
# Add a unit test for PoolManager.getPoolName that tries each of those cases
(explicit property is set, no explicit property but poolNameProperty is used,
or neither is used). Right now your existing unit test checks that setPool
works but there's no test that submits a job with mapred.fairscheduler.pool
directly.
# Instead of assertEquals(0,
scheduler.getPoolManager().getPoolName(job2).compareTo("poolA")) you can
probably use a version of assertEquals that works on strings.
# In the documentation, instead of saying "This property is ignored if
mapred.fairscheduler.pool is specified." for the poolnameproperty, it would be
better to say that the poolnameproperty is used only for jobs in which
mapred.fairscheduler.pool is not explicitly set.
> Provide a jobconf property for explicitly assigning a job to a pool
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-707
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-707
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/fair-share
> Reporter: Matei Zaharia
> Priority: Trivial
> Attachments: MAPREDUCE-707-apache.patch
>
>
> A common use case of the fair scheduler is to have one pool per user, but
> then to define some special pools for various production jobs, import jobs,
> etc. Therefore, it would be nice if jobs went by default to the pool of the
> user who submitted them, but there was a setting to explicitly place a job in
> another pool. Today, this can be achieved through a sort of trick in the
> JobConf:
> {code}
> <property>
> <name>mapred.fairscheduler.poolnameproperty</name>
> <value>pool.name</value>
> </property>
> <property>
> <name>pool.name</name>
> <value>${user.name}</value>
> </property>
> {code}
> This JIRA proposes to add a property called mapred.fairscheduler.pool that
> allows a job to be placed directly into a pool, avoiding the need for this
> trick.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
