[
https://issues.apache.org/jira/browse/MAPREDUCE-546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728272#action_12728272
]
Matei Zaharia commented on MAPREDUCE-546:
-----------------------------------------
I've thought a bit about using a Configuration for the pools file, but I'm
afraid it would become very verbose. A config file like this:
{code}
<allocations>
<pool name="ads">
<minMaps>10</minMaps>
<minReduces>5</minReduces>
<minSharePreemptionTimeout>300</minSharePreemptionTimeout>
</pool>
<user name="bob">
<maxRunningJobs>2</maxRunningJobs>
</user>
</allocations>
{code}
Would become something like this:
{code}
<configuration>
<property>
<name>mapred.fairscheduler.pool.ads.minMaps</name>
<value>10</value>
</property>
<property>
<name>mapred.fairscheduler.pool.ads.minReduces</name>
<value>5</value>
</property>
<property>
<name>mapred.fairscheduler.pool.ads.minSharePreemptionTimeout</name>
<value>600</value>
</property>
<property>
<name>mapred.fairscheduler.user.bob.maxRunningJobs</name>
<value>2</value>
</property>
</configuration>
{code}
I find the first one more readable and more maintainable, especially as
properties relating to the same pool are grouped together. Do you think the
code reuse benefits of using Configuration outweigh the loss in usability? I
actually don't think the configuration reading code will be much smaller using
Configuration because we'd have to parse string names like
mapred.fairscheduler.pool.ads.minReduces instead of parsing XML.
> Provide sample fair scheduler config file in conf/ and set config file
> property to point to this by default
> -----------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-546
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-546
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Matei Zaharia
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: mapreduce-546-v1.patch, mapreduce-546.patch
>
>
> The capacity scheduler includes a config file template in hadoop/conf, so it
> would make sense to create a similar one for the fair scheduler and mention
> it in the README.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
