[
https://issues.apache.org/jira/browse/HADOOP-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12621877#action_12621877
]
Matei Zaharia commented on HADOOP-3930:
---------------------------------------
Making queues explicit makes sense for the purposes of getSchedulingInfo then.
As for what it should do when applied to a job, in the fair scheduler at least
we can have it show the job's fair share of map slots / reduce slots and its
weight in the fair sharing calculations. This was useful both for debugging and
for letting administrators understand the effects of putting jobs in a
particular pool, changing their priority, etc.
Regarding the comparator, I made it that because Owen/Sameer/Arun wanted to
also be able to compare a subset of the jobs, for example to be able to filter
jobs by user or something of that sort. With a comparator, you choose your
subset as you wish and then sort it. (In all this I'm assuming that the
JobTracker or JobQueueManager knows the full list of jobs and can therefore
filter it.) However, it would also be possible to return the whole job list and
filter it afterwards - which one is easier?
> Decide how to integrate scheduler info into CLI and job tracker web page
> ------------------------------------------------------------------------
>
> Key: HADOOP-3930
> URL: https://issues.apache.org/jira/browse/HADOOP-3930
> Project: Hadoop Core
> Issue Type: Improvement
> Reporter: Matei Zaharia
> Priority: Minor
>
> We need a way for job schedulers such as HADOOP-3445 and HADOOP-3476 to
> provide info to display on the JobTracker web interface and in the CLI. The
> main things needed seem to be:
> * A way for schedulers to provide info to show in a column on the web UI and
> in the CLI - something as simple as a single string, or a map<string, int>
> for multiple parameters.
> * Some sorting order for jobs - maybe a method to sort a list of jobs.
> Let's figure out what the best way to do this is and implement it in the
> existing schedulers.
> My first-order proposal at an API: Augment the TaskScheduler with
> * public Map<String, String> getSchedulingInfo(JobInProgress job) -- returns
> key-value pairs which are displayed in columns on the web UI or the CLI.
> * public Comparator<JobInProgress> getJobComparator() -- returns a comparator
> that can be used to determine the order in which jobs will be run, for
> sorting the jobs in the CLI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
