[
https://issues.apache.org/jira/browse/HADOOP-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12626154#action_12626154
]
Hemanth Yamijala commented on HADOOP-3930:
------------------------------------------
One thing we haven't discussed this far is the changes to the framework to aid
the CLI.
Showing scheduling information related to a job seems easy. We can augment
{{JobStatus}} to contain a {{String schedulerInfo}}.
For showing the queue related information, one approach could be as follows:
{code}
public class QueueInfo implements Writable {
String queueName;
String schedulerInfo;
...
}
public interface JobSubmissionProtocol {
...
QueueInfo[] getQueues();
QueueInfo getQueueInfo(String queue);
JobStatus[] getJobs(String queue);
}
{code}
These APIs are similar to the Job related APIs, like {{getAllJobs(),
getJobStatus(JobID), and getMap/ReduceTaskReports(JobID)}}. Still, I am a
little worried adding these to {{JobSubmissionProtocol}} since {{getQueues()
and getQueueInfo()}} don't per-se relate to jobs directly. The alternative
though seems to be to define a new protocol that has this info. Open to
comments on which is better.
> Decide how to integrate scheduler info into CLI and job tracker web page
> ------------------------------------------------------------------------
>
> Key: HADOOP-3930
> URL: https://issues.apache.org/jira/browse/HADOOP-3930
> Project: Hadoop Core
> Issue Type: Improvement
> Affects Versions: 0.17.2
> Reporter: Matei Zaharia
> Assignee: Sreekanth Ramakrishnan
> Priority: Minor
> Attachments: 3930-1.patch, mockup.JPG
>
>
> We need a way for job schedulers such as HADOOP-3445 and HADOOP-3476 to
> provide info to display on the JobTracker web interface and in the CLI. The
> main things needed seem to be:
> * A way for schedulers to provide info to show in a column on the web UI and
> in the CLI - something as simple as a single string, or a map<string, int>
> for multiple parameters.
> * Some sorting order for jobs - maybe a method to sort a list of jobs.
> Let's figure out what the best way to do this is and implement it in the
> existing schedulers.
> My first-order proposal at an API: Augment the TaskScheduler with
> * public Map<String, String> getSchedulingInfo(JobInProgress job) -- returns
> key-value pairs which are displayed in columns on the web UI or the CLI for
> the list of jobs.
> * public Map<String, String> getSchedulingInfo(String queue) -- returns
> key-value pairs which are displayed in columns on the web UI or the CLI for
> the list of queues.
> * public Collection<JobInProgress> getJobs(String queueName) -- returns the
> list of jobs in a given queue, sorted by a scheduler-specific order (the
> order it wants to run them in / schedule the next task in / etc).
> * public List<String> getQueues();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
