[
https://issues.apache.org/jira/browse/HADOOP-1210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lohit updated HADOOP-1210:
--------------------------
Attachment: h-1210-jobstatus.patch
This looks like useful feature to have. As to get it started, I am uploading an
initial patch which will dump the counters onto standard out via hadoop job
-status command. The output would be similar to the one shown below.
Suggestions/Inputs as to what could be changes? The getCounters() API works
great for global counters. Is there a way to fetch counters from Individual
map/reducers?
Job: job_200708231653_0002
file: /hadoop/mapred/system/job_200708231653_0002/job.xml
tracking URL: http://myhost:50030/jobdetails.jsp?jobid=job_200708231653_0002
map() completion: 1.0
reduce() completion: 1.0
Job Counters:
Counter Group : org.apache.hadoop.mapred.JobInProgress$Counter
TOTAL_LAUNCHED_MAPS = 2
TOTAL_LAUNCHED_REDUCES = 1
DATA_LOCAL_MAPS = 2
Counter Group : org.apache.hadoop.mapred.Task$Counter
MAP_INPUT_RECORDS = 1967
MAP_OUTPUT_RECORDS = 5288
MAP_INPUT_BYTES = 74377
MAP_OUTPUT_BYTES = 84327
COMBINE_INPUT_RECORDS = 5288
COMBINE_OUTPUT_RECORDS = 2293
REDUCE_INPUT_GROUPS = 2253
REDUCE_INPUT_RECORDS = 2293
REDUCE_OUTPUT_RECORDS = 2253
> Log counters in job history
> ---------------------------
>
> Key: HADOOP-1210
> URL: https://issues.apache.org/jira/browse/HADOOP-1210
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Albert Chern
> Priority: Minor
> Attachments: h-1210-jobstatus.patch
>
>
> It would be useful if the value of the global counters were logged to the job
> history, perhaps even individually for each task after completion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
