[
https://issues.apache.org/jira/browse/MAPREDUCE-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084465#comment-13084465
]
Hudson commented on MAPREDUCE-901:
----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #765 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/765/])
MAPREDUCE-901. Efficient framework counters. Contributed by Luke Lu.
acmurthy :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1157290
Files :
*
/hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapred/TestJobInProgress.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/FileSystemCounter.properties
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapred/TaskStatus.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapred/TaskInProgress.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/Counter.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/Counters.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapred/InterTrackerProtocol.java
* /hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters/LimitExceededException.java
*
/hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/TestCounters.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/JobCounter.properties
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters/Limits.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters/FrameworkCounterGroup.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters/package-info.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters/CounterGroupBase.java
* /hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapred/Task.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapred/JobInProgress.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapred/JobInProgress.java.orig
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/jobhistory/EventReader.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/CounterGroup.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters/FileSystemCounterGroup.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/util/CountersStrings.java
*
/hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRWithDFS.java
*
/hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapred/TestCombineOutputCollector.java
* /hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapred/Counters.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters/GenericCounter.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters/AbstractCounterGroup.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/MRJobConfig.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/util/ResourceBundles.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/protocol/ClientProtocol.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters/CounterGroupFactory.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/FileSystemCounter.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapred/TaskTracker.java
*
/hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapred/TestSeveral.java
* /hadoop/common/trunk/mapreduce/CHANGES.txt
*
/hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRDFSSort.java
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/TaskCounter.properties
*
/hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapreduce/counters/AbstractCounter.java
> Move Framework Counters into a TaskMetric structure
> ---------------------------------------------------
>
> Key: MAPREDUCE-901
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-901
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: task
> Affects Versions: 0.21.0
> Reporter: Owen O'Malley
> Assignee: Luke Lu
> Fix For: 0.23.0
>
> Attachments: 901_1.patch, 901_1.patch, FrameworkCounterGroup.java,
> MAPREDUCE-901.patch, MAPREDUCE-901.patch, MAPREDUCE-901.patch,
> MAPREDUCE-901.patch_2, mr-901-trunk-v1.patch
>
>
> I think we should move all of the Counters that the framework updates into a
> single class called TaskMetrics. TaskMetrics would have specific fields for
> each of the metrics like input records, input bytes, output records, etc.
> It would both reduce the serialized size of the heartbeats (by shrinking the
> Counters down to just the user's counters) and decrease the latency for
> updates to the JobTracker (since Counters are sent at most 1/minute instead
> of 1/heartbeat).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
