-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2167/
-----------------------------------------------------------
(Updated 2011-10-04 22:58:24.529007)
Review request for hive, Ramkumar Vadali and Yongqiang He.
Changes
-------
MapRedStats provides an enum which just contains those counters that were being
used before this diff. I also made it take a generic enum, and made the task
counter enum used by HadoopJobExecHelper configurable. This way, if one Hadoop
version contains more enums than another, the counters to be logged can be
modified by creating a new enum.
Summary
-------
I added the counters mentioned in the task to the MapRedStats class, and
modified HadoopJobExecHelper to collect them.
I got tired of writing the same code over and over again, so I modified the way
MapRedStats and HadoopJobExecHelper treat task counters. MapRedStats now has
an enum with all of the task counters we want to collect, it is a subset of the
enum in Task$Counter. Task is package private so the enum in it is
unavailable. MapRedStats now contains a map from the enum values to the values
of the counters, if they were set. HadoopJobExecHelper loops over the enum
values and tries to get a value for each counter. As long as the new getter
and setter methods are used the functionality is the same, in particular for
the getter, if a counter was set, it returns the value of the counter,
otherwise it returns -1.
This addresses bug Hive-2479.
https://issues.apache.org/jira/browse/Hive-2479
Diffs (updated)
-----
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1178612
trunk/conf/hive-default.xml 1178612
trunk/ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java 1178612
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java
1178612
Diff: https://reviews.apache.org/r/2167/diff
Testing
-------
I ran some queries to verify the counters were being populated.
I also ran a few of the unit test queries to verify I hadn't broken anything.
Thanks,
Kevin
