Deepesh Khandelwal created MAPREDUCE-5055:
---------------------------------------------
Summary: Binary compatibility for the method
Reporter.incrCounter(String group, String counter, long amount) between
hadoop-1.x and hadoop-2.x
Key: MAPREDUCE-5055
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5055
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.3-alpha
Environment: Linux
Reporter: Deepesh Khandelwal
Try to generate a lot of counters that exceed the default max limit so the job
errors.
To simulate this edit org.apache.hadoop.examples.RandomWriter class and add the
following in the mapper:
{code}
for (int i = 0; i < 150; i++) {
reporter.incrCounter("StringCounter", "counter" + i, 1L);
}
{code}
Attached is the modified code of RandomWriter.java from branch-1.
When I run it against branch-1,
{code}
hadoop jar build/hadoop-examples-1.2.0-SNAPSHOT.jar
-Dtest.randomwrite.total_bytes=8 rw2
{code}
it fails with the following error:
{code}
java.lang.Exception:
org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded
limits on number of counters - Counters=120 Limit=120
{code}
Using the jar compiled with branch-1 when running in a hadoop 2 cluster
completes without any error and don't see the counter. Interestingly there were
two more custom counters in the same job that are incremented and they are
visible. The difference is that they use the method
Reporter.incrCounter(Enum<?> key, long amount).
Compiled this against 2.x/trunk and ran it. The job did fail with the expected
counter exceeded exception. So it seems to be a case of binary incompatibility
between 1.x and 2.x.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
