Massimiliano Mattetti created BLUR-449:
------------------------------------------
Summary: The commitOrAbortJob method of the BlurOutputCommitter
class ends up always with the warning "TaskAttempt JobID [{0}] does not match
JobContext JobId [{1}]"
Key: BLUR-449
URL: https://issues.apache.org/jira/browse/BLUR-449
Project: Apache Blur
Issue Type: Bug
Components: Blur MapReduce
Affects Versions: 0.3.0
Environment: Spark 1.6.0 + Hadoop 2.6.4
Reporter: Massimiliano Mattetti
The equals in this if statement:
if (taskAttemptID.getJobID().equals(jobContext.getJobID()))
is always returning false. The problem is similar to the one described here
https://issues.apache.org/jira/browse/MAPREDUCE-1842. The TaskAttemptID.forName
method instantiate a new TaskAttemptID object whose type comes from the older
API package org.apache.hadoop.mapred. Since the JobContext contains a JobID
form the newer package org.apache.hadoop.mapreduce and the equals method shared
by the two JobIDs uses this.getClass() == that.getClass, the equals between the
two objects will always return false. Comparing the JobID Strings instead of
the objects is a possible workaround otherwise another possibility would be to
instantiate a org.apache.hadoop.mapreduce.TaskAttemptID object using the one
retuned from the TaskAttemptID.forName method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
