I hope this is the right mailing list for this question. I'm trying to use Hadoop's S3MagicGuardCommitter with Pig on Tez(writing to a Hive table), and the committer fails to commit the output because it expects the JobID to be the same between the task commit and the job commit. This does not seem to be true when running my job on Tez, where in my experiments the jtIdentifier of the jobID for the task container consistently has an extra 0 appended after the cluster timestamp portion of the jobID. From some digging, this 0 does not come from the Tez vertex ID. Is there some place where Tez creates a separate JobID for tasks by appending a number (or multiplying the cluster timestamp by 10), and is this intentional behavior?
Thanks Venkatasubrahmanian Narayanan
