[jira] [Commented] (TEZ-4547) Add Tez AM JobID to the JobConf
[ https://issues.apache.org/jira/browse/TEZ-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17826815#comment-17826815 ] Venkatasubrahmanian Narayanan commented on TEZ-4547: Past discussion in: https://issues.apache.org/jira/browse/HADOOP-19091 > Add Tez AM JobID to the JobConf > --- > > Key: TEZ-4547 > URL: https://issues.apache.org/jira/browse/TEZ-4547 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.10.2 >Reporter: Venkatasubrahmanian Narayanan >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Tez creates JobIDs for tasks by appending the vertex index to the cluster > timestamp to avoid multiple jobs run in a single Tez session sharing a JobID. > Hadoop's MagicS3GuardCommitter needs a job-wide UUID to ensure that the task > committers and the job committer write to/read from the same paths and can > hence actually commit data. Adding the AM's JobID to the Configuration > objects allows applications like Hive to pass that as the UUID to the > committer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TEZ-4547) Add Tez AM JobID to the JobConf
[ https://issues.apache.org/jira/browse/TEZ-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17826814#comment-17826814 ] Venkatasubrahmanian Narayanan commented on TEZ-4547: Github PR: https://github.com/apache/tez/pull/339 > Add Tez AM JobID to the JobConf > --- > > Key: TEZ-4547 > URL: https://issues.apache.org/jira/browse/TEZ-4547 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.10.2 >Reporter: Venkatasubrahmanian Narayanan >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Tez creates JobIDs for tasks by appending the vertex index to the cluster > timestamp to avoid multiple jobs run in a single Tez session sharing a JobID. > Hadoop's MagicS3GuardCommitter needs a job-wide UUID to ensure that the task > committers and the job committer write to/read from the same paths and can > hence actually commit data. Adding the AM's JobID to the Configuration > objects allows applications like Hive to pass that as the UUID to the > committer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
