Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-05-28 Thread via GitHub
tez-yetus commented on PR #339: URL: https://github.com/apache/tez/pull/339#issuecomment-2136045430 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 27m 44s | Docker mode

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-05-28 Thread via GitHub
VenkatSNarayanan commented on code in PR #339: URL: https://github.com/apache/tez/pull/339#discussion_r1617772403 ## tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/MRJobConfig.java: ## @@ -131,6 +131,11 @@ public interface MRJobConfig { public static final

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-05-28 Thread via GitHub
VenkatSNarayanan commented on code in PR #339: URL: https://github.com/apache/tez/pull/339#discussion_r1617772160 ## tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/OutputCommitterContextImpl.java: ## @@ -34,14 +34,16 @@ public class OutputCommitterContextImpl implements

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-05-14 Thread via GitHub
abstractdog commented on code in PR #339: URL: https://github.com/apache/tez/pull/339#discussion_r1599582588 ## tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/OutputCommitterContextImpl.java: ## @@ -34,14 +34,16 @@ public class OutputCommitterContextImpl implements

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-05-14 Thread via GitHub
abstractdog commented on code in PR #339: URL: https://github.com/apache/tez/pull/339#discussion_r1599573887 ## tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/MRJobConfig.java: ## @@ -131,6 +131,11 @@ public interface MRJobConfig { public static final String

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-05-07 Thread via GitHub
VenkatSNarayanan commented on PR #339: URL: https://github.com/apache/tez/pull/339#issuecomment-2099211212 @abstractdog @shameersss1 Is there anything else needed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-04-08 Thread via GitHub
tez-yetus commented on PR #339: URL: https://github.com/apache/tez/pull/339#issuecomment-2043555966 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 26m 4s | Docker mode

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-04-05 Thread via GitHub
shameersss1 commented on PR #339: URL: https://github.com/apache/tez/pull/339#issuecomment-2039108306 @abstractdog - Could you please review the same ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-04-05 Thread via GitHub
tez-yetus commented on PR #339: URL: https://github.com/apache/tez/pull/339#issuecomment-2039101580 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-04-04 Thread via GitHub
VenkatSNarayanan commented on PR #339: URL: https://github.com/apache/tez/pull/339#issuecomment-2039000568 @shameersss1 We could actually just set fs.s3a.committer.uuid directly instead of the indirection through the other setting. -- This is an automated message from the Apache Git

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-04-03 Thread via GitHub
VenkatSNarayanan commented on code in PR #339: URL: https://github.com/apache/tez/pull/339#discussion_r1550312953 ## tez-mapreduce/src/main/java/org/apache/tez/mapreduce/committer/MROutputCommitter.java: ## @@ -119,6 +120,7 @@ public void abortOutput(VertexStatus.State

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-04-03 Thread via GitHub
VenkatSNarayanan commented on code in PR #339: URL: https://github.com/apache/tez/pull/339#discussion_r1550307095 ## tez-mapreduce/src/main/java/org/apache/tez/mapreduce/common/Utils.java: ## @@ -63,5 +64,8 @@ public static Counter getMRCounter(TezCounter tezCounter) {

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-04-03 Thread via GitHub
VenkatSNarayanan commented on code in PR #339: URL: https://github.com/apache/tez/pull/339#discussion_r1550306809 ## tez-mapreduce/src/main/java/org/apache/tez/mapreduce/output/MROutput.java: ## @@ -413,6 +414,7 @@ protected List initializeBase() throws IOException,

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-04-01 Thread via GitHub
tez-yetus commented on PR #339: URL: https://github.com/apache/tez/pull/339#issuecomment-2030457385 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 26m 48s | Docker mode

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-03-20 Thread via GitHub
shameersss1 commented on code in PR #339: URL: https://github.com/apache/tez/pull/339#discussion_r1531567044 ## tez-mapreduce/src/main/java/org/apache/tez/mapreduce/committer/MROutputCommitter.java: ## @@ -119,6 +120,7 @@ public void abortOutput(VertexStatus.State finalState)

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-03-19 Thread via GitHub
tez-yetus commented on PR #339: URL: https://github.com/apache/tez/pull/339#issuecomment-2007871466 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 28m 32s | Docker mode

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-03-18 Thread via GitHub
VenkatSNarayanan commented on PR #339: URL: https://github.com/apache/tez/pull/339#issuecomment-2004624012 > If my understanding is correct, Hive/Pig would use the value from `mapreduce.parent.job.id` to set the correct committer UUID right? Yes, that was the plan. The property name

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-03-17 Thread via GitHub
shameersss1 commented on code in PR #339: URL: https://github.com/apache/tez/pull/339#discussion_r1527423514 ## tez-mapreduce/src/main/java/org/apache/tez/mapreduce/committer/MROutputCommitter.java: ## @@ -119,6 +119,7 @@ public void abortOutput(VertexStatus.State finalState)

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-03-13 Thread via GitHub
r0hini commented on code in PR #339: URL: https://github.com/apache/tez/pull/339#discussion_r1523912700 ## tez-mapreduce/src/test/java/org/apache/tez/mapreduce/output/TestMROutput.java: ## @@ -131,6 +133,26 @@ public void testMergeConfig() throws Exception {

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-03-13 Thread via GitHub
r0hini commented on code in PR #339: URL: https://github.com/apache/tez/pull/339#discussion_r1523912081 ## tez-mapreduce/src/main/java/org/apache/tez/mapreduce/output/MROutput.java: ## @@ -417,6 +418,7 @@ protected List initializeBase() throws IOException, InterruptedException

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-03-13 Thread via GitHub
tez-yetus commented on PR #339: URL: https://github.com/apache/tez/pull/339#issuecomment-1995452059 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode

Re: [PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-03-13 Thread via GitHub
tez-yetus commented on PR #339: URL: https://github.com/apache/tez/pull/339#issuecomment-1995452045 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 26m 49s | Docker mode

[PR] TEZ-4547: Add Tez AM JobID to the JobConf [tez]

2024-03-13 Thread via GitHub
VenkatSNarayanan opened a new pull request, #339: URL: https://github.com/apache/tez/pull/339 Some committers require a job-wide UUID to function correctly. Adding the AM JobID to the JobConf will allow applications to pass that to the committers that need it. -- This is an