[jira] [Comment Edited] (TEZ-4149) Speed up TezRecovery tests
[ https://issues.apache.org/jira/browse/TEZ-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17109465#comment-17109465 ] László Bodor edited comment on TEZ-4149 at 5/17/20, 12:57 PM: -- [~jeagles]: thanks for taking a look _"I think there is still value in being able to start hdfs and yarn separately, but I like that they are wrapped in a class like this. We can delay that decision until a test case needs one and not the other."_ Yeah, I was thinking about that. There would be an optional "useHdfs(boolean)" kind of method before calling "start()" to tell the wrapper whether to hdfs or not (default: true). I'm including that into the next patch. I'm about to create a separate jira for adopting this wrapper to all unit tests, and then we can see if we need any additional APIs for the wrapper. _"What is the cost/benefit of using the runner factory as well at the Waiter class. It's a little messy, but may be worth it is limited situations like this."_ I agree, it might look messy, both of them are used because this way I was able to share a common multithreaded runner logic (runWithShutDownConditions) between the 3 unit test methods in this class. Waiters cannot be eliminated as we assert inside the testing methods, and they run in subthreads from now on, we must not miss any assertion failures from those threads (if a subthread called the original Assert.assertXYZ, and failed, it would never propagate to the main thread, only through waiter.assertXYZ). Let me know if we can do this less mess, otherwise I would go on with this approach. _"Let's fix the ASF license header on MiniClusterGroup"_ I'll take care of it in the next patch. was (Author: abstractdog): [~jeagles]: thanks for taking a look _"I think there is still value in being able to start hdfs and yarn separately, but I like that they are wrapped in a class like this. We can delay that decision until a test case needs one and not the other."_ Yeah, I was thinking about that. There would be an optional "useHdfs(boolean)" kind of method before calling "start()" to tell the wrapper whether to hdfs or not (default: true). I'm including that into the next patch. I'm about to create a separate jira for adopting this wrapper to all unit tests, and then we can see if we need any additional APIs for the wrapper. _What is the cost/benefit of using the runner factory as well at the Waiter class. It's a little messy, but may be worth it is limited situations like this._ I agree, it might look messy, both of them are used because this way I was able to share a common multithreaded runner logic (runWithShutDownConditions) between the 3 unit test methods in this class. Waiters cannot be eliminated as we assert inside the testing methods, and they run in subthreads from now on, we must not miss any assertion failures from those threads (if a subthread called the original Assert.assertXYZ, and failed, it would never propagate to the main thread, only through waiter.assertXYZ). Let me know if we can do this less mess, otherwise I would go on with this approach. _Let's fix the ASF license header on MiniClusterGroup_ I'll take care of it in the next patch. > Speed up TezRecovery tests > -- > > Key: TEZ-4149 > URL: https://issues.apache.org/jira/browse/TEZ-4149 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Jonathan Turner Eagles >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4149.01.patch, TEZ-4149.WIP.patch, > org.apache.tez.test.TestRecovery-output.txt > > > Currently, approximately 50% of the tests cases are chosen to run as there > are many failure points chosen to test recovery on. > This can lead to the introduction of bugs into the code as not all test cases > are run for every Tez QA run. > In addition, this can be a real development bottleneck as tests take around > 20 minutes per cycle if all tests are run (10 minutes if 50% of the tests are > run as usual) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-4149) Speed up TezRecovery tests
[ https://issues.apache.org/jira/browse/TEZ-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17109465#comment-17109465 ] László Bodor commented on TEZ-4149: --- [~jeagles]: thanks for taking a look _"I think there is still value in being able to start hdfs and yarn separately, but I like that they are wrapped in a class like this. We can delay that decision until a test case needs one and not the other."_ Yeah, I was thinking about that. There would be an optional "useHdfs(boolean)" kind of method before calling "start()" to tell the wrapper whether to hdfs or not (default: true). I'm including that into the next patch. I'm about to create a separate jira for adopting this wrapper to all unit tests, and then we can see if we need any additional APIs for the wrapper. _What is the cost/benefit of using the runner factory as well at the Waiter class. It's a little messy, but may be worth it is limited situations like this._ I agree, it might look messy, both of them are used because this way I was able to share a common multithreaded runner logic (runWithShutDownConditions) between the 3 unit test methods in this class. Waiters cannot be eliminated as we assert inside the testing methods, and they run in subthreads from now on, we must not miss any assertion failures from those threads (if a subthread called the original Assert.assertXYZ, and failed, it would never propagate to the main thread, only through waiter.assertXYZ). Let me know if we can do this less mess, otherwise I would go on with this approach. _Let's fix the ASF license header on MiniClusterGroup_ I'll take care of it in the next patch. > Speed up TezRecovery tests > -- > > Key: TEZ-4149 > URL: https://issues.apache.org/jira/browse/TEZ-4149 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Jonathan Turner Eagles >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4149.01.patch, TEZ-4149.WIP.patch, > org.apache.tez.test.TestRecovery-output.txt > > > Currently, approximately 50% of the tests cases are chosen to run as there > are many failure points chosen to test recovery on. > This can lead to the introduction of bugs into the code as not all test cases > are run for every Tez QA run. > In addition, this can be a real development bottleneck as tests take around > 20 minutes per cycle if all tests are run (10 minutes if 50% of the tests are > run as usual) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-4129) Delete intermediate attempt data for failed attempts for Shuffle Handler
[ https://issues.apache.org/jira/browse/TEZ-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17109451#comment-17109451 ] Syed Shameerur Rahman commented on TEZ-4129: [~jeagles] Thank you for reviewing the patch. I have addressed all your comments in TEZ-4129.06.patch. And yes It do makes sense to clear off the attempt entry in Index Cache. I have handled that too in the patch > Delete intermediate attempt data for failed attempts for Shuffle Handler > > > Key: TEZ-4129 > URL: https://issues.apache.org/jira/browse/TEZ-4129 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Jonathan Turner Eagles >Assignee: Syed Shameerur Rahman >Priority: Major > Labels: ShuffleHandler > Attachments: TEZ-4129.01.patch, TEZ-4129.02.patch, TEZ-4129.03.patch, > TEZ-4129.04.patch, TEZ-4129.05.patch, TEZ-4129.06.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-4129) Delete intermediate attempt data for failed attempts for Shuffle Handler
[
https://issues.apache.org/jira/browse/TEZ-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17109449#comment-17109449
]
TezQA commented on TEZ-4129:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
48s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
17s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
36s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
46s{color} | {color:green} tez-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} tez-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
39s{color} | {color:green} tez-runtime-library in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
56s{color} | {color:green} tez-dag in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
32s{color} | {color:green} tez-aux-services in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.8 Server=19.03.8 base:
https://builds.apache.org/job/PreCommit-TEZ-Build/436/artifact/out/Dockerfile |
| JIRA Issue | TEZ-4129 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13003188/TEZ-4129.06.patch |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
checkstyle compile |
| uname | Linux 88a0eb36310e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / 07c807b |
| Default Java | 1.8.0_252 |
| Test Results |
https://builds.apache.org/job/PreCommit-TEZ-Build/436/testReport/ |
| Max. process+thread count | 1638 (vs. ulimit of 5500) |
|
[jira] [Updated] (TEZ-4129) Delete intermediate attempt data for failed attempts for Shuffle Handler
[ https://issues.apache.org/jira/browse/TEZ-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Syed Shameerur Rahman updated TEZ-4129: --- Attachment: TEZ-4129.06.patch > Delete intermediate attempt data for failed attempts for Shuffle Handler > > > Key: TEZ-4129 > URL: https://issues.apache.org/jira/browse/TEZ-4129 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Jonathan Turner Eagles >Assignee: Syed Shameerur Rahman >Priority: Major > Labels: ShuffleHandler > Attachments: TEZ-4129.01.patch, TEZ-4129.02.patch, TEZ-4129.03.patch, > TEZ-4129.04.patch, TEZ-4129.05.patch, TEZ-4129.06.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-4129) Delete intermediate attempt data for failed attempts for Shuffle Handler
[ https://issues.apache.org/jira/browse/TEZ-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Syed Shameerur Rahman updated TEZ-4129: --- Attachment: (was: TEZ-4129.05.patch) > Delete intermediate attempt data for failed attempts for Shuffle Handler > > > Key: TEZ-4129 > URL: https://issues.apache.org/jira/browse/TEZ-4129 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Jonathan Turner Eagles >Assignee: Syed Shameerur Rahman >Priority: Major > Labels: ShuffleHandler > Attachments: TEZ-4129.01.patch, TEZ-4129.02.patch, TEZ-4129.03.patch, > TEZ-4129.04.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-4129) Delete intermediate attempt data for failed attempts for Shuffle Handler
[
https://issues.apache.org/jira/browse/TEZ-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17109426#comment-17109426
]
TezQA commented on TEZ-4129:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
24s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
37s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
45s{color} | {color:green} tez-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} tez-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
43s{color} | {color:green} tez-runtime-library in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 4m 26s{color}
| {color:red} tez-dag in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
32s{color} | {color:green} tez-aux-services in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 51m 20s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | tez.dag.app.TestSpeculation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.8 Server=19.03.8 base:
https://builds.apache.org/job/PreCommit-TEZ-Build/435/artifact/out/Dockerfile |
| JIRA Issue | TEZ-4129 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13003179/TEZ-4129.05.patch |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
checkstyle compile |
| uname | Linux 89f031f88411 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / 07c807b |
| Default Java | 1.8.0_252 |
| unit |
[jira] [Commented] (TEZ-4184) Tez-Build-Hadoop3 fails because of protoc 3.0.0
[
https://issues.apache.org/jira/browse/TEZ-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17109418#comment-17109418
]
László Bodor commented on TEZ-4184:
---
I think the problem basically is that this job, as opposed to [precommit tez
job|https://builds.apache.org/job/PreCommit-TEZ-Build/], doesn't run in a
docker container, so it depends on protoc brought by the environment...
> Tez-Build-Hadoop3 fails because of protoc 3.0.0
> ---
>
> Key: TEZ-4184
> URL: https://issues.apache.org/jira/browse/TEZ-4184
> Project: Apache Tez
> Issue Type: Bug
>Reporter: László Bodor
>Priority: Minor
>
> It builds as:
> {code}
> mvn clean test -DskipTests -Phadoop28 '-P!hadoop27'
> -Dhadoop.version=3.0.0-alpha2
> {code}
> https://builds.apache.org/job/Tez-Build-Hadoop3/250/console
> {code}
> [ERROR] Failed to execute goal
> org.apache.hadoop:hadoop-maven-plugins:3.0.0-alpha2:protoc (compile-protoc)
> on project tez-api: org.apache.maven.plugin.MojoExecutionException: protoc
> version is 'libprotoc 3.0.0', expected version is '2.5.0' -> [Help 1]
> [ERROR]
> {code}
> I don't get the exact purpose of this job, but in the current way it doesn't
> make any sense, because hadoop3.x is still on protoc2.x until Hadoop 3.3, so
> the environment of this job should have protoc 2.5 installed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (TEZ-4184) Tez-Build-Hadoop3 fails because of protoc 3.0.0
[
https://issues.apache.org/jira/browse/TEZ-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4184:
--
Description:
It builds as:
{code}
mvn clean test -DskipTests -Phadoop28 '-P!hadoop27'
-Dhadoop.version=3.0.0-alpha2
{code}
https://builds.apache.org/job/Tez-Build-Hadoop3/250/console
{code}
[ERROR] Failed to execute goal
org.apache.hadoop:hadoop-maven-plugins:3.0.0-alpha2:protoc (compile-protoc) on
project tez-api: org.apache.maven.plugin.MojoExecutionException: protoc version
is 'libprotoc 3.0.0', expected version is '2.5.0' -> [Help 1]
[ERROR]
{code}
I don't get the exact purpose of this job, but in the current way it doesn't
make any sense, because hadoop3.x is still on protoc2.x until Hadoop 3.3, so
the environment of this job should have protoc 2.5 installed.
was:
It builds as:
{code}
mvn clean test -DskipTests -Phadoop28 '-P!hadoop27'
-Dhadoop.version=3.0.0-alpha2
{code}
{code}
[ERROR] Failed to execute goal
org.apache.hadoop:hadoop-maven-plugins:3.0.0-alpha2:protoc (compile-protoc) on
project tez-api: org.apache.maven.plugin.MojoExecutionException: protoc version
is 'libprotoc 3.0.0', expected version is '2.5.0' -> [Help 1]
[ERROR]
{code}
I don't get the exact purpose of this job, but in the current way it doesn't
make any sense, because hadoop3.x is still on protoc2.x until Hadoop 3.3, so
the environment of this job should have protoc 2.5 installed.
> Tez-Build-Hadoop3 fails because of protoc 3.0.0
> ---
>
> Key: TEZ-4184
> URL: https://issues.apache.org/jira/browse/TEZ-4184
> Project: Apache Tez
> Issue Type: Bug
>Reporter: László Bodor
>Priority: Minor
>
> It builds as:
> {code}
> mvn clean test -DskipTests -Phadoop28 '-P!hadoop27'
> -Dhadoop.version=3.0.0-alpha2
> {code}
> https://builds.apache.org/job/Tez-Build-Hadoop3/250/console
> {code}
> [ERROR] Failed to execute goal
> org.apache.hadoop:hadoop-maven-plugins:3.0.0-alpha2:protoc (compile-protoc)
> on project tez-api: org.apache.maven.plugin.MojoExecutionException: protoc
> version is 'libprotoc 3.0.0', expected version is '2.5.0' -> [Help 1]
> [ERROR]
> {code}
> I don't get the exact purpose of this job, but in the current way it doesn't
> make any sense, because hadoop3.x is still on protoc2.x until Hadoop 3.3, so
> the environment of this job should have protoc 2.5 installed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (TEZ-4184) Tez-Build-Hadoop3 fails because of protoc 3.0.0
[
https://issues.apache.org/jira/browse/TEZ-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4184:
--
Description:
It builds as:
{code}
mvn clean test -DskipTests -Phadoop28 '-P!hadoop27'
-Dhadoop.version=3.0.0-alpha2
{code}
{code}
[ERROR] Failed to execute goal
org.apache.hadoop:hadoop-maven-plugins:3.0.0-alpha2:protoc (compile-protoc) on
project tez-api: org.apache.maven.plugin.MojoExecutionException: protoc version
is 'libprotoc 3.0.0', expected version is '2.5.0' -> [Help 1]
[ERROR]
{code}
I don't get the exact purpose of this job, but in the current way it doesn't
make any sense, because hadoop3.x is still on protoc2.x until Hadoop 3.3, so
the environment of this job should have protoc 2.5 installed.
> Tez-Build-Hadoop3 fails because of protoc 3.0.0
> ---
>
> Key: TEZ-4184
> URL: https://issues.apache.org/jira/browse/TEZ-4184
> Project: Apache Tez
> Issue Type: Bug
>Reporter: László Bodor
>Priority: Minor
>
> It builds as:
> {code}
> mvn clean test -DskipTests -Phadoop28 '-P!hadoop27'
> -Dhadoop.version=3.0.0-alpha2
> {code}
> {code}
> [ERROR] Failed to execute goal
> org.apache.hadoop:hadoop-maven-plugins:3.0.0-alpha2:protoc (compile-protoc)
> on project tez-api: org.apache.maven.plugin.MojoExecutionException: protoc
> version is 'libprotoc 3.0.0', expected version is '2.5.0' -> [Help 1]
> [ERROR]
> {code}
> I don't get the exact purpose of this job, but in the current way it doesn't
> make any sense, because hadoop3.x is still on protoc2.x until Hadoop 3.3, so
> the environment of this job should have protoc 2.5 installed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (TEZ-4184) Tez-Build-Hadoop3 fails because of protoc 3.0.0
László Bodor created TEZ-4184: - Summary: Tez-Build-Hadoop3 fails because of protoc 3.0.0 Key: TEZ-4184 URL: https://issues.apache.org/jira/browse/TEZ-4184 Project: Apache Tez Issue Type: Bug Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-4129) Delete intermediate attempt data for failed attempts for Shuffle Handler
[ https://issues.apache.org/jira/browse/TEZ-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Syed Shameerur Rahman updated TEZ-4129: --- Attachment: TEZ-4129.05.patch > Delete intermediate attempt data for failed attempts for Shuffle Handler > > > Key: TEZ-4129 > URL: https://issues.apache.org/jira/browse/TEZ-4129 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Jonathan Turner Eagles >Assignee: Syed Shameerur Rahman >Priority: Major > Labels: ShuffleHandler > Attachments: TEZ-4129.01.patch, TEZ-4129.02.patch, TEZ-4129.03.patch, > TEZ-4129.04.patch, TEZ-4129.05.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
