[jira] [Updated] (TEZ-4101) Eliminate some guava dependencies by Java8+ features - Preconditions
[ https://issues.apache.org/jira/browse/TEZ-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated TEZ-4101: -- Attachment: TEZ-4101.05.patch > Eliminate some guava dependencies by Java8+ features - Preconditions > > > Key: TEZ-4101 > URL: https://issues.apache.org/jira/browse/TEZ-4101 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4101.01.patch, TEZ-4101.02.patch, TEZ-4101.03.patch, > TEZ-4101.04.patch, TEZ-4101.05.patch > > > 1. Preconditions.checkNotNull -> Objects.requireNonNull > 2. wrap others into a tez common class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-4101) Eliminate some guava dependencies by Java8+ features - Preconditions
[ https://issues.apache.org/jira/browse/TEZ-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated TEZ-4101: -- Attachment: TEZ-4101.06.patch > Eliminate some guava dependencies by Java8+ features - Preconditions > > > Key: TEZ-4101 > URL: https://issues.apache.org/jira/browse/TEZ-4101 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4101.01.patch, TEZ-4101.02.patch, TEZ-4101.03.patch, > TEZ-4101.04.patch, TEZ-4101.05.patch, TEZ-4101.06.patch > > > 1. Preconditions.checkNotNull -> Objects.requireNonNull > 2. wrap others into a tez common class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-4101) Eliminate some guava dependencies by Java8+ features - Preconditions
[
https://issues.apache.org/jira/browse/TEZ-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024289#comment-17024289
]
TezQA commented on TEZ-4101:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
3s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 17 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
38s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 21s{color} | {color:orange} tez-api: The patch generated 3 new + 315
unchanged - 1 fixed = 318 total (was 316) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 11s{color} | {color:orange} tez-common: The patch generated 2 new + 37
unchanged - 0 fixed = 39 total (was 37) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
43s{color} | {color:green} tez-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} tez-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
40s{color} | {color:green} tez-runtime-internals in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
50s{color} | {color:green} tez-runtime-library in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
16s{color} | {color:green} tez-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
12s{color} | {color:green} tez-examples in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 4m 1s{color}
| {color:red} tez-dag in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 49m 37s{color}
| {color:red} tez-tests in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
54s{color} | {color:green} tez-ext-service-tests in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
26s{color} | {color:green} tez-history-parser in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit
[jira] [Commented] (TEZ-4101) Eliminate some guava dependencies by Java8+ features - Preconditions
[
https://issues.apache.org/jira/browse/TEZ-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024534#comment-17024534
]
TezQA commented on TEZ-4101:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
5s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
3s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 17 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
42s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 9m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 21s{color} | {color:orange} tez-api: The patch generated 2 new + 314
unchanged - 0 fixed = 316 total (was 314) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 12s{color} | {color:orange} tez-common: The patch generated 2 new + 37
unchanged - 0 fixed = 39 total (was 37) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
54s{color} | {color:green} tez-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
24s{color} | {color:green} tez-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
42s{color} | {color:green} tez-runtime-internals in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
42s{color} | {color:green} tez-runtime-library in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
10s{color} | {color:green} tez-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
12s{color} | {color:green} tez-examples in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 5m 39s{color}
| {color:red} tez-dag in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 47m 8s{color}
| {color:red} tez-tests in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
56s{color} | {color:green} tez-ext-service-tests in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
16s{color} | {color:green} tez-history-parser in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit
[jira] [Commented] (TEZ-4114) Upgrade to Jetty 9.4
[
https://issues.apache.org/jira/browse/TEZ-4114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024763#comment-17024763
]
Siyao Meng commented on TEZ-4114:
-
Hi [~abstractdog], just a reminder that we need to increment jetty version to
{{9.4.26.v20200117}} according to CDPD-7365. Thanks.
> Upgrade to Jetty 9.4
>
>
> Key: TEZ-4114
> URL: https://issues.apache.org/jira/browse/TEZ-4114
> Project: Apache Tez
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: TEZ-4114.01.patch
>
>
> Hadoop already did it in: https://issues.apache.org/jira/browse/HADOOP-16152
> Hive: "in progress", only abandoned jiras
> https://issues.apache.org/jira/browse/HIVE-21961
> https://issues.apache.org/jira/browse/HIVE-21211
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (TEZ-4119) TestSpeculation is flaky
[ https://issues.apache.org/jira/browse/TEZ-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated TEZ-4119: -- Attachment: org.apache.tez.dag.app.TestSpeculation-output.txt > TestSpeculation is flaky > > > Key: TEZ-4119 > URL: https://issues.apache.org/jira/browse/TEZ-4119 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: org.apache.tez.dag.app.TestSpeculation-output.txt > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-4119) TestSpeculation is flaky
[ https://issues.apache.org/jira/browse/TEZ-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated TEZ-4119: -- Attachment: (was: TestSpeculation.txt) > TestSpeculation is flaky > > > Key: TEZ-4119 > URL: https://issues.apache.org/jira/browse/TEZ-4119 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: org.apache.tez.dag.app.TestSpeculation-output.txt > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (TEZ-4119) TestSpeculation is flaky
[
https://issues.apache.org/jira/browse/TEZ-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024431#comment-17024431
]
László Bodor edited comment on TEZ-4119 at 1/27/20 3:43 PM:
sometimes it can be reproduced locally, by running all tez-dag tests (normally,
TestSpeculation should finish in 3-5 seconds):
from tez-dag dir:
{code}
mvn --batch-mode clean test -fae
{code}
there is a suspicious no-op for almost 2 minutes:
{code}
2020-01-27 16:33:22,162 INFO [Dispatcher thread {Central}] app.DAGAppMaster
(DAGAppMaster.java:handle(872)) - Completed cleanup for DAG: name=test, with
id=dag_1580139200870_0001_1
2020-01-27 16:35:00,864 INFO [Thread-53] client.TezClient
(TezClient.java:(210)) - Tez Client Version: [ component=tez-api,
version=0.10.1-SNAPSHOT, revision=f049d93a69bdc5bd736301bf7081fa5cef2694bf,
SCM-URL=scm:git:https://gitbox.apache.org/repos/asf/tez.git,
buildTime=2020-01-27T09:27:08Z ]
{code}
jstacks of the surefire process: [^jstack.log] ...later: [^jstack6.log]
was (Author: abstractdog):
sometimes it can be reproduced locally, by running all tez-dag tests (normally,
TestSpeculation should finish in 3-5 seconds):
from tez-dag dir:
{code}
mvn --batch-mode clean test -fae
{code}
there is a suspicious no-op for almost 2 minutes:
{code}
2020-01-27 16:33:22,162 INFO [Dispatcher thread {Central}] app.DAGAppMaster
(DAGAppMaster.java:handle(872)) - Completed cleanup for DAG: name=test, with
id=dag_1580139200870_0001_1
2020-01-27 16:35:00,864 INFO [Thread-53] client.TezClient
(TezClient.java:(210)) - Tez Client Version: [ component=tez-api,
version=0.10.1-SNAPSHOT, revision=f049d93a69bdc5bd736301bf7081fa5cef2694bf,
SCM-URL=scm:git:https://gitbox.apache.org/repos/asf/tez.git,
buildTime=2020-01-27T09:27:08Z ]
{code}
> TestSpeculation is flaky
>
>
> Key: TEZ-4119
> URL: https://issues.apache.org/jira/browse/TEZ-4119
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: jstack.log, jstack6.log,
> org.apache.tez.dag.app.TestSpeculation-output.txt
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (TEZ-4119) TestSpeculation is flaky
[
https://issues.apache.org/jira/browse/TEZ-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024431#comment-17024431
]
László Bodor edited comment on TEZ-4119 at 1/27/20 3:56 PM:
sometimes it can be reproduced locally, by running all tez-dag tests (normally,
TestSpeculation should finish in 3-5 seconds):
from tez-dag dir:
{code}
mvn --batch-mode clean test -fae
{code}
there is a suspicious no-op for almost 2 minutes:
{code}
2020-01-27 16:33:22,162 INFO [Dispatcher thread {Central}] app.DAGAppMaster
(DAGAppMaster.java:handle(872)) - Completed cleanup for DAG: name=test, with
id=dag_1580139200870_0001_1
2020-01-27 16:35:00,864 INFO [Thread-53] client.TezClient
(TezClient.java:(210)) - Tez Client Version: [ component=tez-api,
version=0.10.1-SNAPSHOT, revision=f049d93a69bdc5bd736301bf7081fa5cef2694bf,
SCM-URL=scm:git:https://gitbox.apache.org/repos/asf/tez.git,
buildTime=2020-01-27T09:27:08Z ]
{code}
jstacks of the surefire process: [^jstack.log] ...later: [^jstack4.log]
...later: [^jstack6.log]
something strange:
seems like speculation is not triggered
{code}
// Wait enough time to give chance for the speculator to trigger
// speculation on VB.
// This would fail because of JUnit time out.
do {
Thread.sleep(100);
} while (vSpec.getAllCounters().findCounter(TaskCounter.NUM_SPECULATIONS)
.getValue() <= 0);
{code}
{code}
"Thread-3" #16 prio=5 os_prio=31 tid=0x7fe07a3bc800 nid=0xa403 waiting on
condition [0x762d8000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.tez.dag.app.TestSpeculation.testBasicSpeculationPerVertexConf(TestSpeculation.java:261)
{code}
10 seconds later:
{code}
"Thread-3" #16 prio=5 os_prio=31 tid=0x7fe07a3bc800 nid=0xa403 waiting on
condition [0x762d8000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.tez.dag.app.TestSpeculation.testBasicSpeculationPerVertexConf(TestSpeculation.java:261)
{code}
was (Author: abstractdog):
sometimes it can be reproduced locally, by running all tez-dag tests (normally,
TestSpeculation should finish in 3-5 seconds):
from tez-dag dir:
{code}
mvn --batch-mode clean test -fae
{code}
there is a suspicious no-op for almost 2 minutes:
{code}
2020-01-27 16:33:22,162 INFO [Dispatcher thread {Central}] app.DAGAppMaster
(DAGAppMaster.java:handle(872)) - Completed cleanup for DAG: name=test, with
id=dag_1580139200870_0001_1
2020-01-27 16:35:00,864 INFO [Thread-53] client.TezClient
(TezClient.java:(210)) - Tez Client Version: [ component=tez-api,
version=0.10.1-SNAPSHOT, revision=f049d93a69bdc5bd736301bf7081fa5cef2694bf,
SCM-URL=scm:git:https://gitbox.apache.org/repos/asf/tez.git,
buildTime=2020-01-27T09:27:08Z ]
{code}
jstacks of the surefire process: [^jstack.log] ...later: [^jstack4.log]
...later: [^jstack6.log]
something strange:
testBasicSpeculationPerVertexConf contains a 200ms sleep, but it tooks much
longer
{code}
"Thread-3" #16 prio=5 os_prio=31 tid=0x7fe07a3bc800 nid=0xa403 waiting on
condition [0x762d8000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.tez.dag.app.TestSpeculation.testBasicSpeculationPerVertexConf(TestSpeculation.java:261)
{code}
10 seconds later:
{code}
"Thread-3" #16 prio=5 os_prio=31 tid=0x7fe07a3bc800 nid=0xa403 waiting on
condition [0x762d8000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.tez.dag.app.TestSpeculation.testBasicSpeculationPerVertexConf(TestSpeculation.java:261)
{code}
> TestSpeculation is flaky
>
>
> Key: TEZ-4119
> URL: https://issues.apache.org/jira/browse/TEZ-4119
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: jstack.log, jstack4.log, jstack6.log,
> org.apache.tez.dag.app.TestSpeculation-output.txt
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TEZ-4101) Eliminate some guava dependencies by Java8+ features - Preconditions
[
https://issues.apache.org/jira/browse/TEZ-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024353#comment-17024353
]
TezQA commented on TEZ-4101:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 9m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
2s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 17 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
33s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 18s{color} | {color:orange} tez-api: The patch generated 2 new + 316
unchanged - 0 fixed = 318 total (was 316) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 9s{color} | {color:orange} tez-common: The patch generated 2 new + 37
unchanged - 0 fixed = 39 total (was 37) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
40s{color} | {color:green} tez-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} tez-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
37s{color} | {color:green} tez-runtime-internals in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
22s{color} | {color:green} tez-runtime-library in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
1s{color} | {color:green} tez-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
11s{color} | {color:green} tez-examples in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 5m 19s{color}
| {color:red} tez-dag in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 44m 25s{color}
| {color:red} tez-tests in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
35s{color} | {color:green} tez-ext-service-tests in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
4s{color} | {color:green} tez-history-parser in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit
[jira] [Assigned] (TEZ-4119) TestSpeculation is flaky
[ https://issues.apache.org/jira/browse/TEZ-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned TEZ-4119: - Assignee: László Bodor > TestSpeculation is flaky > > > Key: TEZ-4119 > URL: https://issues.apache.org/jira/browse/TEZ-4119 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (TEZ-4119) TestSpeculation is flaky
László Bodor created TEZ-4119: - Summary: TestSpeculation is flaky Key: TEZ-4119 URL: https://issues.apache.org/jira/browse/TEZ-4119 Project: Apache Tez Issue Type: Improvement Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-4101) Eliminate some guava dependencies by Java8+ features - Preconditions
[ https://issues.apache.org/jira/browse/TEZ-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated TEZ-4101: -- Attachment: TEZ-4101.06.patch > Eliminate some guava dependencies by Java8+ features - Preconditions > > > Key: TEZ-4101 > URL: https://issues.apache.org/jira/browse/TEZ-4101 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4101.01.patch, TEZ-4101.02.patch, TEZ-4101.03.patch, > TEZ-4101.04.patch, TEZ-4101.05.patch, TEZ-4101.06.patch, TEZ-4101.06.patch > > > 1. Preconditions.checkNotNull -> Objects.requireNonNull > 2. wrap others into a tez common class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-4101) Eliminate some guava dependencies by Java8+ features - Preconditions
[ https://issues.apache.org/jira/browse/TEZ-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024420#comment-17024420 ] László Bodor commented on TEZ-4101: --- TestSpeculation passes locally (created TEZ-4119 about that), reuploading last patch > Eliminate some guava dependencies by Java8+ features - Preconditions > > > Key: TEZ-4101 > URL: https://issues.apache.org/jira/browse/TEZ-4101 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4101.01.patch, TEZ-4101.02.patch, TEZ-4101.03.patch, > TEZ-4101.04.patch, TEZ-4101.05.patch, TEZ-4101.06.patch, TEZ-4101.06.patch > > > 1. Preconditions.checkNotNull -> Objects.requireNonNull > 2. wrap others into a tez common class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-4119) TestSpeculation is flaky
[ https://issues.apache.org/jira/browse/TEZ-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated TEZ-4119: -- Attachment: TestSpeculation.txt > TestSpeculation is flaky > > > Key: TEZ-4119 > URL: https://issues.apache.org/jira/browse/TEZ-4119 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TestSpeculation.txt > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-4119) TestSpeculation is flaky
[
https://issues.apache.org/jira/browse/TEZ-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024431#comment-17024431
]
László Bodor commented on TEZ-4119:
---
sometimes it can be reproduced locally, by running all tez-dag tests (normally,
TestSpeculation should finish in 3-5 seconds):
from tez-dag dir:
{code}
mvn --batch-mode clean test -fae
{code}
> TestSpeculation is flaky
>
>
> Key: TEZ-4119
> URL: https://issues.apache.org/jira/browse/TEZ-4119
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: TestSpeculation.txt
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (TEZ-4119) TestSpeculation is flaky
[
https://issues.apache.org/jira/browse/TEZ-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024431#comment-17024431
]
László Bodor edited comment on TEZ-4119 at 1/27/20 3:38 PM:
sometimes it can be reproduced locally, by running all tez-dag tests (normally,
TestSpeculation should finish in 3-5 seconds):
from tez-dag dir:
{code}
mvn --batch-mode clean test -fae
{code}
there is a suspicious no-op for almost 2 minutes:
{code}
2020-01-27 16:33:22,162 INFO [Dispatcher thread {Central}] app.DAGAppMaster
(DAGAppMaster.java:handle(872)) - Completed cleanup for DAG: name=test, with
id=dag_1580139200870_0001_1
2020-01-27 16:35:00,864 INFO [Thread-53] client.TezClient
(TezClient.java:(210)) - Tez Client Version: [ component=tez-api,
version=0.10.1-SNAPSHOT, revision=f049d93a69bdc5bd736301bf7081fa5cef2694bf,
SCM-URL=scm:git:https://gitbox.apache.org/repos/asf/tez.git,
buildTime=2020-01-27T09:27:08Z ]
{code}
was (Author: abstractdog):
sometimes it can be reproduced locally, by running all tez-dag tests (normally,
TestSpeculation should finish in 3-5 seconds):
from tez-dag dir:
{code}
mvn --batch-mode clean test -fae
{code}
> TestSpeculation is flaky
>
>
> Key: TEZ-4119
> URL: https://issues.apache.org/jira/browse/TEZ-4119
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: org.apache.tez.dag.app.TestSpeculation-output.txt
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (TEZ-4119) TestSpeculation is flaky
[ https://issues.apache.org/jira/browse/TEZ-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated TEZ-4119: -- Attachment: jstack.log jstack6.log > TestSpeculation is flaky > > > Key: TEZ-4119 > URL: https://issues.apache.org/jira/browse/TEZ-4119 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: jstack.log, jstack6.log, > org.apache.tez.dag.app.TestSpeculation-output.txt > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-4119) TestSpeculation is flaky
[ https://issues.apache.org/jira/browse/TEZ-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated TEZ-4119: -- Attachment: jstack4.log > TestSpeculation is flaky > > > Key: TEZ-4119 > URL: https://issues.apache.org/jira/browse/TEZ-4119 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: jstack.log, jstack4.log, jstack6.log, > org.apache.tez.dag.app.TestSpeculation-output.txt > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (TEZ-4119) TestSpeculation is flaky
[
https://issues.apache.org/jira/browse/TEZ-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024431#comment-17024431
]
László Bodor edited comment on TEZ-4119 at 1/27/20 3:51 PM:
sometimes it can be reproduced locally, by running all tez-dag tests (normally,
TestSpeculation should finish in 3-5 seconds):
from tez-dag dir:
{code}
mvn --batch-mode clean test -fae
{code}
there is a suspicious no-op for almost 2 minutes:
{code}
2020-01-27 16:33:22,162 INFO [Dispatcher thread {Central}] app.DAGAppMaster
(DAGAppMaster.java:handle(872)) - Completed cleanup for DAG: name=test, with
id=dag_1580139200870_0001_1
2020-01-27 16:35:00,864 INFO [Thread-53] client.TezClient
(TezClient.java:(210)) - Tez Client Version: [ component=tez-api,
version=0.10.1-SNAPSHOT, revision=f049d93a69bdc5bd736301bf7081fa5cef2694bf,
SCM-URL=scm:git:https://gitbox.apache.org/repos/asf/tez.git,
buildTime=2020-01-27T09:27:08Z ]
{code}
jstacks of the surefire process: [^jstack.log] ...later: [^jstack4.log]
...later: [^jstack6.log]
something strange:
testBasicSpeculationPerVertexConf contains a 200ms sleep, but it tooks much
longer
{code}
"Thread-3" #16 prio=5 os_prio=31 tid=0x7fe07a3bc800 nid=0xa403 waiting on
condition [0x762d8000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.tez.dag.app.TestSpeculation.testBasicSpeculationPerVertexConf(TestSpeculation.java:261)
{code}
10 seconds later:
{code}
"Thread-3" #16 prio=5 os_prio=31 tid=0x7fe07a3bc800 nid=0xa403 waiting on
condition [0x762d8000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.tez.dag.app.TestSpeculation.testBasicSpeculationPerVertexConf(TestSpeculation.java:261)
{code}
was (Author: abstractdog):
sometimes it can be reproduced locally, by running all tez-dag tests (normally,
TestSpeculation should finish in 3-5 seconds):
from tez-dag dir:
{code}
mvn --batch-mode clean test -fae
{code}
there is a suspicious no-op for almost 2 minutes:
{code}
2020-01-27 16:33:22,162 INFO [Dispatcher thread {Central}] app.DAGAppMaster
(DAGAppMaster.java:handle(872)) - Completed cleanup for DAG: name=test, with
id=dag_1580139200870_0001_1
2020-01-27 16:35:00,864 INFO [Thread-53] client.TezClient
(TezClient.java:(210)) - Tez Client Version: [ component=tez-api,
version=0.10.1-SNAPSHOT, revision=f049d93a69bdc5bd736301bf7081fa5cef2694bf,
SCM-URL=scm:git:https://gitbox.apache.org/repos/asf/tez.git,
buildTime=2020-01-27T09:27:08Z ]
{code}
jstacks of the surefire process: [^jstack.log] ...later: [^jstack6.log]
> TestSpeculation is flaky
>
>
> Key: TEZ-4119
> URL: https://issues.apache.org/jira/browse/TEZ-4119
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: jstack.log, jstack4.log, jstack6.log,
> org.apache.tez.dag.app.TestSpeculation-output.txt
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
