[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[ https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940236#comment-15940236 ] Peter Cseh commented on OOZIE-2817: --- Thanks for the review [~abhishekbafna]! I promise I won't do an other amend commit for this jira :) > Increase test case stability in pre-commit job > -- > > Key: OOZIE-2817 > URL: https://issues.apache.org/jira/browse/OOZIE-2817 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Peter Cseh > Fix For: 5.0.0 > > Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch, > OOZIE-2817-003.patch, OOZIE-2817-amend2.patch, OOZIE-2817-amend.patch > > > We could collect the failed tests and rerun them in the pre-commit hook. > They will still fail if they are broken, but they'll probably pass if they > are just flaky. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[ https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940228#comment-15940228 ] Abhishek Bafna commented on OOZIE-2817: --- +1. Committed to master. Thanks [~gezapeti]. Just a thought, going forward we should create a separate issue for this, as we have already done two amends. :) > Increase test case stability in pre-commit job > -- > > Key: OOZIE-2817 > URL: https://issues.apache.org/jira/browse/OOZIE-2817 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Peter Cseh > Fix For: 5.0.0 > > Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch, > OOZIE-2817-003.patch, OOZIE-2817-amend2.patch, OOZIE-2817-amend.patch > > > We could collect the failed tests and rerun them in the pre-commit hook. > They will still fail if they are broken, but they'll probably pass if they > are just flaky. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[ https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939029#comment-15939029 ] Peter Cseh commented on OOZIE-2817: --- Thanks [~abhishekbafna], for the investigation! You're right about FAILURES vs failures I'll fix it with an other amend commit I guess :) I don't know why the number of tests is different in Jenkins and in our report. I haven't touched that part but it looks like it just sums up the numbers from the Junit output. Maybe we're running some tests twice as part of a suite and Jenkins merges those results? > Increase test case stability in pre-commit job > -- > > Key: OOZIE-2817 > URL: https://issues.apache.org/jira/browse/OOZIE-2817 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Peter Cseh > Fix For: 5.0.0 > > Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch, > OOZIE-2817-003.patch, OOZIE-2817-amend.patch > > > We could collect the failed tests and rerun them in the pre-commit hook. > They will still fail if they are broken, but they'll probably pass if they > are just flaky. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[
https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15938192#comment-15938192
]
Abhishek Bafna commented on OOZIE-2817:
---
For the Build:
https://builds.apache.org/job/oozie-trunk-precommit-build/3722/consoleFull.
This build had an flask test which got passed in the rerun. But that
information did not got printed in the Jira
[comment|https://issues.apache.org/jira/browse/OOZIE-2701?focusedCommentId=15938155=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15938155].
{code}
flakyTests=$(cat ${TEMPDIR}/${TASKNAME}.flakies.out)
echo "{color:green}+1 ${TASKNAME}{color}" >> $SUMMARYFILE
echo ".Tests run: $testsRun" >> $SUMMARYFILE
if [ -n "${flakyTests}" ]; then
echo ".Tests rerun: ${testsReRun}" >> $SUMMARYFILE
echo ".Tests failed at first run: ${flakyTests}" >> $SUMMARYFILE
{code}
I think issue was:
{code}
failures=$(find . -name '*\.txt' | grep target/surefire-reports | xargs
grep "<<<" | grep -v "Tests run:" | sed 's/.*\.txt\://' | sed 's/ .*//' | grep
-o '(\(.*\))' | tr -d '()' | uniq | awk '{ sub ("$", ""); printf "%s",
$0"," } END { print "" }')
if [ -n "${failures}" ]; then
echo "Rerunning failed tests: ${failures}"
echo $FAILURES > ${TEMPDIR}/${TASKNAME}.flakies.out
{code}
The case for the variable {{FAILURES}} vs {{failures}}.
Total tests run in Jenkins Test report: 1,909 (Link:
https://builds.apache.org/job/oozie-trunk-precommit-build/3722/testReport)
Total tests run in Comment from Report: 1912.
Not sure, why there is difference, could not reach to a conclusion.
[~gezapeti] Can you please take look here?
Thanks.
> Increase test case stability in pre-commit job
> --
>
> Key: OOZIE-2817
> URL: https://issues.apache.org/jira/browse/OOZIE-2817
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Fix For: 5.0.0
>
> Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch,
> OOZIE-2817-003.patch, OOZIE-2817-amend.patch
>
>
> We could collect the failed tests and rerun them in the pre-commit hook.
> They will still fail if they are broken, but they'll probably pass if they
> are just flaky.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[ https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936504#comment-15936504 ] Peter Cseh commented on OOZIE-2817: --- Thanks for the review! Committed to master! > Increase test case stability in pre-commit job > -- > > Key: OOZIE-2817 > URL: https://issues.apache.org/jira/browse/OOZIE-2817 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Peter Cseh > Fix For: 5.0.0 > > Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch, > OOZIE-2817-003.patch, OOZIE-2817-amend.patch > > > We could collect the failed tests and rerun them in the pre-commit hook. > They will still fail if they are broken, but they'll probably pass if they > are just flaky. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[ https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936503#comment-15936503 ] Abhishek Bafna commented on OOZIE-2817: --- +1. > Increase test case stability in pre-commit job > -- > > Key: OOZIE-2817 > URL: https://issues.apache.org/jira/browse/OOZIE-2817 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Peter Cseh > Fix For: 5.0.0 > > Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch, > OOZIE-2817-003.patch, OOZIE-2817-amend.patch > > > We could collect the failed tests and rerun them in the pre-commit hook. > They will still fail if they are broken, but they'll probably pass if they > are just flaky. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[ https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936433#comment-15936433 ] Peter Cseh commented on OOZIE-2817: --- Thanks for the review! Committed to master > Increase test case stability in pre-commit job > -- > > Key: OOZIE-2817 > URL: https://issues.apache.org/jira/browse/OOZIE-2817 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Peter Cseh > Fix For: 5.0.0 > > Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch, > OOZIE-2817-003.patch > > > We could collect the failed tests and rerun them in the pre-commit hook. > They will still fail if they are broken, but they'll probably pass if they > are just flaky. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[ https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930947#comment-15930947 ] Robert Kanter commented on OOZIE-2817: -- It's hard to verify that this is working correctly, but it seems okay as far as I can tell. I think the best way is to commit it and we can see it working as we work on other JIRAs; we can always fix/update it later if need be. +1 > Increase test case stability in pre-commit job > -- > > Key: OOZIE-2817 > URL: https://issues.apache.org/jira/browse/OOZIE-2817 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Peter Cseh > Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch, > OOZIE-2817-003.patch > > > We could collect the failed tests and rerun them in the pre-commit hook. > They will still fail if they are broken, but they'll probably pass if they > are just flaky. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[
https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930849#comment-15930849
]
Hadoop QA commented on OOZIE-2817:
--
Testing JIRA OOZIE-2817
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 7 line(s) longer than 132
characters
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [core].
.{color:green}+1{color} There are no new bugs found in
[hadooplibs/hadoop-utils-2].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1892
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3701/
> Increase test case stability in pre-commit job
> --
>
> Key: OOZIE-2817
> URL: https://issues.apache.org/jira/browse/OOZIE-2817
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch,
> OOZIE-2817-003.patch
>
>
> We could collect the failed tests and rerun them in the pre-commit hook.
> They will still fail if they are broken, but they'll probably pass if they
> are just flaky.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[ https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930673#comment-15930673 ] Peter Cseh commented on OOZIE-2817: --- I don't think the pre-commit job finished for the second time either. I'm attaching the patch again. > Increase test case stability in pre-commit job > -- > > Key: OOZIE-2817 > URL: https://issues.apache.org/jira/browse/OOZIE-2817 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Peter Cseh > Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch > > > We could collect the failed tests and rerun them in the pre-commit hook. > They will still fail if they are broken, but they'll probably pass if they > are just flaky. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[ https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905910#comment-15905910 ] Robert Kanter commented on OOZIE-2817: -- Let me kick this off again > Increase test case stability in pre-commit job > -- > > Key: OOZIE-2817 > URL: https://issues.apache.org/jira/browse/OOZIE-2817 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Peter Cseh > Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch > > > We could collect the failed tests and rerun them in the pre-commit hook. > They will still fail if they are broken, but they'll probably pass if they > are just flaky. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[
https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905905#comment-15905905
]
Hadoop QA commented on OOZIE-2817:
--
Testing JIRA OOZIE-2817
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 7 line(s) longer than 132
characters
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [core].
.{color:green}+1{color} There are no new bugs found in
[hadooplibs/hadoop-utils-2].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3686/
> Increase test case stability in pre-commit job
> --
>
> Key: OOZIE-2817
> URL: https://issues.apache.org/jira/browse/OOZIE-2817
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2817-001.patch, OOZIE-2817-002.patch
>
>
> We could collect the failed tests and rerun them in the pre-commit hook.
> They will still fail if they are broken, but they'll probably pass if they
> are just flaky.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[
https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904175#comment-15904175
]
Hadoop QA commented on OOZIE-2817:
--
Testing JIRA OOZIE-2817
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132
characters
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
.{color:green}+1{color} There are no new bugs found in [core].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in
[hadooplibs/hadoop-utils-2].
.{color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 54
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3677/
> Increase test case stability in pre-commit job
> --
>
> Key: OOZIE-2817
> URL: https://issues.apache.org/jira/browse/OOZIE-2817
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2817-001.patch
>
>
> We could collect the failed tests and rerun them in the pre-commit hook.
> They will still fail if they are broken, but they'll probably pass if they
> are just flaky.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2817) Increase test case stability in pre-commit job
[ https://issues.apache.org/jira/browse/OOZIE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904088#comment-15904088 ] Robert Kanter commented on OOZIE-2817: -- This is a great idea. It should be helpful to get more green runs from Jenkins. Some minor things: # Can you run [shellcheck|https://github.com/koalaman/shellcheck] on this and fix the (reasonable) things it finds? # Can you have it report what the flakey tests were (i.e. they failed the first run but passed on the rerun)? Otherwise, your changes will hide the flakey tests and we won't have a good idea of which tests are flakey anymore. > Increase test case stability in pre-commit job > -- > > Key: OOZIE-2817 > URL: https://issues.apache.org/jira/browse/OOZIE-2817 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Peter Cseh > Attachments: OOZIE-2817-001.patch > > > We could collect the failed tests and rerun them in the pre-commit hook. > They will still fail if they are broken, but they'll probably pass if they > are just flaky. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
