[
https://issues.apache.org/jira/browse/IGNITE-9640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Kuznetsov updated IGNITE-9640:
-------------------------------------
Description:
When the test is already flaky in master branch, developer has to check if
possible failures in PR are the same as in master branch. As for now, this is
done manually by analyzing build history on TeamCity. TC Bot can simplify this
process in the following way:
* When both error descriptions from PR/master build logs start with identical
substring, then the failure should not be reported.
* Otherwise test failure should be reported.
Example of potentially identical substring:
{noformat}
junit.framework.AssertionFailedError
at
org.apache.ignite.internal.GridVersionSelfTest.testVersions(GridVersionSelfTest.java:54)
{noformat}
was:
When someone is analyzing flaky test failure, it's important to distinguish
between newly created failure and pre-existing one. In the latter case, the bot
should not attract contributor's attention to the test.
In more detail, TC build log fragments start with identical substrings for
identical failures very often, e.g.
{noformat}
junit.framework.AssertionFailedError
at
org.apache.ignite.internal.GridVersionSelfTest.testVersions(GridVersionSelfTest.java:54)
{noformat}
> [TC Bot] Determine repetitive failure types by analyzing build log
> ------------------------------------------------------------------
>
> Key: IGNITE-9640
> URL: https://issues.apache.org/jira/browse/IGNITE-9640
> Project: Ignite
> Issue Type: Task
> Reporter: Andrey Kuznetsov
> Priority: Minor
>
> When the test is already flaky in master branch, developer has to check if
> possible failures in PR are the same as in master branch. As for now, this is
> done manually by analyzing build history on TeamCity. TC Bot can simplify
> this process in the following way:
> * When both error descriptions from PR/master build logs start with identical
> substring, then the failure should not be reported.
> * Otherwise test failure should be reported.
> Example of potentially identical substring:
> {noformat}
> junit.framework.AssertionFailedError
> at
> org.apache.ignite.internal.GridVersionSelfTest.testVersions(GridVersionSelfTest.java:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
