[
https://issues.apache.org/jira/browse/CASSANDRA-16655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-16655:
----------------------------------------
Description:
Recent jenkins runs appear broken (red) bc of a cqlsh test failure. That
reports the whole run as broken instead of that specific step with failures and
an unstable (yellow) run.
It appears the reason may be
[this|https://github.com/apache/cassandra/blame/trunk/pylib/cassandra-cqlsh-tests.sh#L146]
change. At that time that was necessary to make circle fail that step.
Otherwise cqlsh failures would be ignored and that step rendered green
regarless making it useless. But it has been found now this makes jenkins
report the whole run red upon a cqlsh test failure.
The solution seems to be to return different codes depending on whether we're
on jenkins or circle. [~mck] has suggested sthg along these lines:
{code}
# circleci wants non-zero exit to report failures, jenkins wants zero exit for
a unstable status
[[ command -v circleci >/dev/null 2>&1 ]] && exit ${RETURN}
exit 0
{code}
Procedure:
- Fix the sh script
- Break a cqlsh test
- Test the test gets reported as a failure in circle
- Test the test gets reported as a failure in jenkins and that the run and step
are yellow (not red)
- Fix the test
- Both circle and jenkins report green for that step
was:
ci-cassandra.apache.org distinguishes between FAILURE as unable to run the
tests, and UNSTABLE as failed tests. A FAILURE (red) comes from a non-zero
result on the job. An UNSTABLE comes from parsing the test results.
circleci only has two states: green or red, and requires a non-zero result to
identify failed tests.
> cassandra-cqlsh-tests.sh should return different return codes on circle vs
> jenkins
> ----------------------------------------------------------------------------------
>
> Key: CASSANDRA-16655
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16655
> Project: Cassandra
> Issue Type: Bug
> Components: CI
> Reporter: Berenguer Blasi
> Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0-rc, 4.x
>
>
> Recent jenkins runs appear broken (red) bc of a cqlsh test failure. That
> reports the whole run as broken instead of that specific step with failures
> and an unstable (yellow) run.
> It appears the reason may be
> [this|https://github.com/apache/cassandra/blame/trunk/pylib/cassandra-cqlsh-tests.sh#L146]
> change. At that time that was necessary to make circle fail that step.
> Otherwise cqlsh failures would be ignored and that step rendered green
> regarless making it useless. But it has been found now this makes jenkins
> report the whole run red upon a cqlsh test failure.
> The solution seems to be to return different codes depending on whether we're
> on jenkins or circle. [~mck] has suggested sthg along these lines:
> {code}
> # circleci wants non-zero exit to report failures, jenkins wants zero exit
> for a unstable status
> [[ command -v circleci >/dev/null 2>&1 ]] && exit ${RETURN}
> exit 0
> {code}
> Procedure:
> - Fix the sh script
> - Break a cqlsh test
> - Test the test gets reported as a failure in circle
> - Test the test gets reported as a failure in jenkins and that the run and
> step are yellow (not red)
> - Fix the test
> - Both circle and jenkins report green for that step
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
