ASF GitHub Bot commented on FLINK-9257:

Github user florianschmidt1994 commented on a diff in the pull request:

    --- Diff: 
flink-end-to-end-tests/test-scripts/test_local_recovery_and_scheduling.sh ---
    @@ -71,8 +69,14 @@ function run_local_recovery_test {
         local incremental=$4
         local kill_jvm=$5
    -    echo "Running local recovery test on ${backend} backend: incremental 
checkpoints = ${incremental}, kill JVM = ${kill_jvm}."
    +    echo "Running local recovery test with configuration:
    +        parallelism: ${parallelism}
    +        max attempts: ${max_attempts}
    +        backend: ${backend}
    +        incremental checkpoints: ${incremental}
    +        kill JVM: ${kill_jvm}"
    --- End diff --
    Only works if we export END_TO_END_DIR in run_nightly/precommit_tests.sh, 
but I'll just add that there as well

> End-to-end tests prints "All tests PASS" even if individual test-script 
> returns non-zero exit code
> --------------------------------------------------------------------------------------------------
>                 Key: FLINK-9257
>                 URL: https://issues.apache.org/jira/browse/FLINK-9257
>             Project: Flink
>          Issue Type: Bug
>          Components: Tests
>    Affects Versions: 1.5.0
>            Reporter: Florian Schmidt
>            Assignee: Florian Schmidt
>            Priority: Critical
>             Fix For: 1.6.0
> In some cases the test-suite exits with non-zero exit code but still prints 
> "All tests PASS" to stdout. This happens because how the test runner works, 
> which is roughly as follows
>  # Either run-nightly-tests.sh or run-precommit-tests.sh executes a suite of 
> tests consisting of one multiple bash scripts.
>  # As soon as one of those bash scripts exists with non-zero exit code, the 
> tests won't continue to run and the test-suite will also exit with non-zero 
> exit code.
>  # *During the cleanup hook (trap cleanup EXIT in common.sh) it will be 
> checked whether there are non-empty out files or log files with certain 
> exceptions. If a tests fails with non-zero exit code, but does not have any 
> exceptions or .out files, this will still print "All tests PASS" to stdout, 
> even though they don't*

This message was sent by Atlassian JIRA

Reply via email to