[
https://issues.apache.org/jira/browse/FLINK-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16486731#comment-16486731
]
ASF GitHub Bot commented on FLINK-9257:
---------------------------------------
Github user tzulitai commented on a diff in the pull request:
https://github.com/apache/flink/pull/6053#discussion_r190125404
--- Diff: flink-end-to-end-tests/test-scripts/test-runner-common.sh ---
@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(pwd)"/test-scripts/common.sh
+
+#######################################
+# Prints the given description, runs the given test and prints how long
the execution took.
+# Arguments:
+# $1: description of the test
+# $2: command to execute
+#######################################
+function run_test {
+ description="$1"
+ command="$2"
+
+ printf
"\n==============================================================================\n"
+ printf "Running '${description}'\n"
+ printf
"==============================================================================\n"
+ start_timer
+ ${command}
+ exit_code="$?"
+ time_elapsed=$(end_timer)
+
+ check_logs_for_errors
+ check_logs_for_exceptions
+ check_logs_for_non_empty_out_files
--- End diff --
Maybe we should move all these methods:
```
start_timer
end_timer
check_logs_for_errors
check_logs_for_exceptions
check_logs_for_non_empty_out_files
```
to `test-runner-common.sh` since that's the only place they are used anyways
> End-to-end tests prints "All tests PASS" even if individual test-script
> returns non-zero exit code
> --------------------------------------------------------------------------------------------------
>
> Key: FLINK-9257
> URL: https://issues.apache.org/jira/browse/FLINK-9257
> Project: Flink
> Issue Type: Bug
> Components: Tests
> Affects Versions: 1.5.0
> Reporter: Florian Schmidt
> Assignee: Florian Schmidt
> Priority: Critical
> Fix For: 1.5.0
>
>
> In some cases the test-suite exits with non-zero exit code but still prints
> "All tests PASS" to stdout. This happens because how the test runner works,
> which is roughly as follows
> # Either run-nightly-tests.sh or run-precommit-tests.sh executes a suite of
> tests consisting of one multiple bash scripts.
> # As soon as one of those bash scripts exists with non-zero exit code, the
> tests won't continue to run and the test-suite will also exit with non-zero
> exit code.
> # *During the cleanup hook (trap cleanup EXIT in common.sh) it will be
> checked whether there are non-empty out files or log files with certain
> exceptions. If a tests fails with non-zero exit code, but does not have any
> exceptions or .out files, this will still print "All tests PASS" to stdout,
> even though they don't*
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
