[GitHub] [flink] azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests] Harden test_streaming_kinesis with kinesalite docker image download/run retries
azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests]
Harden test_streaming_kinesis with kinesalite docker image download/run retries
URL: https://github.com/apache/flink/pull/9419#discussion_r313775744
##
File path: flink-end-to-end-tests/test-scripts/common.sh
##
@@ -747,3 +747,47 @@ function retry_times() {
echo "Command: ${command} failed ${retriesNumber} times."
return 1
}
+
+##
+# Retry action until it has successfully exited and wait for condition.
+#
+# Globals:
+# -
+# Arguments:
+# $1 - action to run
+# $2 - boolean function to check whether the action has been successfully
done
+# $3 - action name for logs
+# $4 - (default: 10) max number of retries to run action until passed and
check done
+# $5 - (default: 0) backoff delay in seconds between retry attempts
+# Returns:
+# Done or failed message
+###
+function retry_until_passed_and_wait_condition {
Review comment:
We cannot retry docker run if it has succeeded, only checking that the
container started,
but true, it is possible to reuse `retry_times` in 2 steps. I pushed the
change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests] Harden test_streaming_kinesis with kinesalite docker image download/run retries
azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests]
Harden test_streaming_kinesis with kinesalite docker image download/run retries
URL: https://github.com/apache/flink/pull/9419#discussion_r313755688
##
File path: flink-end-to-end-tests/test-scripts/test_streaming_kinesis.sh
##
@@ -28,9 +28,26 @@ export AWS_SECRET_KEY=flinkKinesisTestFakeAccessKey
KINESALITE_PORT=4567
-#docker run -d --rm --name flink-test-kinesis -p
${KINESALITE_PORT}:${KINESALITE_PORT} instructure/kinesalite
-# override entrypoint to enable SSL
-docker run -d --rm --entrypoint "/tini" --name flink-test-kinesis -p
${KINESALITE_PORT}:${KINESALITE_PORT} instructure/kinesalite --
/usr/src/app/node_modules/kinesalite/cli.js --path /var/lib/kinesalite --ssl
+function START_KINESALITE {
+#docker run -d --rm --name flink-test-kinesis -p
${KINESALITE_PORT}:${KINESALITE_PORT} instructure/kinesalite
+# override entrypoint to enable SSL
+docker run -d --rm --entrypoint "/tini" \
+--name flink-test-kinesis \
+-p ${KINESALITE_PORT}:${KINESALITE_PORT} \
+instructure/kinesalite -- \
+/usr/src/app/node_modules/kinesalite/cli.js --path /var/lib/kinesalite
--ssl
+}
+
+function KINESALITE_STARTED {
+if [[ "$(docker ps --format '{{.Names}}' | grep flink-test-kinesis)" =
"flink-test-kinesis" ]]; then
+return 0
+else
+return 1
+fi
+}
+
+START_KINESALITE_MAX_RETRY_NUMBER=10
+retry_until_passed_and_wait_condition START_KINESALITE KINESALITE_STARTED
"start kinesalite" ${START_KINESALITE_MAX_RETRY_NUMBER}
Review comment:
I have not looped it on travis, only tested it by turning off and on
connection and seeing that it downloads docker image eventually. I can try
looping it on Travis but not sure we can do much more against the download
hiccups.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests] Harden test_streaming_kinesis with kinesalite docker image download/run retries
azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests]
Harden test_streaming_kinesis with kinesalite docker image download/run retries
URL: https://github.com/apache/flink/pull/9419#discussion_r313752529
##
File path: flink-end-to-end-tests/test-scripts/test_streaming_kinesis.sh
##
@@ -28,9 +28,26 @@ export AWS_SECRET_KEY=flinkKinesisTestFakeAccessKey
KINESALITE_PORT=4567
-#docker run -d --rm --name flink-test-kinesis -p
${KINESALITE_PORT}:${KINESALITE_PORT} instructure/kinesalite
-# override entrypoint to enable SSL
-docker run -d --rm --entrypoint "/tini" --name flink-test-kinesis -p
${KINESALITE_PORT}:${KINESALITE_PORT} instructure/kinesalite --
/usr/src/app/node_modules/kinesalite/cli.js --path /var/lib/kinesalite --ssl
+function START_KINESALITE {
Review comment:
true, leftover
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
