[GitHub] [flink] azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests] Harden test_streaming_kinesis with kinesalite docker image download/run retries
azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests] Harden test_streaming_kinesis with kinesalite docker image download/run retries URL: https://github.com/apache/flink/pull/9419#discussion_r313775744 ## File path: flink-end-to-end-tests/test-scripts/common.sh ## @@ -747,3 +747,47 @@ function retry_times() { echo "Command: ${command} failed ${retriesNumber} times." return 1 } + +## +# Retry action until it has successfully exited and wait for condition. +# +# Globals: +# - +# Arguments: +# $1 - action to run +# $2 - boolean function to check whether the action has been successfully done +# $3 - action name for logs +# $4 - (default: 10) max number of retries to run action until passed and check done +# $5 - (default: 0) backoff delay in seconds between retry attempts +# Returns: +# Done or failed message +### +function retry_until_passed_and_wait_condition { Review comment: We cannot retry docker run if it has succeeded, only checking that the container started, but true, it is possible to reuse `retry_times` in 2 steps. I pushed the change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests] Harden test_streaming_kinesis with kinesalite docker image download/run retries
azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests] Harden test_streaming_kinesis with kinesalite docker image download/run retries URL: https://github.com/apache/flink/pull/9419#discussion_r313755688 ## File path: flink-end-to-end-tests/test-scripts/test_streaming_kinesis.sh ## @@ -28,9 +28,26 @@ export AWS_SECRET_KEY=flinkKinesisTestFakeAccessKey KINESALITE_PORT=4567 -#docker run -d --rm --name flink-test-kinesis -p ${KINESALITE_PORT}:${KINESALITE_PORT} instructure/kinesalite -# override entrypoint to enable SSL -docker run -d --rm --entrypoint "/tini" --name flink-test-kinesis -p ${KINESALITE_PORT}:${KINESALITE_PORT} instructure/kinesalite -- /usr/src/app/node_modules/kinesalite/cli.js --path /var/lib/kinesalite --ssl +function START_KINESALITE { +#docker run -d --rm --name flink-test-kinesis -p ${KINESALITE_PORT}:${KINESALITE_PORT} instructure/kinesalite +# override entrypoint to enable SSL +docker run -d --rm --entrypoint "/tini" \ +--name flink-test-kinesis \ +-p ${KINESALITE_PORT}:${KINESALITE_PORT} \ +instructure/kinesalite -- \ +/usr/src/app/node_modules/kinesalite/cli.js --path /var/lib/kinesalite --ssl +} + +function KINESALITE_STARTED { +if [[ "$(docker ps --format '{{.Names}}' | grep flink-test-kinesis)" = "flink-test-kinesis" ]]; then +return 0 +else +return 1 +fi +} + +START_KINESALITE_MAX_RETRY_NUMBER=10 +retry_until_passed_and_wait_condition START_KINESALITE KINESALITE_STARTED "start kinesalite" ${START_KINESALITE_MAX_RETRY_NUMBER} Review comment: I have not looped it on travis, only tested it by turning off and on connection and seeing that it downloads docker image eventually. I can try looping it on Travis but not sure we can do much more against the download hiccups. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests] Harden test_streaming_kinesis with kinesalite docker image download/run retries
azagrebin commented on a change in pull request #9419: [FLINK-13599][e2e tests] Harden test_streaming_kinesis with kinesalite docker image download/run retries URL: https://github.com/apache/flink/pull/9419#discussion_r313752529 ## File path: flink-end-to-end-tests/test-scripts/test_streaming_kinesis.sh ## @@ -28,9 +28,26 @@ export AWS_SECRET_KEY=flinkKinesisTestFakeAccessKey KINESALITE_PORT=4567 -#docker run -d --rm --name flink-test-kinesis -p ${KINESALITE_PORT}:${KINESALITE_PORT} instructure/kinesalite -# override entrypoint to enable SSL -docker run -d --rm --entrypoint "/tini" --name flink-test-kinesis -p ${KINESALITE_PORT}:${KINESALITE_PORT} instructure/kinesalite -- /usr/src/app/node_modules/kinesalite/cli.js --path /var/lib/kinesalite --ssl +function START_KINESALITE { Review comment: true, leftover This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services