See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8229/display/redirect?page=changes>
Changes: [ruwan.lambrichts] Clarify additional_bq_parameters argument [esert] Bump a throttling counter on BigQueryRead retries due to [noreply] [BEAM-5097] Increment counter for "small words" in go SDK example ------------------------------------------ [...truncated 98.71 KB...] registries, e.g. `docker pull gcr.io/project-id/my-image`. Add `--verbosity=error` to silence this warning: `gcloud docker --verbosity=error -- pull gcr.io/project-id/my-image`. See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker > Task :sdks:python:test-suites:dataflow:py37:validatesContainer function cleanup_container { # Delete the container locally and remotely docker rmi $CONTAINER:$TAG || echo "Failed to remove container image" for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH) do docker rmi $image || echo "Failed to remove prebuilt sdk container image" done gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container" for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)") do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image" done echo "Removed the container" } trap cleanup_container EXIT echo ">>> Successfully built and push container $CONTAINER" cd sdks/python SDK_LOCATION=$2 # Run ValidatesRunner tests on Google Cloud Dataflow service echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST" pytest -o junit_suite_name=$IMAGE_NAME \ -m="it_validatescontainer" \ --show-capture=no \ --numprocesses=1 \ --timeout=900 \ --junitxml=$XUNIT_FILE \ --ignore-glob '.*py3\d?\.py$' \ --log-cli-level=INFO \ --test-pipeline-options=" \ --runner=TestDataflowRunner \ --project=$PROJECT \ --region=$REGION \ --sdk_container_image=$CONTAINER:$TAG \ --staging_location=$GCS_LOCATION/staging-validatesrunner-test \ --temp_location=$GCS_LOCATION/temp-validatesrunner-test \ --output=$GCS_LOCATION/output \ --sdk_location=$SDK_LOCATION \ --num_workers=1 \ --prebuild_sdk_container_base_image=$CONTAINER:$TAG \ --docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH" > Task :sdks:python:test-suites:dataflow:py36:validatesContainer function cleanup_container { # Delete the container locally and remotely docker rmi $CONTAINER:$TAG || echo "Failed to remove container image" for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH) do docker rmi $image || echo "Failed to remove prebuilt sdk container image" done gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container" for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)") do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image" done echo "Removed the container" } trap cleanup_container EXIT echo ">>> Successfully built and push container $CONTAINER" cd sdks/python SDK_LOCATION=$2 # Run ValidatesRunner tests on Google Cloud Dataflow service echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST" pytest -o junit_suite_name=$IMAGE_NAME \ -m="it_validatescontainer" \ --show-capture=no \ --numprocesses=1 \ --timeout=900 \ --junitxml=$XUNIT_FILE \ --ignore-glob '.*py3\d?\.py$' \ --log-cli-level=INFO \ --test-pipeline-options=" \ --runner=TestDataflowRunner \ --project=$PROJECT \ --region=$REGION \ --sdk_container_image=$CONTAINER:$TAG \ --staging_location=$GCS_LOCATION/staging-validatesrunner-test \ --temp_location=$GCS_LOCATION/temp-validatesrunner-test \ --output=$GCS_LOCATION/output \ --sdk_location=$SDK_LOCATION \ --num_workers=1 \ --prebuild_sdk_container_base_image=$CONTAINER:$TAG \ --docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH" > Task :sdks:python:test-suites:dataflow:py38:validatesContainer function cleanup_container { # Delete the container locally and remotely docker rmi $CONTAINER:$TAG || echo "Failed to remove container image" for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH) do docker rmi $image || echo "Failed to remove prebuilt sdk container image" done gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container" for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)") do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image" done echo "Removed the container" } trap cleanup_container EXIT echo ">>> Successfully built and push container $CONTAINER" cd sdks/python SDK_LOCATION=$2 # Run ValidatesRunner tests on Google Cloud Dataflow service echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST" pytest -o junit_suite_name=$IMAGE_NAME \ -m="it_validatescontainer" \ --show-capture=no \ --numprocesses=1 \ --timeout=900 \ --junitxml=$XUNIT_FILE \ --ignore-glob '.*py3\d?\.py$' \ --log-cli-level=INFO \ --test-pipeline-options=" \ --runner=TestDataflowRunner \ --project=$PROJECT \ --region=$REGION \ --sdk_container_image=$CONTAINER:$TAG \ --staging_location=$GCS_LOCATION/staging-validatesrunner-test \ --temp_location=$GCS_LOCATION/temp-validatesrunner-test \ --output=$GCS_LOCATION/output \ --sdk_location=$SDK_LOCATION \ --num_workers=1 \ --prebuild_sdk_container_base_image=$CONTAINER:$TAG \ --docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH" > Task :sdks:python:test-suites:dataflow:py36:validatesContainer cleanup_container docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:2320e8e9f0841ae949b94af80fa259578788bb360c62e431e7927e020611f87f Associated tags: - 20210910-061236810367421 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210910-061236810367421 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210910-061236810367421]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:2320e8e9f0841ae949b94af80fa259578788bb360c62e431e7927e020611f87f]. gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)" Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:76a063e7232a16b67189f3c419ca9430ed8fadc83a9ec39a4ccd19b6a48a5383 Associated tags: - ad468e47-30ba-4aa7-99c0-88dcd6519db3 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:ad468e47-30ba-4aa7-99c0-88dcd6519db3]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:76a063e7232a16b67189f3c419ca9430ed8fadc83a9ec39a4ccd19b6a48a5383]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:a9913b39def78f66af16b63f5ae61e9b3ddd559faac0f7a5a298ee274a510384 Associated tags: - 546ef9a5-0512-4653-933c-1f4deed87b0f Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:546ef9a5-0512-4653-933c-1f4deed87b0f]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:a9913b39def78f66af16b63f5ae61e9b3ddd559faac0f7a5a298ee274a510384]. > Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED > Task :sdks:python:test-suites:dataflow:py38:validatesContainer echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST" cleanup_container docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:44df7184eb570ba50d3a7c8f9043115e113df96a51a01d1bee5dd89b35ff9b41 Associated tags: - 20210910-061316920218574 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210910-061316920218574 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210910-061316920218574]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:44df7184eb570ba50d3a7c8f9043115e113df96a51a01d1bee5dd89b35ff9b41]. gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)" Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:9c90315f91d8e00ce579e21919f89e9946b984badf5383ae83ec6338442ca5f0 Associated tags: - 01b81e28-a0a5-439e-9b38-661b15bb1992 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:01b81e28-a0a5-439e-9b38-661b15bb1992]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:9c90315f91d8e00ce579e21919f89e9946b984badf5383ae83ec6338442ca5f0]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:3cd08063c619c8e2412114a771f7a5aa7a444f2bb7963430457ebdbee1d26ac7 Associated tags: - ea4d40bf-617e-4ac6-ab8e-a044d78ee20e Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:ea4d40bf-617e-4ac6-ab8e-a044d78ee20e]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:3cd08063c619c8e2412114a771f7a5aa7a444f2bb7963430457ebdbee1d26ac7]. > Task :sdks:python:test-suites:dataflow:py37:validatesContainer echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST" cleanup_container docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:1363b56ccee9e22984dcbf64edb4dac05f8ddd789e20a97520b61eff75925ffa Associated tags: - 20210910-061235611095702 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210910-061235611095702 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210910-061235611095702]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:1363b56ccee9e22984dcbf64edb4dac05f8ddd789e20a97520b61eff75925ffa]. gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)" Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:5170a654dc3e499c54c16141626214ea162598e5ec6878880204825c7e9569a4 Associated tags: - e837bbc7-f4e0-4cd1-b692-93061db17624 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:e837bbc7-f4e0-4cd1-b692-93061db17624]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:5170a654dc3e499c54c16141626214ea162598e5ec6878880204825c7e9569a4]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:ca3f9210015266993cc395e21f3ee7c8d905123f1abc99a54937fe6c9410177a Associated tags: - 572c12bb-3dea-4fe1-92c3-e99b4395ec2e Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:572c12bb-3dea-4fe1-92c3-e99b4395ec2e]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:ca3f9210015266993cc395e21f3ee7c8d905123f1abc99a54937fe6c9410177a]. FAILURE: Build failed with an exception. * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 57m 58s 44 actionable tasks: 40 executed, 4 up-to-date Publishing build scan... https://gradle.com/s/trd2keg2mbtx2 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
