See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/10117/display/redirect?page=changes>
Changes: [noreply] [Spark Dataset runner] Reuse SparkSession when testing using class rule. ------------------------------------------ [...truncated 1.81 MB...] --verbosity=error -- pull gcr.io/project-id/my-image`. See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker > Task :sdks:python:test-suites:dataflow:py39:validatesContainer name unknown: Failed to get blob sizes. > Task :sdks:python:test-suites:dataflow:py39:validatesContainer FAILED > Task :sdks:python:test-suites:dataflow:py310:validatesContainer function cleanup_container { # Delete the container locally and remotely docker rmi $CONTAINER:$TAG || echo "Failed to remove container image" for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH) do docker rmi $image || echo "Failed to remove prebuilt sdk container image" done gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container" for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)") do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image" done echo "Removed the container" } trap cleanup_container EXIT echo ">>> Successfully built and push container $CONTAINER" cd sdks/python SDK_LOCATION=$2 # Run ValidatesRunner tests on Google Cloud Dataflow service echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST" pytest -o junit_suite_name=$IMAGE_NAME \ -m="it_validatescontainer" \ --show-capture=no \ --numprocesses=1 \ --timeout=1800 \ --junitxml=$XUNIT_FILE \ --ignore-glob '.*py3\d?\.py$' \ --log-cli-level=INFO \ --test-pipeline-options=" \ --runner=TestDataflowRunner \ --project=$PROJECT \ --region=$REGION \ --sdk_container_image=$CONTAINER:$TAG \ --staging_location=$GCS_LOCATION/staging-validatesrunner-test \ --temp_location=$GCS_LOCATION/temp-validatesrunner-test \ --output=$GCS_LOCATION/output \ --sdk_location=$SDK_LOCATION \ --num_workers=1 \ --docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH" > Task :sdks:python:test-suites:dataflow:py38:validatesContainer function cleanup_container { # Delete the container locally and remotely docker rmi $CONTAINER:$TAG || echo "Failed to remove container image" for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH) do docker rmi $image || echo "Failed to remove prebuilt sdk container image" done gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container" for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)") do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image" done echo "Removed the container" } trap cleanup_container EXIT echo ">>> Successfully built and push container $CONTAINER" cd sdks/python SDK_LOCATION=$2 # Run ValidatesRunner tests on Google Cloud Dataflow service echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST" pytest -o junit_suite_name=$IMAGE_NAME \ -m="it_validatescontainer" \ --show-capture=no \ --numprocesses=1 \ --timeout=1800 \ --junitxml=$XUNIT_FILE \ --ignore-glob '.*py3\d?\.py$' \ --log-cli-level=INFO \ --test-pipeline-options=" \ --runner=TestDataflowRunner \ --project=$PROJECT \ --region=$REGION \ --sdk_container_image=$CONTAINER:$TAG \ --staging_location=$GCS_LOCATION/staging-validatesrunner-test \ --temp_location=$GCS_LOCATION/temp-validatesrunner-test \ --output=$GCS_LOCATION/output \ --sdk_location=$SDK_LOCATION \ --num_workers=1 \ --docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH" > Task :sdks:python:test-suites:dataflow:py37:validatesContainer function cleanup_container { # Delete the container locally and remotely docker rmi $CONTAINER:$TAG || echo "Failed to remove container image" for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH) do docker rmi $image || echo "Failed to remove prebuilt sdk container image" done gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container" for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)") do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image" done echo "Removed the container" } trap cleanup_container EXIT echo ">>> Successfully built and push container $CONTAINER" cd sdks/python SDK_LOCATION=$2 # Run ValidatesRunner tests on Google Cloud Dataflow service echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST" pytest -o junit_suite_name=$IMAGE_NAME \ -m="it_validatescontainer" \ --show-capture=no \ --numprocesses=1 \ --timeout=1800 \ --junitxml=$XUNIT_FILE \ --ignore-glob '.*py3\d?\.py$' \ --log-cli-level=INFO \ --test-pipeline-options=" \ --runner=TestDataflowRunner \ --project=$PROJECT \ --region=$REGION \ --sdk_container_image=$CONTAINER:$TAG \ --staging_location=$GCS_LOCATION/staging-validatesrunner-test \ --temp_location=$GCS_LOCATION/temp-validatesrunner-test \ --output=$GCS_LOCATION/output \ --sdk_location=$SDK_LOCATION \ --num_workers=1 \ --docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH" > Task :sdks:python:test-suites:dataflow:py310:validatesContainer echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST" cleanup_container WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly. Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:576a1720494363820fcf7a2cb1677f34e09953fb67d4a7d16a0cc4102331b4e4 Associated tags: - 20221229-105500849395077 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221229-105500849395077 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221229-105500849395077]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:576a1720494363820fcf7a2cb1677f34e09953fb67d4a7d16a0cc4102331b4e4]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:6c184ceb50a350944b974a06894e9416b2e92c49ce77b05df9b2353e677ec60e Associated tags: - 61aa8ab1-04fc-4e6f-b693-0581c05cc584 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:61aa8ab1-04fc-4e6f-b693-0581c05cc584]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:6c184ceb50a350944b974a06894e9416b2e92c49ce77b05df9b2353e677ec60e]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:67223c6913ae548a2388c0f1607199c8d996f81d8ce8ee1c751948882ac656b9 Associated tags: - 475b4214-abc5-4817-88d7-da0107e69aa5 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:475b4214-abc5-4817-88d7-da0107e69aa5]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:67223c6913ae548a2388c0f1607199c8d996f81d8ce8ee1c751948882ac656b9]. > Task :sdks:python:test-suites:dataflow:py37:validatesContainer echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST" cleanup_container WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly. Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:af876473a6f4acbefc49f2cc3c8b1c42b2194997b1288ef07292d0ccf0368bf4 Associated tags: - 20221229-105501034659887 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20221229-105501034659887 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20221229-105501034659887]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:af876473a6f4acbefc49f2cc3c8b1c42b2194997b1288ef07292d0ccf0368bf4]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:b41916ac3bc20127cbaca2d55f79f2efabb7d6935c24c86d7c947f47f59b6136 Associated tags: - e8432bad-a9e3-4e4c-a700-ad84700b6be8 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:e8432bad-a9e3-4e4c-a700-ad84700b6be8]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:b41916ac3bc20127cbaca2d55f79f2efabb7d6935c24c86d7c947f47f59b6136]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:88e1d9b001fb5b6a1ecc02ca8e031e659113a4d8340a49febf46d6bf85932b43 Associated tags: - 05589e4d-b875-477d-bf61-c7b477b891d2 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:05589e4d-b875-477d-bf61-c7b477b891d2]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:88e1d9b001fb5b6a1ecc02ca8e031e659113a4d8340a49febf46d6bf85932b43]. > Task :sdks:python:test-suites:dataflow:py38:validatesContainer echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST" cleanup_container WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly. Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:8f6be70b3e000dd229fb547dfbc6ced89e1e73ba3d5fc016dcf7a76196192bed Associated tags: - 20221229-105500946906042 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221229-105500946906042 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221229-105500946906042]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:8f6be70b3e000dd229fb547dfbc6ced89e1e73ba3d5fc016dcf7a76196192bed]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:25d3d4104d16bbc0bd3ade738a9bd72489ff450bd244ad44fa80f740eb1877bf Associated tags: - 8c0ef6bf-add5-4106-a169-dfdc3d420859 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:8c0ef6bf-add5-4106-a169-dfdc3d420859]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:25d3d4104d16bbc0bd3ade738a9bd72489ff450bd244ad44fa80f740eb1877bf]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:ed9553b5dab63ae168cde54fd4a916cf86ed75ece5cce62c8a37ef46a71e6aed Associated tags: - ba67bd13-6458-4801-ba80-bf5802d04250 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:ba67bd13-6458-4801-ba80-bf5802d04250]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:ed9553b5dab63ae168cde54fd4a916cf86ed75ece5cce62c8a37ef46a71e6aed]. FAILURE: Build failed with an exception. * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 331 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py39:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 56m 26s 62 actionable tasks: 51 executed, 4 from cache, 7 up-to-date Publishing build scan... https://gradle.com/s/652auoohdxwfe Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
