See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/10031/display/redirect?page=changes>
Changes:
[noreply] pubsub: fix typo in grpc client factory
[Kenneth Knowles] Suppress keyfor warnings
[Kenneth Knowles] Suppress checker warnings that are confusing and difficult
[Kenneth Knowles] Add @Pure annotations to MongoDbIO autovalue fields
[Kenneth Knowles] Suppress checker in FnApiDoFnRunner due to crash
[Kenneth Knowles] Suppress checker framework in Dataflow
[Kenneth Knowles] Fix some nullness errors in Spark runner
[Kenneth Knowles] Upgrade checker framework to 3.27.0
[noreply] fix go lints (#24566)
[noreply] Samza runner support for non unique stateId across multiple ParDos
[noreply] Bump to Hadoop 3.3.4 for performance tests (#24550)
------------------------------------------
[...truncated 1.81 MB...]
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=1800 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
function cleanup_container {
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container image"
for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH)
do docker rmi $image || echo "Failed to remove prebuilt sdk container image"
done
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to
delete container"
for digest in $(gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)")
do gcloud container images delete
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest
--force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container
image"
done
echo "Removed the container"
}
trap cleanup_container EXIT
echo ">>> Successfully built and push container $CONTAINER"
cd sdks/python
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=1800 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:77c417f8db0de4273ee41010aba6d14260dcb6b804c9e64159ceb13308638d43
Associated tags:
- 20221207-230450821551537
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221207-230450821551537
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221207-230450821551537].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:77c417f8db0de4273ee41010aba6d14260dcb6b804c9e64159ceb13308638d43].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:1fdecbe26ff4cface287228c219b809260c37b1c83dc3bf2afaf6d23531ea8b0
Associated tags:
- 5728691a-c8a1-4a98-8501-3a2e699cfa47
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:5728691a-c8a1-4a98-8501-3a2e699cfa47].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:1fdecbe26ff4cface287228c219b809260c37b1c83dc3bf2afaf6d23531ea8b0].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:33cb557bb2efa93005c2464238797041ed4c9db98602ae9c3f743454a52dff88
Associated tags:
- 760df575-14d3-4708-8d56-94da666c6a5c
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:760df575-14d3-4708-8d56-94da666c6a5c].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:33cb557bb2efa93005c2464238797041ed4c9db98602ae9c3f743454a52dff88].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py310:validatesContainer
cleanup_container
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:7ee6c6b857d79b303fc6af4569858ea6b65ff6fa5d36a440dbb6a54d5442a9bd
Associated tags:
- 20221207-230450828644093
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221207-230450828644093
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221207-230450828644093].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:7ee6c6b857d79b303fc6af4569858ea6b65ff6fa5d36a440dbb6a54d5442a9bd].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:0ebec93ae0d9e6c78eac06e08441b07c1afe77743f858478d82c4362fe4b942c
Associated tags:
- 20221207-230453879975565
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20221207-230453879975565
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20221207-230453879975565].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:0ebec93ae0d9e6c78eac06e08441b07c1afe77743f858478d82c4362fe4b942c].
> Task :sdks:python:test-suites:dataflow:py310:validatesContainer FAILED
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:9ae9fe2ac871e46872e6aa66d7ff8da1a2ad16d07e749231f04f7ca5293cbd94
Associated tags:
- 20221207-230445917088277
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221207-230445917088277
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221207-230445917088277].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:9ae9fe2ac871e46872e6aa66d7ff8da1a2ad16d07e749231f04f7ca5293cbd94].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:df3f745fe7eeca277dc6edde577ffe44e5c1b102f2c946d0cc9ddc28c699cd24
Associated tags:
- 9bcb58a2-b48c-427d-97df-dd557281d118
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:9bcb58a2-b48c-427d-97df-dd557281d118].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:df3f745fe7eeca277dc6edde577ffe44e5c1b102f2c946d0cc9ddc28c699cd24].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:3bc5190dca956e6c6cdc90aee37c8982df1306a6ab53f5449479fab58b0cf4d9
Associated tags:
- ef949379-dbb0-4ea5-9e58-4e6ad6e25fae
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:ef949379-dbb0-4ea5-9e58-4e6ad6e25fae].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:3bc5190dca956e6c6cdc90aee37c8982df1306a6ab53f5449479fab58b0cf4d9].
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer FAILED
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:427c7401836d6bbcf940416c9048260265358e549c493798210b429f3c035291
Associated tags:
- 4d698ed7-75d8-4ebb-8d87-52920f88e1fe
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:4d698ed7-75d8-4ebb-8d87-52920f88e1fe].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:427c7401836d6bbcf940416c9048260265358e549c493798210b429f3c035291].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:1f1d1bb63ffb5ddfd5805f3041a31054365c245a2695ad2b8d61b0100d81d901
Associated tags:
- ecc35629-5bc9-47de-b019-a0999f5e9ccc
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:ecc35629-5bc9-47de-b019-a0999f5e9ccc].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:1f1d1bb63ffb5ddfd5805f3041a31054365c245a2695ad2b8d61b0100d81d901].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:cb9d256e258e0a9fc57f43d1747e9fb8e3be50e4296687517efb992427b9efd9
Associated tags:
- b658c8fa-44d8-4239-85c7-94e957b96632
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:b658c8fa-44d8-4239-85c7-94e957b96632].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:cb9d256e258e0a9fc57f43d1747e9fb8e3be50e4296687517efb992427b9efd9].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:b66e771a8158700f1239a4880918b638bd7136abb296ad0c9be0dcd8a9c64afe
Associated tags:
- b6e8c00f-bbee-4e9f-9e65-b1e6d22aca3e
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:b6e8c00f-bbee-4e9f-9e65-b1e6d22aca3e].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:b66e771a8158700f1239a4880918b638bd7136abb296ad0c9be0dcd8a9c64afe].
Removed the container
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 331
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 331
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py310:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 331
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py39:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 331
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 2m 59s
62 actionable tasks: 51 executed, 4 from cache, 7 up-to-date
Publishing build scan...
https://gradle.com/s/x2q4dr3w7e5cs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
