See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/10402/display/redirect?page=changes>
Changes:
[noreply] Disable Spark UI for load tests (#25792)
------------------------------------------
[...truncated 102.17 KB...]
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer
function cleanup_container {
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container image"
for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH)
do docker rmi $image || echo "Failed to remove prebuilt sdk container image"
done
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to
delete container"
for digest in $(gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)")
do gcloud container images delete
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest
--force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container
image"
done
echo "Removed the container"
}
trap cleanup_container EXIT
echo ">>> Successfully built and push container $CONTAINER"
cd sdks/python
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=1800 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST"
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:44070afc174c31b35edf6b9f7489e72ea8abae0565b64694673f0f79d5379f9e
Associated tags:
- 20230310-165554028152521
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20230310-165554028152521
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20230310-165554028152521].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:44070afc174c31b35edf6b9f7489e72ea8abae0565b64694673f0f79d5379f9e].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:b64eb3c6bdd4f5e682e9209168fc8a7808531718feebaae734aede06b022e1ea
Associated tags:
- fca25ef2-e227-4745-a810-a1150e3ab9f1
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:fca25ef2-e227-4745-a810-a1150e3ab9f1].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:b64eb3c6bdd4f5e682e9209168fc8a7808531718feebaae734aede06b022e1ea].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:3204fe5724c0159eeb951f8813638579374941defffba734d797525b97410a35
Associated tags:
- b57fbf95-57c4-417a-8665-49b5bf3aefe9
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:b57fbf95-57c4-417a-8665-49b5bf3aefe9].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:3204fe5724c0159eeb951f8813638579374941defffba734d797525b97410a35].
> Task :sdks:python:test-suites:dataflow:py310:validatesContainer
[gw0] [ 80%] PASSED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_use_sibling_sdk_workers
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
[gw0] [100%] SKIPPED
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
=============================== warnings summary ===============================
../../build/gradleenv/2050596098/lib/python3.10/site-packages/hdfs/config.py:15
../../build/gradleenv/2050596098/lib/python3.10/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib and
slated for removal in Python 3.12; see the module's documentation for
alternative uses
from imp import load_source
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:121
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:121
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py>:121:
DeprecationWarning: pkg_resources is deprecated as an API
warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2870:
38 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2870:
26 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2349
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py>:2349:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2870
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2870
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2870
../../build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py:2870
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.iam')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596098/lib/python3.10/site-packages/google/rpc/__init__.py:20
../../build/gradleenv/2050596098/lib/python3.10/site-packages/google/rpc/__init__.py:20
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/google/rpc/__init__.py>:20:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
pkg_resources.declare_namespace(__name__)
../../build/gradleenv/2050596098/lib/python3.10/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/google/api_core/operations_v1/abstract_operations_client.py>:17:
DeprecationWarning: The distutils package is deprecated and slated for removal
in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils import util
../../build/gradleenv/2050596098/lib/python3.10/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/tenacity/_asyncio.py>:42:
DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use
"async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/typehints/pandas_type_compatibility_test.py:67
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:67:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
}).set_index(pd.Int64Index(range(123, 223), name='an_index')),
apache_beam/typehints/pandas_type_compatibility_test.py:90
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:90:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(123, 223), name='an_index'),
apache_beam/typehints/pandas_type_compatibility_test.py:91
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:91:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(475, 575), name='another_index'),
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.10_sdk.xml>
-
============ 4 passed, 9 skipped, 87 warnings in 2155.02s (0:35:55) ============
>>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST
echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST"
cleanup_container
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20230310-165553933909593
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:9d5b4d3336d0e8db8d078f836e6b294b53980950cebc610d82c5d7f9fc399059
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:8b1126ae-99f8-4be6-969c-26921b6a9be0
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:e404b56618ebc339c3175c229c7a76c579609295ffcfe277f128f45938020c00
Deleted: sha256:16f627a645dac50bfacf4c1c26016d37a4e69bacd3225938f94d4babc7116dbe
Deleted: sha256:adb7379784b2c48fc4c5ba1aa7e8b237844f8552096fe5e32996f07e3fa67c98
Deleted: sha256:9d2e57ac33d83a5f54c19d999ad6d96342c00871b249fef637e923f2cf537597
Deleted: sha256:c29ac5f4cc12a7d74145782b57f2fa65f8b1df0b6ea1616e132890352f733775
Deleted: sha256:07f3bbf8a3d40017a559c212ec69f9172954f5c759a1133c9a4fb8d23931d9fd
Deleted: sha256:d1fd6bce6d5281bdf2bc892f1f03ce4fbe33f8a40b2810465490bff0f7e736bf
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:9d5b4d3336d0e8db8d078f836e6b294b53980950cebc610d82c5d7f9fc399059
Associated tags:
- 20230310-165553933909593
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20230310-165553933909593
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20230310-165553933909593].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:9d5b4d3336d0e8db8d078f836e6b294b53980950cebc610d82c5d7f9fc399059].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:e404b56618ebc339c3175c229c7a76c579609295ffcfe277f128f45938020c00
Associated tags:
- 8b1126ae-99f8-4be6-969c-26921b6a9be0
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:8b1126ae-99f8-4be6-969c-26921b6a9be0].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:e404b56618ebc339c3175c229c7a76c579609295ffcfe277f128f45938020c00].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:9940c7de04ad926a33360faea34c656a3a669f3ce68ee7c43d683b8b5ef26fa9
Associated tags:
- 734a0975-e7f0-4ade-9c79-47d7a0862425
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:734a0975-e7f0-4ade-9c79-47d7a0862425].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:9940c7de04ad926a33360faea34c656a3a669f3ce68ee7c43d683b8b5ef26fa9].
Removed the container
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer
echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST"
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:162368649cdb52306f167cbbfffcde780f37840999559378588d3cd01db26fc5
Associated tags:
- 20230310-165554423291823
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20230310-165554423291823
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20230310-165554423291823].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:162368649cdb52306f167cbbfffcde780f37840999559378588d3cd01db26fc5].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:53fc9821bdf042d29918fde03a0f9887fbacaa0feb39ace214539aac6820df67
Associated tags:
- 9e0652bd-592a-4ffd-90c9-d1330211f07d
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:9e0652bd-592a-4ffd-90c9-d1330211f07d].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:53fc9821bdf042d29918fde03a0f9887fbacaa0feb39ace214539aac6820df67].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:2cc912e2171b06528a97dee3574e5e1002c7c5cd958cef0c54fc3b3e4f2a80e7
Associated tags:
- 4cf80339-b567-49c5-9f6d-b0d98fbe2415
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:4cf80339-b567-49c5-9f6d-b0d98fbe2415].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:2cc912e2171b06528a97dee3574e5e1002c7c5cd958cef0c54fc3b3e4f2a80e7].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:python:container:py37:docker'.
> Process 'command 'docker'' finished with non-zero exit value 2
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 52m 54s
61 actionable tasks: 50 executed, 4 from cache, 7 up-to-date
Publishing build scan...
https://gradle.com/s/a62c2r7m4spn2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
