See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/112/display/redirect>
Changes:
------------------------------------------
[...truncated 2.78 MB...]
> Task :sdks:python:container:py39:docker FAILED
#24 9.267 Traceback (most recent call last):
#24 9.267 File "/tmp/license_scripts/pull_licenses_py.py", line 190, in
<module>
#24 9.267 Skip pulling license for bs4
#24 9.267 raise RuntimeError(
#24 9.267 RuntimeError: Could not retrieve licences for packages ['keras'] in
Python3.9 environment.
#24 9.267 These licenses were not able to be pulled automatically. Please
search code source of the dependencies on the internet and add urls to RAW
license file at sdks/python/container/license_scripts/dep_urls_py.yaml for each
missing license and rerun the test. If no such urls can be found, you need to
manually add LICENSE and NOTICE (if available) files at
sdks/python/container/license_scripts/manual_licenses/{dep}/ and add entries to
sdks/python/container/license_scripts/dep_urls_py.yaml.
#24 ERROR: executor failed running [/bin/sh -c if [ "$pull_licenses" = "true" ]
; then pip install 'pip-licenses<5' pyyaml tenacity && python
/tmp/license_scripts/pull_licenses_py.py ; fi]: exit code: 1
------
> [third_party_licenses 4/4] RUN if [ "true" = "true" ] ; then pip
> install 'pip-licenses<5' pyyaml tenacity && python
> /tmp/license_scripts/pull_licenses_py.py ; fi:
#24 1.359 Downloading wcwidth-0.2.6-py2.py3-none-any.whl (29 kB)
#24 2.514 Installing collected packages: wcwidth, prettytable, pip-licenses
#24 2.684 Successfully installed pip-licenses-4.3.1 prettytable-3.7.0
wcwidth-0.2.6
#24 2.685 WARNING: Running pip as the 'root' user can result in broken
permissions and conflicting behaviour with the system package manager. It is
recommended to use a virtual environment instead:
https://pip.pypa.io/warnings/venv
#24 9.267 Traceback (most recent call last):
#24 9.267 File "/tmp/license_scripts/pull_licenses_py.py", line 190, in
<module>
#24 9.267 Skip pulling license for bs4
#24 9.267 raise RuntimeError(
#24 9.267 RuntimeError: Could not retrieve licences for packages ['keras'] in
Python3.9 environment.
#24 9.267 These licenses were not able to be pulled automatically. Please
search code source of the dependencies on the internet and add urls to RAW
license file at sdks/python/container/license_scripts/dep_urls_py.yaml for each
missing license and rerun the test. If no such urls can be found, you need to
manually add LICENSE and NOTICE (if available) files at
sdks/python/container/license_scripts/manual_licenses/{dep}/ and add entries to
sdks/python/container/license_scripts/dep_urls_py.yaml.
------
ERROR: failed to solve: executor failed running [/bin/sh -c if [
"$pull_licenses" = "true" ] ; then pip install 'pip-licenses<5' pyyaml
tenacity && python /tmp/license_scripts/pull_licenses_py.py ; fi]:
exit code: 1
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
0d006173e332: Pushed
8ed60e3187c3: Pushed
20230513-022805711681501: digest:
sha256:46ab2b927ea5b1c6e622aad276099e054ab395f68ee8955edbac6cb1d7b9bd4c size:
5563
function cleanup_container {
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container image"
for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH)
do docker rmi $image || echo "Failed to remove prebuilt sdk container image"
done
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to
delete container"
for digest in $(gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)")
do gcloud container images delete
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest
--force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container
image"
done
echo "Removed the container"
}
trap cleanup_container EXIT
echo ">>> Successfully built and push container $CONTAINER"
cd sdks/python
>>> Successfully built and push container
>>> us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk
SDK_LOCATION=$2
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=1800 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
============================= test session starts ==============================
platform linux -- Python 3.7.12, pytest-7.3.1, pluggy-1.0.0
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/sdks/python>
configfile: pytest.ini
plugins: xdist-3.3.0, hypothesis-6.75.2, timeout-2.1.0, requests-mock-1.10.0
timeout: 1800.0s
timeout method: signal
timeout func_only: False
created: 1/1 worker
1 worker [5 items]
scheduling tests via LoadScheduling
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_fnapi_it
[gw0] [ 20%] PASSED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_fnapi_it
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build
[gw0] [ 40%] PASSED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker
[gw0] [ 60%] PASSED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_use_sibling_sdk_workers
[gw0] [ 80%] PASSED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_use_sibling_sdk_workers
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
[gw0] [100%] SKIPPED
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
=============================== warnings summary ===============================
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib; see
the module's documentation for alternative uses
from imp import load_source
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:121
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:121
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:121:
DeprecationWarning: pkg_resources is deprecated as an API
warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870:
36 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870:
26 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2349:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.iam')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/rpc/__init__.py:20
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/rpc/__init__.py:20
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/google/rpc/__init__.py>:20:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
pkg_resources.declare_namespace(__name__)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/sdks/python/pytest-beam_python3.7_sdk.xml>
-
=========== 4 passed, 10 skipped, 80 warnings in 1696.51s (0:28:16) ============
echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST"
cleanup_container
>>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20230513-022805711681501
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:46ab2b927ea5b1c6e622aad276099e054ab395f68ee8955edbac6cb1d7b9bd4c
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:aa4d381d-aa57-4346-b530-d6584172494a
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:588e761b0418f3b4c0aa8dbecbcac48f12f48988146c9c39eb2297f5ed6eec9e
Deleted: sha256:fe2d31a05422898c9c373009104c71af606e48d8336eb16f2257e88f9f57c0bb
Deleted: sha256:7d203ab472fd8e000d94d4a1b156f8063dac6c4bc9494a19aabcf3063f8dafc4
Deleted: sha256:3f37c2190d2846cda7db804158ead34e3feaa506f76088f86b66f0bbe97d0373
Deleted: sha256:49c498298d9dfe4139f07a2bd0ceaf4811b9da2be254971b716e621088f7c4c8
Deleted: sha256:785f4a6e238c7045f48f586c5f6d1c13edbb2b7844bbc6e48c54e2ed52e544d2
Deleted: sha256:7e8d7a5dca401a675384483a834105ca2098a78d5c1bf155a27a6269cbaf97c6
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:46ab2b927ea5b1c6e622aad276099e054ab395f68ee8955edbac6cb1d7b9bd4c
Associated tags:
- 20230513-022805711681501
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20230513-022805711681501
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20230513-022805711681501].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:46ab2b927ea5b1c6e622aad276099e054ab395f68ee8955edbac6cb1d7b9bd4c].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:588e761b0418f3b4c0aa8dbecbcac48f12f48988146c9c39eb2297f5ed6eec9e
Associated tags:
- aa4d381d-aa57-4346-b530-d6584172494a
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:aa4d381d-aa57-4346-b530-d6584172494a].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:588e761b0418f3b4c0aa8dbecbcac48f12f48988146c9c39eb2297f5ed6eec9e].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:3edfa57048ad18e997282191d93d83127e672104eb831454c0329e2596206395
Associated tags:
- 0a932ea7-b000-4755-ba58-f4cf5fa144f9
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:0a932ea7-b000-4755-ba58-f4cf5fa144f9].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:3edfa57048ad18e997282191d93d83127e672104eb831454c0329e2596206395].
Removed the container
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py310:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py311:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py38:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py39:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 44m 8s
74 actionable tasks: 68 executed, 6 up-to-date
Publishing build scan...
https://ge.apache.org/s/hym3quwdwrnmy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
