See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/107/display/redirect>
Changes:
------------------------------------------
[...truncated 2.78 MB...]
#24 11.43 File "/tmp/license_scripts/pull_licenses_py.py", line 190, in
<module>
#24 11.43 raise RuntimeError(
#24 11.43 RuntimeError: Could not retrieve licences for packages ['keras'] in
Python3.9 environment.
#24 11.43 These licenses were not able to be pulled automatically. Please
search code source of the dependencies on the internet and add urls to RAW
license file at sdks/python/container/license_scripts/dep_urls_py.yaml for each
missing license and rerun the test. If no such urls can be found, you need to
manually add LICENSE and NOTICE (if available) files at
sdks/python/container/license_scripts/manual_licenses/{dep}/ and add entries to
sdks/python/container/license_scripts/dep_urls_py.yaml.
------
ERROR: failed to solve: executor failed running [/bin/sh -c if [
"$pull_licenses" = "true" ] ; then pip install 'pip-licenses<5' pyyaml
tenacity && python /tmp/license_scripts/pull_licenses_py.py ; fi]:
exit code: 1
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
68096861ac95: Pushed
d2eb76bdc3b4: Pushed
20230511-203124569016583: digest:
sha256:cd651983acdcf71eb253ef200eb9c43f8692752ed30dc380278bff97d8b1e9ce size:
5563
function cleanup_container {
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container image"
for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH)
do docker rmi $image || echo "Failed to remove prebuilt sdk container image"
done
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to
delete container"
for digest in $(gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)")
do gcloud container images delete
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest
--force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container
image"
done
echo "Removed the container"
}
trap cleanup_container EXIT
echo ">>> Successfully built and push container $CONTAINER"
>>> Successfully built and push container
>>> us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
cd sdks/python
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=1800 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
============================= test session starts ==============================
platform linux -- Python 3.7.12, pytest-7.3.1, pluggy-1.0.0
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/sdks/python>
configfile: pytest.ini
plugins: xdist-3.2.1, hypothesis-6.75.2, timeout-2.1.0, requests-mock-1.10.0
timeout: 1800.0s
timeout method: signal
timeout func_only: False
gw0 I
[gw0] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
gw0 [5]
scheduling tests via LoadScheduling
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_fnapi_it
[gw0] [ 20%] PASSED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_fnapi_it
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build
[gw0] [ 40%] PASSED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker
[gw0] [ 60%] PASSED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_use_sibling_sdk_workers
[gw0] [ 80%] PASSED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_use_sibling_sdk_workers
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
[gw0] [100%] SKIPPED
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
=============================== warnings summary ===============================
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib; see
the module's documentation for alternative uses
from imp import load_source
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:121
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:121
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:121:
DeprecationWarning: pkg_resources is deprecated as an API
warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870:
36 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870:
26 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2349:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.iam')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/rpc/__init__.py:20
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/rpc/__init__.py:20
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/google/rpc/__init__.py>:20:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
pkg_resources.declare_namespace(__name__)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/sdks/python/pytest-beam_python3.7_sdk.xml>
-
=========== 4 passed, 10 skipped, 80 warnings in 1690.32s (0:28:10) ============
echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST
cleanup_container
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20230511-203124569016583
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:cd651983acdcf71eb253ef200eb9c43f8692752ed30dc380278bff97d8b1e9ce
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:34310d4e-a201-45ec-83be-68e78bf41bf4
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:1843294c1034785ffe72bafdbfb78152c34a8385832886e39523212134d2c0b1
Deleted: sha256:690d64b9c25e257963453edaea8476369743acd126985375092aba878ae24733
Deleted: sha256:74215d0c50b7334d56e74d28e7c5a210f67ae62928b73bc21ee10c06dbf879b4
Deleted: sha256:cb87f80fd9d5ecafb6a04d5801895aa3b2043de5444c8e54c01ba263658ecce9
Deleted: sha256:52683e1c13467801f1ee7f177462bb0c6f60d8094bbfe1e3410339c399424c8f
Deleted: sha256:8798885d8df425127ab515594342734a63eb2713e3991f5723c4d91d1984b601
Deleted: sha256:698cf6970edd2c1314b80c0c4ee79b6610ad17c5d8da905ad319aecbde3d6b13
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:cd651983acdcf71eb253ef200eb9c43f8692752ed30dc380278bff97d8b1e9ce
Associated tags:
- 20230511-203124569016583
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20230511-203124569016583
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20230511-203124569016583].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:cd651983acdcf71eb253ef200eb9c43f8692752ed30dc380278bff97d8b1e9ce].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:1843294c1034785ffe72bafdbfb78152c34a8385832886e39523212134d2c0b1
Associated tags:
- 34310d4e-a201-45ec-83be-68e78bf41bf4
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:34310d4e-a201-45ec-83be-68e78bf41bf4].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:1843294c1034785ffe72bafdbfb78152c34a8385832886e39523212134d2c0b1].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:fd98c542fd0b8779325aee4f924e311826a79178b73833fd050b025ef121e495
Associated tags:
- 0972a129-ff27-43e3-b913-405ebf1a05b6
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:0972a129-ff27-43e3-b913-405ebf1a05b6].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:fd98c542fd0b8779325aee4f924e311826a79178b73833fd050b025ef121e495].
Removed the container
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py310:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py38:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py311:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py39:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 47m 25s
74 actionable tasks: 62 executed, 4 from cache, 8 up-to-date
Build scan background action failed.
java.lang.IllegalArgumentException:
com.gradle.enterprise.gradleplugin.internal.extension.a is not an interface
at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:590)
at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:557)
at java.lang.reflect.WeakCache$Factory.get(WeakCache.java:230)
at java.lang.reflect.WeakCache.get(WeakCache.java:127)
at java.lang.reflect.Proxy.getProxyClass0(Proxy.java:419)
at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:719)
at
com.gradle.ProxyFactory$ProxyingInvocationHandler.createLocalProxy(ProxyFactory.java:66)
at
com.gradle.ProxyFactory$ProxyingInvocationHandler.lambda$adaptActionArg$1(ProxyFactory.java:61)
at
com.gradle.enterprise.gradleplugin.internal.extension.b$3.run(SourceFile:100)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Publishing build scan...
https://ge.apache.org/s/cabbpnsawxbtq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
