See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/110/display/redirect?page=changes>
Changes: [noreply] Capture build scans on ge.apache.org to benefit from deep build insights ------------------------------------------ [...truncated 2.50 MB...] #24 2.872 Successfully installed pip-licenses-4.3.1 prettytable-3.7.0 wcwidth-0.2.6 #24 2.873 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv #24 10.11 Skip pulling license for bs4 #24 10.11 Traceback (most recent call last): #24 10.11 File "/tmp/license_scripts/pull_licenses_py.py", line 190, in <module> #24 10.11 raise RuntimeError( #24 10.11 RuntimeError: Could not retrieve licences for packages ['keras'] in Python3.9 environment. #24 10.11 These licenses were not able to be pulled automatically. Please search code source of the dependencies on the internet and add urls to RAW license file at sdks/python/container/license_scripts/dep_urls_py.yaml for each missing license and rerun the test. If no such urls can be found, you need to manually add LICENSE and NOTICE (if available) files at sdks/python/container/license_scripts/manual_licenses/{dep}/ and add entries to sdks/python/container/license_scripts/dep_urls_py.yaml. #24 ERROR: executor failed running [/bin/sh -c if [ "$pull_licenses" = "true" ] ; then pip install 'pip-licenses<5' pyyaml tenacity && python /tmp/license_scripts/pull_licenses_py.py ; fi]: exit code: 1 ------ > [third_party_licenses 4/4] RUN if [ "true" = "true" ] ; then pip > install 'pip-licenses<5' pyyaml tenacity && python > /tmp/license_scripts/pull_licenses_py.py ; fi: #24 1.375 Downloading wcwidth-0.2.6-py2.py3-none-any.whl (29 kB) #24 2.697 Installing collected packages: wcwidth, prettytable, pip-licenses #24 2.872 Successfully installed pip-licenses-4.3.1 prettytable-3.7.0 wcwidth-0.2.6 #24 2.873 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv #24 10.11 Skip pulling license for bs4 #24 10.11 Traceback (most recent call last): #24 10.11 File "/tmp/license_scripts/pull_licenses_py.py", line 190, in <module> #24 10.11 raise RuntimeError( #24 10.11 RuntimeError: Could not retrieve licences for packages ['keras'] in Python3.9 environment. #24 10.11 These licenses were not able to be pulled automatically. Please search code source of the dependencies on the internet and add urls to RAW license file at sdks/python/container/license_scripts/dep_urls_py.yaml for each missing license and rerun the test. If no such urls can be found, you need to manually add LICENSE and NOTICE (if available) files at sdks/python/container/license_scripts/manual_licenses/{dep}/ and add entries to sdks/python/container/license_scripts/dep_urls_py.yaml. ------ ERROR: failed to solve: executor failed running [/bin/sh -c if [ "$pull_licenses" = "true" ] ; then pip install 'pip-licenses<5' pyyaml tenacity && python /tmp/license_scripts/pull_licenses_py.py ; fi]: exit code: 1 > Task :sdks:python:test-suites:dataflow:py37:validatesContainer 52cb261b6574: Pushed 20230512-142839985652324: digest: sha256:6d76c84fcb6c89ff4c0ba3e2fb28d166805d190881d8c75900836c4e8d6836e8 size: 5563 function cleanup_container { # Delete the container locally and remotely docker rmi $CONTAINER:$TAG || echo "Failed to remove container image" for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH) do docker rmi $image || echo "Failed to remove prebuilt sdk container image" done gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container" for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)") do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image" done echo "Removed the container" } trap cleanup_container EXIT echo ">>> Successfully built and push container $CONTAINER" >>> Successfully built and push container >>> us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk >>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST cd sdks/python SDK_LOCATION=$2 # Run ValidatesRunner tests on Google Cloud Dataflow service echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST" pytest -o junit_suite_name=$IMAGE_NAME \ -m="it_validatescontainer" \ --show-capture=no \ --numprocesses=1 \ --timeout=1800 \ --junitxml=$XUNIT_FILE \ --ignore-glob '.*py3\d?\.py$' \ --log-cli-level=INFO \ --test-pipeline-options=" \ --runner=TestDataflowRunner \ --project=$PROJECT \ --region=$REGION \ --sdk_container_image=$CONTAINER:$TAG \ --staging_location=$GCS_LOCATION/staging-validatesrunner-test \ --temp_location=$GCS_LOCATION/temp-validatesrunner-test \ --output=$GCS_LOCATION/output \ --sdk_location=$SDK_LOCATION \ --num_workers=1 \ --docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH" ============================= test session starts ============================== platform linux -- Python 3.7.12, pytest-7.3.1, pluggy-1.0.0 rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/sdks/python> configfile: pytest.ini plugins: xdist-3.2.1, hypothesis-6.75.2, timeout-2.1.0, requests-mock-1.10.0 timeout: 1800.0s timeout method: signal timeout func_only: False gw0 I [gw0] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0] gw0 [5] scheduling tests via LoadScheduling apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_fnapi_it [gw0] [ 20%] PASSED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_fnapi_it apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build [gw0] [ 40%] PASSED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker [gw0] [ 60%] PASSED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_use_sibling_sdk_workers [gw0] [ 80%] PASSED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_use_sibling_sdk_workers apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it [gw0] [100%] SKIPPED apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it =============================== warnings summary =============================== ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py>:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses from imp import load_source ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:121 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:121 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:121: DeprecationWarning: pkg_resources is deprecated as an API warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870: 36 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870: 26 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.cloud')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2349: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(parent) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.logging')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.iam')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/rpc/__init__.py:20 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/rpc/__init__.py:20 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/google/rpc/__init__.py>:20: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.rpc')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages pkg_resources.declare_namespace(__name__) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/sdks/python/pytest-beam_python3.7_sdk.xml> - =========== 4 passed, 10 skipped, 80 warnings in 1721.31s (0:28:41) ============ >>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST" cleanup_container Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20230512-142839985652324 Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:6d76c84fcb6c89ff4c0ba3e2fb28d166805d190881d8c75900836c4e8d6836e8 Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:88a5ad19-f523-462c-a7ce-4a1e26311792 Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:1dc9be72d923975d57a365a96043d4e5fe38a8afa9aee6f85f2cbcd9e556f83d Deleted: sha256:afa0b51b7d191d286bc2c0e4e22f31bd1aa0f451291b0e5d202c3af3797b6f94 Deleted: sha256:0ec552361dad223e6ea6522399aa3299bc9f0b9af3250f20227b5460305004f4 Deleted: sha256:39a7657caaa15b07d64e2074deb29628ef60ded9607203b6adc2eae3b69825ed Deleted: sha256:19539d480191d1014e8c5a6e4574ae47b31563312d3b293a3fc44a199af01113 Deleted: sha256:e43499524f5ae3c9eec94b153a51ebddbc6b32259d48f31590a47726104fc871 Deleted: sha256:f6d1b44648ab9305e4372501c45f29ef24448e6298294da82ede4ae62495c4c1 WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly. Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:6d76c84fcb6c89ff4c0ba3e2fb28d166805d190881d8c75900836c4e8d6836e8 Associated tags: - 20230512-142839985652324 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20230512-142839985652324 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20230512-142839985652324]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:6d76c84fcb6c89ff4c0ba3e2fb28d166805d190881d8c75900836c4e8d6836e8]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:1dc9be72d923975d57a365a96043d4e5fe38a8afa9aee6f85f2cbcd9e556f83d Associated tags: - 88a5ad19-f523-462c-a7ce-4a1e26311792 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:88a5ad19-f523-462c-a7ce-4a1e26311792]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:1dc9be72d923975d57a365a96043d4e5fe38a8afa9aee6f85f2cbcd9e556f83d]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:b0a7d7452fc3c7c3e29448cef4aecfaefa06c99328905ee6a04ab246592c847f Associated tags: - a8325582-f570-4265-9cb8-1fb9857cee32 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:a8325582-f570-4265-9cb8-1fb9857cee32]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:b0a7d7452fc3c7c3e29448cef4aecfaefa06c99328905ee6a04ab246592c847f]. Removed the container FAILURE: Build completed with 4 failures. 1: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':sdks:python:container:py310:docker'. > Process 'command 'docker'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== 2: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':sdks:python:container:py311:docker'. > Process 'command 'docker'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== 3: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':sdks:python:container:py38:docker'. > Process 'command 'docker'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== 4: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':sdks:python:container:py39:docker'. > Process 'command 'docker'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 45m 2s 74 actionable tasks: 68 executed, 6 up-to-date Publishing build scan... https://ge.apache.org/s/3weu2yvkcgecm Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
