See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7655/display/redirect?page=changes>
Changes: [baetensmatthias] [BEAM-11546] Cross-language pipeline pattern [Boyuan Zhang] Decrease the memorized backlog from 5s to 1s [heejong] [BEAM-12141] Print sha256 and size when downloading artifacts via ------------------------------------------ [...truncated 112.98 KB...] WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter. WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter. > Task :sdks:python:test-suites:dataflow:py37:validatesContainer WARNING:apache_beam.utils.retry:Retry with exponential backoff: waiting for 2.6775114833937734 seconds before retrying _read_with_retry because we caught exception: OSError: No such file or directory: gs://temp-storage-for-end-to-end-tests/output/1617971240421/results*-of-* Traceback for above exception (most recent call last): File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 260, in wrapper return fun(*args, **kwargs) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/testing/pipeline_verifiers.py";,> line 122, in _read_with_retry raise IOError('No such file or directory: %s' % self.file_path) WARNING:apache_beam.utils.retry:Retry with exponential backoff: waiting for 8.263287258180645 seconds before retrying _read_with_retry because we caught exception: OSError: No such file or directory: gs://temp-storage-for-end-to-end-tests/output/1617971240421/results*-of-* Traceback for above exception (most recent call last): File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 260, in wrapper return fun(*args, **kwargs) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/testing/pipeline_verifiers.py";,> line 122, in _read_with_retry raise IOError('No such file or directory: %s' % self.file_path) WARNING:apache_beam.utils.retry:Retry with exponential backoff: waiting for 10.685118557145499 seconds before retrying _read_with_retry because we caught exception: OSError: No such file or directory: gs://temp-storage-for-end-to-end-tests/output/1617971240421/results*-of-* Traceback for above exception (most recent call last): File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 260, in wrapper return fun(*args, **kwargs) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/testing/pipeline_verifiers.py";,> line 122, in _read_with_retry raise IOError('No such file or directory: %s' % self.file_path) WARNING:apache_beam.utils.retry:Retry with exponential backoff: waiting for 39.97480221971276 seconds before retrying _read_with_retry because we caught exception: OSError: No such file or directory: gs://temp-storage-for-end-to-end-tests/output/1617971240421/results*-of-* Traceback for above exception (most recent call last): File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 260, in wrapper return fun(*args, **kwargs) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/testing/pipeline_verifiers.py";,> line 122, in _read_with_retry raise IOError('No such file or directory: %s' % self.file_path) > Task :sdks:python:test-suites:dataflow:py36:validatesContainer WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter. > Task :sdks:python:test-suites:dataflow:py37:validatesContainer WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter. > Task :sdks:python:test-suites:dataflow:py38:validatesContainer WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter. > Task :sdks:python:test-suites:dataflow:py36:validatesContainer WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter. > Task :sdks:python:test-suites:dataflow:py37:validatesContainer WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter. > Task :sdks:python:test-suites:dataflow:py38:validatesContainer WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter. > Task :sdks:python:test-suites:dataflow:py36:validatesContainer WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter. > Task :sdks:python:test-suites:dataflow:py37:validatesContainer WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter. > Task :sdks:python:test-suites:dataflow:py38:validatesContainer WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter. > Task :sdks:python:test-suites:dataflow:py37:validatesContainer test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR test_wordcount_it_with_prebuilt_sdk_container_cloud_build (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok test_wordcount_it_with_prebuilt_sdk_container_local_docker (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok ====================================================================== ERROR: test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py";,> line 55, in test_wordcount_fnapi_it self._run_wordcount_it(wordcount.run, experiment='beam_fn_api') File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py";,> line 112, in _run_wordcount_it save_main_session=False, File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py";,> line 89, in run output | 'Write' >> WriteToText(known_args.output) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,> line 576, in __exit__ self.result = self.run() File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,> line 531, in run self._options).run(False) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,> line 555, in run return self.runner.run_pipeline(self, self._options) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py";,> line 72, in run_pipeline hc_assert_that(self.result, pickler.loads(on_success_matcher)) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/hamcrest/core/assert_that.py";,> line 44, in assert_that _assert_match(actual=arg1, matcher=arg2, reason=arg3) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/hamcrest/core/assert_that.py";,> line 52, in _assert_match if not matcher.matches(actual): File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/hamcrest/core/core/allof.py";,> line 19, in matches if not matcher.matches(item): File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/hamcrest/core/base_matcher.py";,> line 28, in matches match_result = self._matches(item) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/testing/pipeline_verifiers.py";,> line 142, in _matches read_lines = self._read_with_retry() File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 273, in wrapper raise_with_traceback(exn, exn_traceback) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/future/utils/__init__.py";,> line 446, in raise_with_traceback raise exc.with_traceback(traceback) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 260, in wrapper return fun(*args, **kwargs) File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/testing/pipeline_verifiers.py";,> line 122, in _read_with_retry raise IOError('No such file or directory: %s' % self.file_path) OSError: No such file or directory: gs://temp-storage-for-end-to-end-tests/output/1617971240421/results*-of-* -------------------- >> begin captured stdout << --------------------- Worker logs: https://console.cloud.google.com/dataflow/jobs/us-central1/2021-04-09_05_27_22-14534535811886893161?project=apache-beam-testing --------------------- >> end captured stdout << ---------------------- ---------------------------------------------------------------------- XML: nosetests-beam_python3.7_sdk.xml ---------------------------------------------------------------------- XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 4 tests in 1514.777s FAILED (errors=1) cleanup_container docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:9b07920cd0cb79fa47f11a4aceb0b69549c9c5247371e740c850b5826d966eaa Associated tags: - 20210409-122355009771261 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210409-122355009771261 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210409-122355009771261]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:9b07920cd0cb79fa47f11a4aceb0b69549c9c5247371e740c850b5826d966eaa]. gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)" Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:90c6b52c6860a0a93550662a1cbbc86e1502867da0b6f3b5166c73cdb97ecab6 Associated tags: - 54f1b33d-da96-48b8-8d0f-485c1828e07f Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:54f1b33d-da96-48b8-8d0f-485c1828e07f]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:90c6b52c6860a0a93550662a1cbbc86e1502867da0b6f3b5166c73cdb97ecab6]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:c00ee8e7c3763f7080113f0028a2f7f40278079e2e3d7846eee579c75d0ffc31 Associated tags: - 118b8ae1-661a-4c63-a7ef-5fdd160a2dd6 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:118b8ae1-661a-4c63-a7ef-5fdd160a2dd6]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:c00ee8e7c3763f7080113f0028a2f7f40278079e2e3d7846eee579c75d0ffc31]. > Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED > Task :sdks:python:test-suites:dataflow:py36:validatesContainer test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok test_wordcount_it_with_prebuilt_sdk_container_cloud_build (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok test_wordcount_it_with_prebuilt_sdk_container_local_docker (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok ---------------------------------------------------------------------- XML: nosetests-beam_python3.6_sdk.xml ---------------------------------------------------------------------- XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 4 tests in 1528.458s OK echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST" cleanup_container docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:1faf4082d80f3827f8d575e867fa6952a892e98031fb043a3e5ece837b95ea8d Associated tags: - 20210409-122354970748638 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210409-122354970748638 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210409-122354970748638]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:1faf4082d80f3827f8d575e867fa6952a892e98031fb043a3e5ece837b95ea8d]. gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)" Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:75de193f4199d0c57872ef9a2772ed2a11438e038f4d2ef3a974d2b96b415feb Associated tags: - 9a835574-3072-4ed7-bbb2-0d3ba601348a Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:9a835574-3072-4ed7-bbb2-0d3ba601348a]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:75de193f4199d0c57872ef9a2772ed2a11438e038f4d2ef3a974d2b96b415feb]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:091dcada56a55dec7da1df48c2ccf0ffb47f0ed862b14de1db5d09308e8e2196 Associated tags: - 0400817b-b583-4007-993b-79068668ff23 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:0400817b-b583-4007-993b-79068668ff23]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:091dcada56a55dec7da1df48c2ccf0ffb47f0ed862b14de1db5d09308e8e2196]. > Task :sdks:python:test-suites:dataflow:py38:validatesContainer test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok test_wordcount_it_with_prebuilt_sdk_container_cloud_build (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok test_wordcount_it_with_prebuilt_sdk_container_local_docker (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok ---------------------------------------------------------------------- XML: nosetests-beam_python3.8_sdk.xml ---------------------------------------------------------------------- XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 4 tests in 1507.614s OK echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST" cleanup_container docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:382705614154db23dae7c2f41391ef843e9af3185e2b7fe6b6040612b9581915 Associated tags: - 20210409-122612740160566 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210409-122612740160566 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210409-122612740160566]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:382705614154db23dae7c2f41391ef843e9af3185e2b7fe6b6040612b9581915]. gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)" Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:860fe1f5dde92294960665e44bbb646e06f8f400845aed703f77236c9bf8e0f0 Associated tags: - a7bf47cf-e909-462b-b514-b1f02cfdde95 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:a7bf47cf-e909-462b-b514-b1f02cfdde95]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:860fe1f5dde92294960665e44bbb646e06f8f400845aed703f77236c9bf8e0f0]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:d8379f977954da357ac09055fbd1f45ce6fc1078242909f6f02aa656e5e67b8a Associated tags: - a5c22f92-3691-4dfa-b46b-b79226b3a777 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:a5c22f92-3691-4dfa-b46b-b79226b3a777]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:d8379f977954da357ac09055fbd1f45ce6fc1078242909f6f02aa656e5e67b8a]. FAILURE: Build failed with an exception. * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 248 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 53m 19s 51 actionable tasks: 47 executed, 4 up-to-date Gradle was unable to watch the file system for changes. The inotify watches limit is too low. Publishing build scan... https://gradle.com/s/lli3yxth5tf7i Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
