See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8226/display/redirect>
Changes: ------------------------------------------ [...truncated 108.66 KB...] > Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED > Task :sdks:python:test-suites:dataflow:py36:validatesContainer [gw0] [ 50%] FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker [gw0] [ 75%] FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it [gw0] [100%] PASSED apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it =================================== FAILURES =================================== ____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_cloud_build _____ [gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967053/bin/python3.6> self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_cloud_build> @pytest.mark.it_validatescontainer def test_wordcount_it_with_prebuilt_sdk_container_cloud_build(self): self._run_wordcount_it( wordcount.run, experiment='beam_fn_api', > prebuild_sdk_container_engine='cloud_build') apache_beam/examples/wordcount_it_test.py:67: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ apache_beam/examples/wordcount_it_test.py:110: in _run_wordcount_it save_main_session=False, apache_beam/examples/wordcount.py:89: in run output | 'Write' >> WriteToText(known_args.output) apache_beam/pipeline.py:586: in __exit__ self.result = self.run() apache_beam/pipeline.py:541: in run self._options).run(False) apache_beam/pipeline.py:565: in run return self.runner.run_pipeline(self, self._options) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f5111956518> pipeline = <apache_beam.pipeline.Pipeline object at 0x7f5110e82128> options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f5111956438> def run_pipeline(self, pipeline, options): """Execute test pipeline and verify test matcher""" test_options = options.view_as(TestOptions) on_success_matcher = test_options.on_success_matcher wait_duration = test_options.wait_until_finish_duration is_streaming = options.view_as(StandardOptions).streaming # [BEAM-1889] Do not send this to remote workers also, there is no need to # send this option to remote executors. test_options.on_success_matcher = None self.result = super(TestDataflowRunner, self).run_pipeline(pipeline, options) if self.result.has_job: # TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs # in some cases. print('Worker logs: %s' % self.build_console_url(options)) try: self.wait_until_in_state(PipelineState.RUNNING) if is_streaming and not wait_duration: _LOGGER.warning('Waiting indefinitely for streaming job.') self.result.wait_until_finish(duration=wait_duration) if on_success_matcher: from hamcrest import assert_that as hc_assert_that > hc_assert_that(self.result, pickler.loads(on_success_matcher)) E AssertionError: E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0) E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError ____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_local_docker ____ [gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967053/bin/python3.6> self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_local_docker> @pytest.mark.it_validatescontainer def test_wordcount_it_with_prebuilt_sdk_container_local_docker(self): self._run_wordcount_it( wordcount.run, experiment='beam_fn_api', > prebuild_sdk_container_engine='local_docker') apache_beam/examples/wordcount_it_test.py:60: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ apache_beam/examples/wordcount_it_test.py:110: in _run_wordcount_it save_main_session=False, apache_beam/examples/wordcount.py:89: in run output | 'Write' >> WriteToText(known_args.output) apache_beam/pipeline.py:586: in __exit__ self.result = self.run() apache_beam/pipeline.py:541: in run self._options).run(False) apache_beam/pipeline.py:565: in run return self.runner.run_pipeline(self, self._options) apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline self.result.wait_until_finish(duration=wait_duration) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <DataflowPipelineResult <Job clientRequestId: '20210909122844098117-7153' createTime: '2021-09-09T12:28:45.444913Z' ...021-09-09T12:28:45.444913Z' steps: [] tempFiles: [] type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7f51102e3940> duration = None def wait_until_finish(self, duration=None): if not self.is_in_terminal_state(): if not self.has_job: raise IOError('Failed to get the Dataflow job id.') thread = threading.Thread( target=DataflowRunner.poll_for_job_completion, args=(self._runner, self, duration)) # Mark the thread as a daemon thread so a keyboard interrupt on the main # thread will terminate everything. This is also the reason we will not # use thread.join() to wait for the polling thread. thread.daemon = True thread.start() while thread.is_alive(): time.sleep(5.0) # TODO: Merge the termination code in poll_for_job_completion and # is_in_terminal_state. terminated = self.is_in_terminal_state() assert duration or terminated, ( 'Job did not reach to a terminal state after waiting indefinitely.') if terminated and self.state != PipelineState.DONE: # TODO(BEAM-1290): Consider converting this to an error log based on # theresolution of the issue. raise DataflowRuntimeException( 'Dataflow pipeline failed. State: %s, Error:\n%s' % (self.state, getattr(self._runner, 'last_error_msg', None)), > self) E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error: E Workflow failed. apache_beam/runners/dataflow/dataflow_runner.py:1635: DataflowRuntimeException =============================== warnings summary =============================== apache_beam/io/filesystems_test.py:54 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string apache_beam/io/filesystems_test.py:62 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string -- Docs: https://docs.pytest.org/en/latest/warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.6_sdk.xml> - ============== 2 failed, 2 passed, 2 warnings in 1321.15 seconds =============== cleanup_container Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210909-121243179651936 Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:c5a1f3270bdb04906bd8406ff21a149653638009989864537ec9a226397cb49c docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:c2c71633-b70d-47df-a4e2-c6a0a59cbd76 Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:b011b37a49c197a5a1888da9cd7d0e62c2ae9726df20040c3e6c6d407698b997 Deleted: sha256:ecd2cf84a2f71d1325976e4567395086be536c91f9b3449b1e2b2ff96f66cf2e Deleted: sha256:44a1646098cfe6860478ec7ddb1cca2d0e5cb9364559ab0eee81b3c1c9153dd3 Deleted: sha256:05a25a72eb71b495b7a6f5fe25c6a927cd8c765b9c10abc21404a4ac24a11be0 Deleted: sha256:e73833309dfaf347fd530495c31064296c56bdb6ae6e48c0fa943ef72c9287bd Deleted: sha256:7a5d8bb7604653f38566be7d9718e44eb2e952419b2429a4eef3d9391e41d99f Deleted: sha256:9dac3d45b832a93cf4289950bb737d9255576e70d99f7dc648fec05b47e92ced Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:c5a1f3270bdb04906bd8406ff21a149653638009989864537ec9a226397cb49c Associated tags: - 20210909-121243179651936 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210909-121243179651936 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210909-121243179651936]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:c5a1f3270bdb04906bd8406ff21a149653638009989864537ec9a226397cb49c]. gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)" Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:b011b37a49c197a5a1888da9cd7d0e62c2ae9726df20040c3e6c6d407698b997 Associated tags: - c2c71633-b70d-47df-a4e2-c6a0a59cbd76 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:c2c71633-b70d-47df-a4e2-c6a0a59cbd76]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:b011b37a49c197a5a1888da9cd7d0e62c2ae9726df20040c3e6c6d407698b997]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:6a8cf78f2284ca8482be48e12aeabf47a2d22b8ed351b3af4c09add237cb4e56 Associated tags: - 0a52bdc8-e10f-4cdd-af56-37ba16ad37a4 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:0a52bdc8-e10f-4cdd-af56-37ba16ad37a4]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:6a8cf78f2284ca8482be48e12aeabf47a2d22b8ed351b3af4c09add237cb4e56]. Removed the container > Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED FAILURE: Build completed with 3 failures. 1: Task failed with an exception. ----------- * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 3: Task failed with an exception. ----------- * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 36m 29s 44 actionable tasks: 40 executed, 4 up-to-date Publishing build scan... https://gradle.com/s/6mwwuog3u4lrs Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
