See <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/5747/display/redirect>
Changes: ------------------------------------------ [...truncated 99.14 KB...] Resolve mutations for :sdks:python:test-suites:dataflow:py37:initializeForDataflowJob (Thread[Execution **** Thread 5,5,main]) started. Resolve mutations for :sdks:python:test-suites:dataflow:py37:initializeForDataflowJob (Thread[Execution **** Thread 5,5,main]) completed. Took 0.0 secs. :sdks:python:test-suites:dataflow:py37:initializeForDataflowJob (Thread[Execution **** Thread 7,5,main]) started. > Task :sdks:python:test-suites:dataflow:py37:initializeForDataflowJob Skipping task ':sdks:python:test-suites:dataflow:py37:initializeForDataflowJob' as it has no actions. :sdks:python:test-suites:dataflow:py37:initializeForDataflowJob (Thread[Execution **** Thread 7,5,main]) completed. Took 0.0 secs. Resolve mutations for :sdks:python:test-suites:dataflow:py37:runPerformanceTest (Thread[Execution **** Thread 2,5,main]) started. Resolve mutations for :sdks:python:test-suites:dataflow:py37:runPerformanceTest (Thread[Execution **** Thread 2,5,main]) completed. Took 0.0 secs. :sdks:python:test-suites:dataflow:py37:runPerformanceTest (Thread[Execution **** Thread 5,5,main]) started. > Task :sdks:python:test-suites:dataflow:py37:runPerformanceTest Caching disabled for task ':sdks:python:test-suites:dataflow:py37:runPerformanceTest' because: Build cache is disabled Task ':sdks:python:test-suites:dataflow:py37:runPerformanceTest' is not up-to-date because: Task has not declared any outputs despite executing actions. Starting process 'command 'sh''. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/sdks/python> Command: sh -c . <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/build/gradleenv/-1734967052/bin/activate> && pytest -o junit_suite_name=runPerformanceTest-df-py37 apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it --test-pipeline-options="--project=apache-beam-testing --region=us-central1 --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --job_name=performance-tests-wordcount-python37-batch-1gb0518005605 --runner=TestDataflowRunner --publish_to_big_query=true --metrics_dataset=beam_performance --metrics_table=wordcount_py37_pkb_results --influx_measurement=wordcount_py37_results --influx_db_name=beam_test_metrics --influx_hostname=http://10.128.0.96:8086 --input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000* --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710 --num_****s=10 --autoscaling_algorithm=NONE --sdk_location=<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/sdks/python/build/apache-beam.tar.gz";> --junitxml=pytest-runPerformanceTest-df-py37.xml --timeout=1800 Successfully started process 'command 'sh'' ============================= test session starts ============================== platform linux -- Python 3.7.12, pytest-7.3.1, pluggy-1.0.0 rootdir: <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/sdks/python> configfile: pytest.ini plugins: xdist-3.3.0, hypothesis-6.75.3, timeout-2.1.0, requests-mock-1.10.0 timeout: 1800.0s timeout method: signal timeout func_only: False collected 1 item apache_beam/examples/wordcount_it_test.py F [100%] =================================== FAILURES =================================== ________________________ WordCountIT.test_wordcount_it _________________________ self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fdb381eb950> pipeline = <apache_beam.pipeline.Pipeline object at 0x7fdb6c98e250> options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fdb381d1bd0> def run_pipeline(self, pipeline, options): """Execute test pipeline and verify test matcher""" test_options = options.view_as(TestOptions) on_success_matcher = test_options.on_success_matcher wait_duration = test_options.wait_until_finish_duration is_streaming = options.view_as(StandardOptions).streaming # [BEAM-1889] Do not send this to remote ****s also, there is no need to # send this option to remote executors. test_options.on_success_matcher = None self.result = super().run_pipeline(pipeline, options) if self.result.has_job: # TODO(markflyhigh)(https://github.com/apache/beam/issues/18254): Use # print since Nose dosen't show logs in some cases. print('Worker logs: %s' % self.build_console_url(options)) _LOGGER.info('Console log: ') _LOGGER.info(self.build_console_url(options)) try: > self.wait_until_in_state(PipelineState.RUNNING) apache_beam/runners/dataflow/test_dataflow_runner.py:62: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fdb381eb950> expected_state = 'RUNNING', timeout = 600 def wait_until_in_state(self, expected_state, timeout=WAIT_IN_STATE_TIMEOUT): """Wait until Dataflow pipeline enters a certain state.""" consoleUrl = ( "Console URL: https://console.cloud.google.com/dataflow/"; f"<regionId>/{self.result.job_id()}?project=<projectId>") if not self.result.has_job: _LOGGER.error(consoleUrl) raise IOError('Failed to get the Dataflow job id.') start_time = time.time() while time.time() - start_time <= timeout: job_state = self.result.state if self.result.is_in_terminal_state() or job_state == expected_state: return job_state time.sleep(5) _LOGGER.error(consoleUrl) raise RuntimeError( 'Timeout after %d seconds while waiting for job %s ' 'enters expected state %s. Current state is %s.' % > (timeout, self.result.job_id(), expected_state, self.result.state)) E RuntimeError: Timeout after 600 seconds while waiting for job 2023-05-17_19_47_19-12205644304683824642 enters expected state RUNNING. Current state is PENDING. apache_beam/runners/dataflow/test_dataflow_runner.py:106: RuntimeError During handling of the above exception, another exception occurred: self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it> @pytest.mark.it_postcommit def test_wordcount_it(self): > self._run_wordcount_it(wordcount.run) apache_beam/examples/wordcount_it_test.py:49: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ apache_beam/examples/wordcount_it_test.py:155: in _run_wordcount_it save_main_session=False, apache_beam/examples/wordcount.py:106: in run output | 'Write' >> WriteToText(known_args.output) apache_beam/pipeline.py:600: in __exit__ self.result = self.run() apache_beam/pipeline.py:553: in run self._options).run(False) apache_beam/pipeline.py:577: in run return self.runner.run_pipeline(self, self._options) apache_beam/runners/dataflow/test_dataflow_runner.py:73: in run_pipeline self.result.cancel() apache_beam/runners/dataflow/dataflow_runner.py:1589: in cancel self.job_id(), 'JOB_STATE_CANCELLED'): apache_beam/utils/retry.py:275: in wrapper return fun(*args, **kwargs) apache_beam/runners/dataflow/internal/apiclient.py:950: in modify_job_state self._client.projects_locations_jobs.Update(request) apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py:877: in Update return self._RunMethod(config, request, global_params=global_params) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/base_api.py:731: in _RunMethod return self.ProcessHttpResponse(method_config, http_response, request) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/base_api.py:737: in ProcessHttpResponse self.__ProcessHttpResponse(method_config, http_response, request)) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <apache_beam.runners.dataflow.internal.clients.dataflow.dataflow_v1b3_client.DataflowV1b3.ProjectsLocationsJobsService object at 0x7fdb37baa4d0> method_config = <ApiMethodInfo relative_path: 'v1b3/projects/{projectId}/locations/{location}/jobs/{jobId}' method_id: 'dataflow.pro...DataflowProjectsLocationsJobsUpdateRequest' response_type_name: 'Job' request_field: 'job' supports_download: False> http_response = Response(info={'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 1...is.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2023-05-17_19_47_19-12205644304683824642?alt=json') request = <DataflowProjectsLocationsJobsUpdateRequest job: <Job requestedState: RequestedStateValueValuesEnum(JOB_STATE_CANCEL...les: []> jobId: '2023-05-17_19_47_19-12205644304683824642' location: 'us-central1' projectId: 'apache-beam-testing'> def __ProcessHttpResponse(self, method_config, http_response, request): """Process the given http response.""" if http_response.status_code not in (http_client.OK, http_client.CREATED, http_client.NO_CONTENT): raise exceptions.HttpError.FromResponse( > http_response, method_config=method_config, request=request) E apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2023-05-17_19_47_19-12205644304683824642?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 18 May 2023 02:57:28 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '284', '-content-encoding': 'gzip'}>, content <{ E "error": { E "code": 400, E "message": "(69334f6865462d20): Workflow modification failed. Causes: (871462ee33cd9a9f): Cannot perform operation 'cancel' on Job: 2023-05-17_19_47_19-12205644304683824642. Job has no backend assigned..", E "status": "FAILED_PRECONDITION" E } E } E > ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/base_api.py:604: HttpBadRequestError ----------------------------- Captured stdout call ----------------------------- Worker logs: https://console.cloud.google.com/dataflow/jobs/us-central1/2023-05-17_19_47_19-12205644304683824642?project=apache-beam-testing ------------------------------ Captured log call ------------------------------- ERROR apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:102 Console URL: https://console.cloud.google.com/dataflow/<regionId>/2023-05-17_19_47_19-12205644304683824642?project=<projectId> =============================== warnings summary =============================== ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15 <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py>:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses from imp import load_source ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:121 <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:121: DeprecationWarning: pkg_resources is deprecated as an API warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870: 18 warnings <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870: 13 warnings <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.cloud')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2349 <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2349: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(parent) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870 <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.logging')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py:2870 <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/pkg_resources/__init__.py>:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.iam')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967052/lib/python3.7/site-packages/google/rpc/__init__.py:20 <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/google/rpc/__init__.py>:20: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.rpc')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages pkg_resources.declare_namespace(__name__) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/sdks/python/pytest-runPerformanceTest-df-py37.xml> - =========================== short test summary info ============================ FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it - apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2023-05-17_19_47_19-12205644304683824642?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 18 May 2023 02:57:28 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '284', '-content-encoding': 'gzip'}>, content <{ "error": { "code": 400, "message": "(69334f6865462d20): Workflow modification failed. Causes: (871462ee33cd9a9f): Cannot perform operation 'cancel' on Job: 2023-05-17_19_47_19-12205644304683824642. Job has no backend assigned..", "status": "FAILED_PRECONDITION" } } > ================== 1 failed, 40 warnings in 612.41s (0:10:12) ================== > Task :sdks:python:test-suites:dataflow:py37:runPerformanceTest FAILED :sdks:python:test-suites:dataflow:py37:runPerformanceTest (Thread[Execution **** Thread 5,5,main]) completed. Took 10 mins 15.517 secs. FAILURE: Build failed with an exception. * Where: Script '<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 301 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py37:runPerformanceTest'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --debug option to get more log output. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 12m 4s 15 actionable tasks: 15 executed Publishing build scan... https://ge.apache.org/s/tf32mppymvhqw Stopped 1 **** daemon(s). Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
