See <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/3327/display/redirect?page=changes>
Changes: [Heejong Lee] [BEAM-14251] add output_coder_override to ExpansionRequest [Heejong Lee] remove null [Heejong Lee] better error msg [Heejong Lee] update from comments [noreply] Merge pull request #17382: [BEAM-12356] Close DatasetService leak as ------------------------------------------ [...truncated 79.07 KB...] [gw1] [32mPASSED[0m apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_non_liftable_combine apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_return [gw2] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_with_different_defaults apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_without_unique_labels [gw0] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_and_as_dict_side_inputs apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_twice [gw3] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_impulse apache_beam/transforms/ptransform_test.py::PTransformTest::test_multiple_empty_outputs [gw7] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_one_single_pcollection apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_dict_twice [gw4] [32mPASSED[0m apache_beam/transforms/dofn_lifecycle_test.py::DoFnLifecycleTest::test_dofn_lifecycle apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_default_value_singleton_side_input [gw6] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_yield apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_empty_singleton_side_input [gw5] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_undeclared_outputs apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_flattened_side_input [gw2] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_without_unique_labels apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_reiterable_side_input [gw1] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_return apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_iterable_side_input [gw3] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_multiple_empty_outputs [gw0] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_twice apache_beam/transforms/util_test.py::ReshuffleTest::test_reshuffle_preserves_timestamps [gw4] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_default_value_singleton_side_input [gw7] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_dict_twice [gw2] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_reiterable_side_input [gw1] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_iterable_side_input [gw6] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_empty_singleton_side_input [gw5] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_flattened_side_input [gw0] [32mPASSED[0m apache_beam/transforms/util_test.py::ReshuffleTest::test_reshuffle_preserves_timestamps [33m=============================== warnings summary ===============================[0m <unknown>:54 <unknown>:54 <unknown>:54 <unknown>:54 <unknown>:54 <unknown>:54 <unknown>:54 <unknown>:54 <unknown>:54: DeprecationWarning: invalid escape sequence \c <unknown>:62 <unknown>:62 <unknown>:62 <unknown>:62 <unknown>:62 <unknown>:62 <unknown>:62 <unknown>:62 <unknown>:62: DeprecationWarning: invalid escape sequence \d <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead def call(self, fn, *args, **kwargs): apache_beam/runners/direct/transform_evaluator.py:1029 apache_beam/runners/direct/transform_evaluator.py:1029 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/apache_beam/runners/direct/transform_evaluator.py>:1029: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working isinstance(element.value, collections.Iterable) and -- Docs: https://docs.pytest.org/en/latest/warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py39-xdist.xml> - [33m[1m============= 32 passed, 1 skipped, 26 warnings in 2348.58 seconds =============[0m >>> RUNNING integration tests with pipeline options: >>> --runner=TestDataflowRunner --project=apache-beam-testing >>> --region=us-central1 >>> --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it >>> --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it >>> --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output >>> --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/build/apache-beam.tar.gz> >>> --requirements_file=postcommit_requirements.txt --num_workers=1 >>> --sleep_secs=20 --streaming >>> --dataflow_worker_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.39.0-SNAPSHOT.jar> >>> --experiments=use_runner_v2 --experiments=shuffle_mode=appliance >>> --enable_streaming_engine >>> pytest options: --capture=no --timeout=4500 --color=yes >>> --log-cli-level=INFO >>> collect markers: -m=it_validatesrunner and not no_sickbay_streaming and >>> no_xdist [1m============================= test session starts ==============================[0m platform linux -- Python 3.9.10, pytest-4.6.11, py-1.11.0, pluggy-0.13.1 rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python,> inifile: pytest.ini plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3 timeout: 4500.0s timeout method: signal timeout func_only: False [1m----------------------------- live log collection ------------------------------[0m [33mWARNING [0m root:avroio_test.py:51 python-snappy is not installed; some tests will be skipped. [33mWARNING [0m root:tfrecordio_test.py:55 Tensorflow is not installed, so skipping some tests. [33mWARNING [0m apache_beam.runners.interactive.interactive_environment:interactive_environment.py:190 Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features. [33mWARNING [0m apache_beam.runners.interactive.interactive_environment:interactive_environment.py:199 You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal. [33mWARNING [0m root:environments.py:371 Make sure that locally built Python SDK docker image has Python 3.9 interpreter. [32mINFO [0m root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.9_sdk:2.39.0.dev collected 5318 items / 5317 deselected / 1 selected apache_beam/runners/dataflow/dataflow_exercise_streaming_metrics_pipeline_test.py::ExerciseStreamingMetricsPipelineTest::test_streaming_pipeline_returns_expected_user_metrics_fnapi_it [31mFAILED[0m =================================== FAILURES =================================== [31m[1m_ ExerciseStreamingMetricsPipelineTest.test_streaming_pipeline_returns_expected_user_metrics_fnapi_it _[0m args = (name: "projects/apache-beam-testing/subscriptions/exercise_streaming_metrics_subscription_input93659236-df7d-46c6-bfb...ic: "projects/apache-beam-testing/topics/exercise_streaming_metrics_topic_input93659236-df7d-46c6-bfb8-1c0ae13bcfcb" ,) kwargs = {'metadata': [('x-goog-request-params', 'name=projects/apache-beam-testing/subscriptions/exercise_streaming_metrics_su...t93659236-df7d-46c6-bfb8-1c0ae13bcfcb'), ('x-goog-api-client', 'gl-python/3.9.10 grpc/1.44.0 gax/1.31.5 gccl/2.12.0')]} [1m @six.wraps(callable_)[0m [1m def error_remapped_callable(*args, **kwargs):[0m [1m try:[0m [1m> return callable_(*args, **kwargs)[0m [1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/google/api_core/grpc_helpers.py[0m:67: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <grpc._channel._UnaryUnaryMultiCallable object at 0x7ff4606361f0> request = name: "projects/apache-beam-testing/subscriptions/exercise_streaming_metrics_subscription_input93659236-df7d-46c6-bfb8...opic: "projects/apache-beam-testing/topics/exercise_streaming_metrics_topic_input93659236-df7d-46c6-bfb8-1c0ae13bcfcb" timeout = None metadata = [('x-goog-request-params', 'name=projects/apache-beam-testing/subscriptions/exercise_streaming_metrics_subscription_input93659236-df7d-46c6-bfb8-1c0ae13bcfcb'), ('x-goog-api-client', 'gl-python/3.9.10 grpc/1.44.0 gax/1.31.5 gccl/2.12.0')] credentials = None, wait_for_ready = None, compression = None [1m def __call__(self,[0m [1m request,[0m [1m timeout=None,[0m [1m metadata=None,[0m [1m credentials=None,[0m [1m wait_for_ready=None,[0m [1m compression=None):[0m [1m state, call, = self._blocking(request, timeout, metadata, credentials,[0m [1m wait_for_ready, compression)[0m [1m> return _end_unary_response_blocking(state, call, False, None)[0m [1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/grpc/_channel.py[0m:946: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ state = <grpc._channel._RPCState object at 0x7ff4609979d0> call = <grpc._cython.cygrpc.SegregatedCall object at 0x7ff460998c40> with_call = False, deadline = None [1m def _end_unary_response_blocking(state, call, with_call, deadline):[0m [1m if state.code is grpc.StatusCode.OK:[0m [1m if with_call:[0m [1m rendezvous = _MultiThreadedRendezvous(state, call, None, deadline)[0m [1m return state.response, rendezvous[0m [1m else:[0m [1m return state.response[0m [1m else:[0m [1m> raise _InactiveRpcError(state)[0m [1m[31mE grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:[0m [1m[31mE status = StatusCode.DEADLINE_EXCEEDED[0m [1m[31mE details = "Deadline expired before operation could complete."[0m [1m[31mE debug_error_string = "{"created":"@1650342702.539282645","description":"Error received from peer ipv4:74.125.124.95:443","file":"src/core/lib/surface/call.cc","file_line":903,"grpc_message":"Deadline expired before operation could complete.","grpc_status":4}"[0m [1m[31mE >[0m [1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/grpc/_channel.py[0m:849: _InactiveRpcError [33mThe above exception was the direct cause of the following exception:[0m self = <apache_beam.runners.dataflow.dataflow_exercise_streaming_metrics_pipeline_test.ExerciseStreamingMetricsPipelineTest testMethod=test_streaming_pipeline_returns_expected_user_metrics_fnapi_it> [1m def setUp(self):[0m [1m """Creates all required topics and subs."""[0m [1m self.test_pipeline = TestPipeline(is_integration_test=True)[0m [1m self.project = self.test_pipeline.get_option('project')[0m [1m self.uuid = str(uuid.uuid4())[0m [1m [0m [1m # Set up PubSub environment.[0m [1m from google.cloud import pubsub[0m [1m self.pub_client = pubsub.PublisherClient()[0m [1m self.input_topic_name = INPUT_TOPIC + self.uuid[0m [1m self.input_topic = self.pub_client.create_topic([0m [1m name=self.pub_client.topic_path(self.project, self.input_topic_name))[0m [1m [0m [1m self.output_topic_name = OUTPUT_TOPIC + self.uuid[0m [1m self.output_topic = self.pub_client.create_topic([0m [1m name=self.pub_client.topic_path(self.project, self.output_topic_name))[0m [1m [0m [1m self.sub_client = pubsub.SubscriberClient()[0m [1m self.input_sub_name = INPUT_SUB + self.uuid[0m [1m> self.input_sub = self.sub_client.create_subscription([0m [1m name=self.sub_client.subscription_path([0m [1m self.project, self.input_sub_name),[0m [1m topic=self.input_topic.name)[0m [1m[31mapache_beam/runners/dataflow/dataflow_exercise_streaming_metrics_pipeline_test.py[0m:81: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/google/pubsub_v1/services/subscriber/client.py[0m:656: in create_subscription [1m response = rpc([0m [1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/google/api_core/gapic_v1/method.py[0m:145: in __call__ [1m return wrapped_func(*args, **kwargs)[0m [1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/google/api_core/retry.py[0m:286: in retry_wrapped_func [1m return retry_target([0m [1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/google/api_core/retry.py[0m:189: in retry_target [1m return target()[0m [1m[31m../../build/gradleenv/-1734967050/lib/python3.9/site-packages/google/api_core/grpc_helpers.py[0m:69: in error_remapped_callable [1m six.raise_from(exceptions.from_grpc_error(exc), exc)[0m _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ value = None from_value = <_InactiveRpcError of RPC that terminated with: status = StatusCode.DEADLINE_EXCEEDED details = "Deadline expired be...surface/call.cc","file_line":903,"grpc_message":"Deadline expired before operation could complete.","grpc_status":4}" > [1m> ???[0m [1m[31mE google.api_core.exceptions.DeadlineExceeded: 504 Deadline expired before operation could complete.[0m [1m[31m<string>[0m:3: DeadlineExceeded [33m=============================== warnings summary ===============================[0m <unknown>:54 <unknown>:54: DeprecationWarning: invalid escape sequence \c <unknown>:62 <unknown>:62: DeprecationWarning: invalid escape sequence \d <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead def call(self, fn, *args, **kwargs): -- Docs: https://docs.pytest.org/en/latest/warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py39-noxdist.xml> - [31m[1m=========== 1 failed, 5317 deselected, 3 warnings in 140.22 seconds ============[0m > Task :sdks:python:test-suites:dataflow:py39:validatesRunnerStreamingTests > FAILED FAILURE: Build failed with an exception. * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 267 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py39:validatesRunnerStreamingTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 1h 23m 17s 93 actionable tasks: 61 executed, 30 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/ih3infllzzpgu Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
