See <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/4090/display/redirect?page=changes>
Changes: [Moritz Mack] Remove obsolete code from Spark 3 runner. [noreply] Fixing Get Started header link (#23490) [noreply] Bump cloud.google.com/go/bigquery from 1.42.0 to 1.43.0 in /sdks [noreply] Bump google.golang.org/api from 0.100.0 to 0.101.0 in /sdks (#23842) [noreply] Update ReadDataFromKinesis URN to registered URN (fixes #23693) (#23849) ------------------------------------------ [...truncated 89.08 KB...] apache_beam/pipeline_test.py::DoFnTest::test_element_param apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_combine apache_beam/pipeline_test.py::DoFnTest::test_key_param apache_beam/runners/portability/fn_api_runner/translations_test.py::TranslationsTest::test_run_packable_combine_per_key apache_beam/metrics/metric_test.py::MetricsTest::test_user_counter_using_pardo apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiBasedStateBackedCoderTest::test_gbk_many_values apache_beam/runners/portability/fn_api_runner/translations_test.py::TranslationsTest::test_run_packable_combine_limit apache_beam/runners/portability/fn_api_runner/translations_test.py::TranslationsTest::test_run_packable_combine_globally [gw2] [32mPASSED[0m apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiBasedStateBackedCoderTest::test_gbk_many_values apache_beam/transforms/combiners_test.py::TimestampCombinerTest::test_combiner_latest [gw2] [32mPASSED[0m apache_beam/transforms/combiners_test.py::TimestampCombinerTest::test_combiner_latest apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_pcollections > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests [gw6] [32mPASSED[0m apache_beam/metrics/metric_test.py::MetricsTest::test_user_counter_using_pardo apache_beam/transforms/combiners_test.py::TimestampCombinerTest::test_combiner_earliest [gw6] [32mPASSED[0m apache_beam/transforms/combiners_test.py::TimestampCombinerTest::test_combiner_earliest apache_beam/transforms/ptransform_test.py::PTransformTest::test_impulse > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests [gw1] [32mPASSED[0m apache_beam/pipeline_test.py::DoFnTest::test_key_param apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_non_liftable_combine > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests [gw2] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_pcollections apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_same_pcollections > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests [gw5] [32mPASSED[0m apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_combine apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_one_single_pcollection [gw7] [32mPASSED[0m apache_beam/runners/portability/fn_api_runner/translations_test.py::TranslationsTest::test_run_packable_combine_per_key apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_multiple_pcollections_having_multiple_consumers > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests [gw7] [32mPASSED[0m apache_beam/pipeline_test.py::DoFnTest::test_element_param apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_combining_value_state [gw7] [33mSKIPPED[0m apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_combining_value_state apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_yield [gw5] [32mPASSED[0m apache_beam/pipeline_test.py::DoFnTest::test_key_param apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_non_liftable_combine [gw3] [32mPASSED[0m apache_beam/runners/portability/fn_api_runner/translations_test.py::TranslationsTest::test_run_packable_combine_globally apache_beam/transforms/dofn_lifecycle_test.py::DoFnLifecycleTest::test_dofn_lifecycle [gw4] [32mPASSED[0m apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_combine apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_one_single_pcollection [gw1] [32mPASSED[0m apache_beam/runners/portability/fn_api_runner/translations_test.py::TranslationsTest::test_run_packable_combine_per_key apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_multiple_pcollections_having_multiple_consumers > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests [gw2] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_pcollections apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_same_pcollections [gw0] [32mPASSED[0m apache_beam/pipeline_test.py::DoFnTest::test_element_param apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_combining_value_state [gw0] [33mSKIPPED[0m apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_combining_value_state apache_beam/transforms/ptransform_test.py::PTransformTest::test_read_metrics [gw3] [32mPASSED[0m apache_beam/metrics/metric_test.py::MetricsTest::test_user_counter_using_pardo apache_beam/transforms/combiners_test.py::TimestampCombinerTest::test_combiner_earliest [gw3] [32mPASSED[0m apache_beam/transforms/combiners_test.py::TimestampCombinerTest::test_combiner_earliest apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_dict_twice > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests [gw0] [32mPASSED[0m apache_beam/runners/portability/fn_api_runner/translations_test.py::TranslationsTest::test_run_packable_combine_limit apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_a_flattened_pcollection > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests [gw6] [32mPASSED[0m apache_beam/runners/portability/fn_api_runner/translations_test.py::TranslationsTest::test_run_packable_combine_limit apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_a_flattened_pcollection [gw4] [32mPASSED[0m apache_beam/runners/portability/fn_api_runner/translations_test.py::TranslationsTest::test_run_packable_combine_globally apache_beam/transforms/dofn_lifecycle_test.py::DoFnLifecycleTest::test_dofn_lifecycle [gw1] [32mPASSED[0m apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_non_liftable_combine apache_beam/transforms/ptransform_test.py::PTransformTest::test_impulse > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests [gw3] [32mPASSED[0m apache_beam/transforms/dofn_lifecycle_test.py::DoFnLifecycleTest::test_dofn_lifecycle apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_dict_twice > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests [gw7] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_multiple_pcollections_having_multiple_consumers apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_return > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests [gw6] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_impulse apache_beam/transforms/ptransform_test.py::PTransformTest::test_multiple_empty_outputs [gw4] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_one_single_pcollection apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_and_as_dict_side_inputs [gw1] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_multiple_pcollections_having_multiple_consumers apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_twice > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests [gw5] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_one_single_pcollection apache_beam/transforms/ptransform_test.py::PTransformTest::test_multiple_empty_outputs > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests [gw2] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_same_pcollections apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_return [gw5] [32mPASSED[0m apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_non_liftable_combine apache_beam/transforms/ptransform_test.py::PTransformTest::test_undeclared_outputs > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests [gw4] [32mPASSED[0m apache_beam/transforms/dofn_lifecycle_test.py::DoFnLifecycleTest::test_dofn_lifecycle apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_with_different_defaults > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests [gw7] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_yield apache_beam/transforms/ptransform_test.py::PTransformTest::test_read_metrics > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests [gw3] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_dict_twice apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_and_as_dict_side_inputs [gw2] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_same_pcollections apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_yield [gw0] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_read_metrics apache_beam/transforms/ptransform_test.py::PTransformTest::test_undeclared_outputs [gw6] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_a_flattened_pcollection apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_twice > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests [gw0] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_a_flattened_pcollection apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_with_different_defaults [gw3] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_dict_twice apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_without_unique_labels > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests [gw1] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_impulse apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_without_unique_labels [gw4] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_with_different_defaults apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_flattened_side_input [gw7] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_return apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_default_value_singleton_side_input > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests [gw6] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_multiple_empty_outputs apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_default_value_singleton_side_input [gw4] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_and_as_dict_side_inputs apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_empty_singleton_side_input [gw7] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_read_metrics apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_reiterable_side_input > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests [gw3] [32mPASSED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_and_as_dict_side_inputs apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_iterable_side_input [gw2] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_yield apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_multi_valued_singleton_side_input [gw5] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_multiple_empty_outputs apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_empty_singleton_side_input > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests [gw5] [32mPASSED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_undeclared_outputs apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_multi_valued_singleton_side_input [gw2] node down: Not properly terminated [gw2] [31mFAILED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_return replacing crashed worker gw2 [1m[gw8] linux Python 3.7.12 cwd: <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python> [0m[1m[gw8] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0] [0m[gw1] node down: Not properly terminated [gw1] [31mFAILED[0m apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_twice Terminated Traceback (most recent call last): File "<string>", line 1, in <module> File "<string>", line 5, in <module> > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests FAILED > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests [gw0] node down: Not properly terminated [gw0] [31mFAILED[0m apache_beam/transforms/ptransform_test.py::PTransformTest::test_undeclared_outputs Terminated Remove shutdown hook failed java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82) at java.lang.Runtime.removeShutdownHook(Runtime.java:239) at org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:38) at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208) at org.gradle.process.internal.DefaultExecHandle.aborted(DefaultExecHandle.java:366) at org.gradle.process.internal.ExecHandleRunner.completed(ExecHandleRunner.java:108) at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:84) at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64) at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerBatchTests FAILED > Task :sdks:python:test-suites:dataflow:py310:validatesRunnerStreamingTests > FAILED Remove shutdown hook failed java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82) at java.lang.Runtime.removeShutdownHook(Runtime.java:239) at org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:38) at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208) at org.gradle.process.internal.DefaultExecHandle.failed(DefaultExecHandle.java:370) at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:87) at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64) at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) > Task :sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests >>> RUNNING integration tests with pipeline options: >>> --runner=TestDataflowRunner --project=apache-beam-testing >>> --region=us-central1 >>> --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it >>> --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it >>> --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output >>> --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/build/apache-beam.tar.gz> >>> --requirements_file=postcommit_requirements.txt --num_workers=1 >>> --sleep_secs=20 --streaming >>> --dataflow_worker_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.44.0-SNAPSHOT.jar> >>> --experiments=use_runner_v2 --experiments=shuffle_mode=appliance >>> --enable_streaming_engine >>> pytest options: --capture=no --timeout=4500 --color=yes >>> --log-cli-level=INFO --numprocesses=8 >>> collect markers: -m=it_validatesrunner and not no_sickbay_streaming and >>> not no_xdist Daemon will be stopped at the end of the build after the daemon was no longer found in the daemon registry The message received from the daemon indicates that the daemon has disappeared. Build request sent: Build{id=a23aa3b7-db62-47df-8183-3aa8e865f30c, currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src}> Attempting to read last messages from the daemon log... Daemon pid: 903659 log file: /home/jenkins/.gradle/daemon/7.5.1/daemon-903659.out.log ----- Last 20 lines from daemon log file - daemon-903659.out.log ----- at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Daemon will be stopped at the end of the build after the daemon was no longer found in the daemon registry Remove shutdown hook failed java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82) at java.lang.Runtime.removeShutdownHook(Runtime.java:239) at org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:38) at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208) at org.gradle.process.internal.DefaultExecHandle.aborted(DefaultExecHandle.java:366) at org.gradle.process.internal.ExecHandleRunner.completed(ExecHandleRunner.java:108) at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:84) at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64) at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) ----- End of the daemon log ----- FAILURE: Build failed with an exception. * What went wrong: Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed) * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
