Abacn commented on issue #22283: URL: https://github.com/apache/beam/issues/22283#issuecomment-1191837062
Found this issue is introduced in grpcio-1.46.0. Nailed this down to changes added to grpc master between March 21 to March 22, most likely this commit: https://github.com/grpc/grpc/pull/29050 Also found the server that does not shutdown before grace time is mostly state_server; and control_server also prevents shutdown in some tests: no graceful shutsown for control_server alone: ``` 9.83s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithDisabledCaching::test_pack_combiners 9.77s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_pack_combiners 9.68s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpc::test_pack_combiners 5.47s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_pardo_side_and_main_outputs 5.22s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_batch_pardo_fusion_break 5.21s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_pardo_side_outputs 5.19s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_pardo_unfusable_side_inputs_with_separation 5.18s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpc::test_batch_to_element_pardo 5.16s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpc::test_batch_pardo_fusion_break 5.16s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerSplitTest::test_checkpoint_draining_sdf 5.11s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_flatten_same_pcollections 5.11s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_assert_that 5.01s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_pardo_state_only 5.00s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_flattened_side_input 5.00s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_multimap_side_input 5.00s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_sdf 5.00s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_batch_to_element_pardo 5.00s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_callbacks_with_exception 5.00s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_pardo_windowed_side_inputs 5.00s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_combine_per_key 5.00s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerSplitTest::test_split_crazy_draining_sdf 5.00s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_batch_pardo_overlapping_windows 4.99s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_pardo_side_inputs 4.94s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpc::test_custom_merging_window 4.88s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_custom_merging_window 4.86s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpc::test_pardo_side_outputs 4.82s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_batch_rebatch_pardos 4.68s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_pardo_state_timers_non_standard_coder 4.67s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_windowing 4.65s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithDisabledCaching::test_multimap_side_input_type_coercion 4.64s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpc::test_pardo_side_and_main_outputs 4.56s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerSplitTest::test_checkpoint_sdf 4.56s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpc::test_pardo_side_inputs 4.51s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithDisabledCaching::test_pardo_side_and_main_outputs 4.44s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_pardo_timers_clear 4.42s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_multimap_multiside_input 4.41s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_pardo_side_input_dependencies 4.38s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_sdf_synthetic_source 4.25s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithDisabledCaching::test_register_finalizations 4.18s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_flatten 4.00s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpc::test_sdf_synthetic_source 3.59s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpc::test_register_finalizations 3.14s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_multimap_side_input_type_coercion 2.88s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_reshuffle 2.26s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerSplitTest::test_checkpoint 2.07s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithGrpcAndMultiWorkers::test_sdf_with_dofn_as_restriction_provider 1.97s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerSplitTest::test_split_crazy_sdf 1.23s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithBundleRepeatAndMultiWorkers::test_pardo_side_and_main_outputs 1.17s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTest::test_register_finalizations 1.14s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithBundleRepeatAndMultiWorkers::test_pack_combiners 0.93s call apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTestWithMultiWorkers::test_pack_combiners ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
