github-actions[bot] commented on code in PR #69422:
URL: https://github.com/apache/doris/pull/69422#discussion_r4344942386
##########
be/src/runtime/exec_env.cpp:
##########
@@ -191,20 +194,41 @@ std::map<TNetworkAddress, FrontendInfo>
ExecEnv::get_running_frontends() {
return res;
}
+void ExecEnv::wait_for_shutdown_heartbeat() {
+ int32_t elapsed = 0;
+ while (!k_shutdown_heartbeat_sent && elapsed <
config::grace_shutdown_heartbeat_wait_seconds) {
+ sleep(1);
+ ++elapsed;
+ }
+ LOG(INFO) << "shutdown heartbeat prepared: " <<
k_shutdown_heartbeat_sent.load() << ", waited "
+ << elapsed << " seconds";
+ // The response may still be in transit. This buffer also gives followers
time
+ // to replay shutdown state. It does not prove that every FE has applied
it.
+ sleep(std::max(0, config::grace_shutdown_heartbeat_propagation_seconds));
+}
+
void ExecEnv::wait_for_all_tasks_done() {
- // For graceful shutdown, need to wait for all running queries to stop
+ // Both receiver implementations and the load coordinator must finish. For
example,
+ // an HTTP upload or transaction commit can outlive all of its pipeline
fragments.
int32_t wait_seconds_passed = 0;
while (true) {
int num_queries = _fragment_mgr->running_query_num();
- if (num_queries < 1) {
+ size_t num_channels = _load_channel_mgr->num_active_channels();
+ size_t num_streams = _load_stream_mgr->get_load_stream_num();
+ size_t num_loads = StreamLoadContext::num_active_contexts();
Review Comment:
[P2] Release failed transactional contexts before counting them as active
A transactional INSERT can leave this count positive until process teardown
after setup has already failed. FragmentMgr::exec_plan_fragment() puts its
StreamLoadContext in NewLoadStreamMgr before execute_plan_fragment(), but an
early _get_or_create_query_ctx() failure (for example, a backend tag change
causes an older workload-group publication to shut down the selected normal
group before set_workload_group()) returns before any PipelineFragmentContext
and completion callback exist. The manager retains a shared owner with no
timeout cleanup; the rollback RPC only cancels its pipe. The new lifetime
census therefore makes a later, otherwise idle shutdown wait all of
grace_shutdown_wait_seconds (120 seconds by default) for a failed load that
cannot make progress. Release the registered context and finish rollback on
these early setup failures, and cover failure before fragment creation in the
shutdown tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
