Gabriel39 commented on code in PR #67053:
URL: https://github.com/apache/doris/pull/67053#discussion_r3843857453
##########
be/src/exec/pipeline/pipeline_fragment_context.cpp:
##########
@@ -2246,8 +2261,9 @@ bool PipelineFragmentContext::_close_fragment_instance() {
}
Defer defer_op {[&]() { _is_fragment_instance_closed = true; }};
_fragment_level_profile->total_time_counter()->update(_fragment_watcher.elapsed_time());
- if (!_need_notify_close) {
- auto st = send_report(true);
+ if (!_need_notify_close && !_final_report_submitted) {
+ _final_report_submitted = true;
+ auto st = send_report(true, _finish_load_fragment());
Review Comment:
Fixed in e81012f040 by removing fragment completion counting entirely. Every
LOAD done report now carries a cumulative statistics snapshot in the same
ReportExecStatus request. Intermediate recursive generations do not send the
logical final report, while FINAL_CLOSE is still deferred until the last target
task closes. Sequence ordering makes concurrently produced snapshots
deterministic without counting rebuildable PFCs.
##########
be/src/exec/pipeline/pipeline_fragment_context.cpp:
##########
@@ -2290,6 +2306,20 @@ bool PipelineFragmentContext::_close_fragment_instance()
{
return !_need_notify_close;
}
+bool PipelineFragmentContext::_finish_load_fragment() {
+ // FINAL_CLOSE can also release a context whose preparation never
installed RuntimeState;
+ // cleanup in that state must not attempt to publish load statistics.
+ if (_runtime_state == nullptr) {
+ return false;
+ }
+ if (_runtime_state->query_type() != TQueryType::LOAD) {
+ return false;
+ }
+ // A fragment closes only after its tasks and sink writers close. Let only
the last group of
+ // local LOAD instances trigger the final snapshot before the coordinator
can start auditing.
+ return _query_ctx->finish_fragment(_num_instances);
Review Comment:
Fixed in e81012f040. There is no planned-fragment equality gate anymore.
Normal and cancellation-driven done reports carry cumulative statistics
automatically. Initial and recursive REBUILD prepare failures, which have no
later task-close callback, synchronously send the same combined
failure/statistics report before the execution RPC returns the failure to FE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
