mrhhsg commented on code in PR #68032:
URL: https://github.com/apache/doris/pull/68032#discussion_r4256994337
##########
fe/fe-core/src/main/java/org/apache/doris/resource/workloadschedpolicy/WorkloadRuntimeStatusMgr.java:
##########
@@ -178,11 +180,11 @@ private List<AuditEvent> getQueryNeedAudit() {
if (waitTimeMs <= queryAuditLogTimeout) {
continue;
}
- Set<Long> expectedBackendIds =
externalDmlAuditBackendIds.get(ae);
+ Set<Long> expectedBackendIds = auditBackendIds.get(ae);
Review Comment:
Not a regression of this PR. On master, a Flight statement's audit event is
released after `query_audit_log_timeout_ms` (5 s) with whatever BE statistics
have arrived, for every statistics field (scan bytes, local spill bytes, ...).
This PR only makes events with known participating BEs wait for their final
reports, up to `be_report_query_statistics_timeout_ms`. Deferring Flight audits
until DoGet finishes is a separate change to Flight auditing.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ShowDataCommand.java:
##########
@@ -585,8 +594,22 @@ public String toString() {
return toSql();
}
+ /**
+ * Bytes of query spill currently held in object storage, summed over the
alive BEs from their
+ * latest disk reports. Only the master FE keeps these values, so the
command is forwarded to it.
+ */
+ private long getRemoteSpillSize() throws AnalysisException {
+ long total = 0;
+ for (Backend backend :
Env.getCurrentSystemInfo().getAllBackendsByAllCluster().values()) {
+ if (backend.isAlive()) {
Review Comment:
Kept on purpose. A BE that the master considers dead may be down, and its
last report would be counted indefinitely: a gracefully stopped BE has normally
deleted its spill, so its last non-zero value would overcount until it comes
back. Heartbeat failing while disk reports keep succeeding is a transient
state; the BE is counted again as soon as its heartbeat recovers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
