github-actions[bot] commented on code in PR #68282:
URL: https://github.com/apache/doris/pull/68282#discussion_r4064954098
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##########
@@ -3904,7 +3904,7 @@ public void truncateTable(String dbName, String
tableName, PartitionNamesInfo pa
oldPartitions = truncateTableInternal(olapTable, newPartitions,
truncateEntireTable, recyclePartitionParamMap, forceDrop,
version, versionTimeMs);
if (truncateEntireTable) {
-
Env.getCurrentEnv().getAnalysisManager().removeTableStats(olapTable.getId());
+
Env.getCurrentEnv().getAnalysisManager().resetTableStats(olapTable);
Review Comment:
Cloud row-count updates can repopulate this newly retained record with rows
that the truncate discarded. CloudGlobalTransactionMgr.afterCommitTxnResp()
sends a tableId -> updated_row_count map into
AnalysisManager.updateUpdatedRows(), which applies it by table id and then
journals it, but this path has no truncate generation/tablet fence and is not
serialized by the table write lock (non-MoW tables may not take a commit lock).
A callback that is delayed until after this reset therefore counts pre-truncate
rows as post-truncate rows; if apply and journaling straddle the truncate,
followers/restarts can count them even when the master was reset afterward.
Carry a truncate/transaction generation through the cloud event (or
serialize/reject stale events) before retaining the record.
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/analysis/TableStatsMeta.java:
##########
@@ -130,6 +137,58 @@ public TableStatsMeta(long rowCount, AnalysisInfo
analyzedJob, TableIf table) {
update(analyzedJob, table);
}
+ /**
+ * Create a record for a table which doesn't have one yet, in the state of
an empty table. The rows
+ * loaded into the table are accumulated by {@link
AnalysisManager#replayUpdateRowsRecord}, so a record
+ * has to exist before the first load, otherwise these rows can never be
turned into a row count.
+ */
+ public TableStatsMeta(OlapTable table) {
+ this.ctlId = table.getDatabase().getCatalog().getId();
+ this.ctlName = table.getDatabase().getCatalog().getName();
+ this.dbId = table.getDatabase().getId();
+ this.dbName = table.getDatabase().getFullName();
+ this.tblId = table.getId();
+ this.tblName = table.getName();
+ this.idxId = -1;
+ this.indexesRowCount = buildEmptyIndexRowCount(table);
+ this.updatedRowsBase.set(0);
+ }
+
Review Comment:
This reset seeds every materialized index with 0, but StatsCalculator adds a
table-wide base-index delta whenever the selected index has no BE row count.
For an aggregate/rollup index, 100 rows loaded after TRUNCATE can collapse to
one rollup row; until BE reports that index, the fallback here becomes 0 + 100
and materially overestimates it. Keep the post-truncate fallback scoped to the
base index or track per-index deltas/baselines, and add coverage for a selected
aggregate index before its BE row-count report.
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java:
##########
@@ -6767,9 +6767,10 @@ public void replayTruncateTable(TruncateTableInfo info)
throws MetaNotFoundExcep
// In previous versions(before 2.1.8), there is no catalog info in
TruncateTableInfo,
// So if the catalog info is empty, we assume it's internal table.
getInternalCatalog().replayTruncateTable(info);
- if (info.isEntireTable()) {
-
Env.getCurrentEnv().getAnalysisManager().removeTableStats(info.getTblId());
- } else {
+ // The stats record of a wholly truncated table is reset by
InternalCatalog.replayTruncateTable()
+ // instead of being removed, so that the rows loaded after the
truncation are still accounted
+ // for. Only the rows removed by a partition truncation are
accounted here.
+ if (!info.isEntireTable()) {
Review Comment:
The truncate opcode is not mixed-version safe: this new master no longer
journals a separate stats reset, so an older FE replaying the same
OP_TRUNCATE_TABLE still executes its pre-change removeTableStats() branch. That
old FE then drops subsequent OP_UPDATE_ROWS events because no record exists,
leaving SHOW/planner fallback unknown after post-truncate loads and diverging
on failover/restart. Preserve a compatible reset marker/journal (or gate
serving/leadership during upgrade) and add a rolling-version replay test; the
existing reverse-direction compatibility handling does not cover this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
