morrySnow commented on code in PR #68282:
URL: https://github.com/apache/doris/pull/68282#discussion_r4245754130
##########
fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java:
##########
@@ -3543,7 +3543,7 @@ public void modifyTableLightSchemaChange(String rawSql,
Database db, OlapTable o
if (!FeConstants.runningUnitTest) {
Env.getCurrentEnv().getEditLog().logModifyTableAddOrDropInvertedIndices(info);
// Drop table column stats after light schema change
finished.
-
Env.getCurrentEnv().getAnalysisManager().removeTableStats(olapTable.getId());
+
Env.getCurrentEnv().getAnalysisManager().removeTableStatsAndLog(olapTable.getId());
Review Comment:
这里再写一个editlog的目的是什么？在replay ModifyTableAddOrDropInvertedIndices
时，不会顺带把removeTableStats执行了嘛？
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/analysis/AnalysisManager.java:
##########
@@ -1487,6 +1487,75 @@ public void removeTableStats(long tableId) {
}
}
+ /**
+ * Remove the statistics record of a table and write the journal entry of
the removal while holding the
+ * monitor of the statistics records, so that this transition and a
concurrent truncation, which writes
+ * its own entry under the same monitor, are ordered the same way in
memory and in the journal. Replay
+ * paths, which must not journal, use {@link #removeTableStats(long)}.
+ */
+ public void removeTableStatsAndLog(long tableId) {
+ synchronized (idToTblStats) {
+ idToTblStats.remove(tableId);
+ Env.getCurrentEnv().getEditLog().logDeleteTableStats(new
TableStatsDeletionLog(tableId));
+ }
+ }
+
+ /**
+ * TRUNCATE TABLE removes all the data of the table, but the table itself
stays and can be loaded
+ * immediately. The stats record must be kept, otherwise the row count of
the newly loaded data can
+ * never be reported: the backends report the row count of the new empty
tablets with a delay of up to
+ * {@code tablet_stat_update_interval_second}, and without a record there
is nothing to accumulate the
+ * loaded rows into. So reset the record to the state of an empty table
instead of removing it.
+ *
+ * <p>The reset and the journal entry of the truncation are written while
holding the monitor of the
+ * statistics records, and so is the deletion of a record, so that the
order of two concurrent
+ * transitions between the states of a record is the same in memory and in
the journal: otherwise a
+ * truncation and a deletion which do not share a table lock could be
applied in one order and replayed
+ * in the other, leaving the followers with a record the master removed
(or the reverse).
+ *
+ * @return whether the record had to be created, i.e. the table had no
statistics record
+ */
+ public boolean resetTableStats(OlapTable table, TruncateTableInfo
truncateInfo) {
+ synchronized (idToTblStats) {
+ TableStatsMeta tableStats = idToTblStats.get(table.getId());
+ boolean recordCreated = false;
+ if (tableStats == null) {
+ tableStats = new TableStatsMeta(table);
+ idToTblStats.put(table.getId(), tableStats);
+ recordCreated = true;
+ }
+ // The entry records what this truncation did to the record, so
that the replay reproduces it.
+ truncateInfo.setTableStatsRecordCreated(recordCreated);
+ tableStats.reset(table);
+ Env.getCurrentEnv().getEditLog().logTruncateTable(truncateInfo);
+ return recordCreated;
+ }
+ }
+
+ /**
+ * Apply the transition which the given truncate entry recorded, instead
of the one this frontend would
+ * apply now.
+ *
+ * <p>A truncate creates the statistics record of a table which has none,
and that creation is not a
+ * journal entry of its own. The truncate entry therefore carries whether
the truncate created the record,
+ * so that the replay reproduces exactly what the master did and a
truncate entry never resurrects a
+ * record which another journaled transition, for instance a whole table
DROP STATS or the cleanup of an
+ * empty table by the analyzer, removed around it.
+ */
+ public void replayResetTableStats(OlapTable table, boolean recordCreated) {
Review Comment:
`recordCreated` 的目的是什么？什么情况下 tableStats == null 并且 recordCreated == false？
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##########
@@ -3903,19 +3907,21 @@ public void truncateTable(String dbName, String
tableName, PartitionNamesInfo pa
long versionTimeMs = Config.isNotCloudMode() ?
System.currentTimeMillis() : 0L;
oldPartitions = truncateTableInternal(olapTable, newPartitions,
truncateEntireTable, recyclePartitionParamMap, forceDrop,
version, versionTimeMs);
- if (truncateEntireTable) {
-
Env.getCurrentEnv().getAnalysisManager().removeTableStats(olapTable.getId());
- } else {
- Env.getCurrentEnv().getAnalysisManager().updateUpdatedRows(
- updateRecords, db.getId(), olapTable.getId(), 0);
- }
-
// write edit log
TruncateTableInfo info =
new TruncateTableInfo(db.getId(), db.getFullName(),
olapTable.getId(), olapTable.getName(),
newPartitions, truncateEntireTable,
rawTruncateSql, oldPartitions, forceDrop,
updateRecords, version, versionTimeMs);
- Env.getCurrentEnv().getEditLog().logTruncateTable(info);
Review Comment:
这个log移动到else里面，是为什么？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
