github-actions[bot] commented on code in PR #69896:
URL: https://github.com/apache/doris/pull/69896#discussion_r4425241002
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/analysis/TableStatsMeta.java:
##########
@@ -414,6 +426,31 @@ public synchronized long
getRowCountWithDeltaRows(OlapTable table, long indexId)
return rowCount + getBaseIndexDeltaRowCount(table);
}
+ /**
+ * Add the rows a load wrote to an index which has no collected row count
of its own.
+ */
+ public synchronized void addIndexUpdatedRows(long indexId, long rows) {
+ if (rows <= 0) {
+ return;
+ }
+ indexUpdatedRows().merge(indexId, rows, Long::sum);
+ }
Review Comment:
[P2] Reset changelog deltas when a partition is truncated. Partial TRUNCATE
PARTITION replaces the partition and its changelog tablets but keeps the
table's TableStatsMeta; the subsequent dropStats invalidates column/partition
statistics without clearing indexUpdatedRows. A later changelog row-count
fallback therefore returns rows from the deleted partition as if they were
still present. Clear or partition-scope this accumulator as part of the
truncate/replay lifecycle so the estimate cannot retain removed rows.
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/analysis/TableStatsMeta.java:
##########
@@ -391,6 +397,12 @@ public long getBaseIndexDeltaRowCount(OlapTable table) {
* {@link #updatedRows} and reports them as delta rows.
*/
public synchronized long getRowCountWithDeltaRows(OlapTable table, long
indexId) {
+ if (isChangelogIndex(table, indexId)) {
+ // The changelog index has no collected row count: the rows a load
wrote to it are the
+ // only count this record keeps, and they are added rather than
taken as a lower bound
Review Comment:
[P2] Preserve the existing changelog baseline when recreating table
statistics. After DROP STATS (or another removal/recreation of TableStatsMeta),
new TableStatsMeta(OlapTable) initializes no changelog row count. While the
changelog tablet's FE row count is still unknown, StatsCalculator falls back to
this branch, which returns only indexUpdatedRows from loads after the record
was created. Existing changelog rowsets therefore remain visible but are
estimated as zero/partial until a separate tablet report supplies a count.
Preserve an existing changelog baseline or keep the fallback unknown instead of
converting a missing baseline to zero.
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/analysis/AnalysisManager.java:
##########
@@ -1512,8 +1513,17 @@ public void replayUpdateRowsRecord(UpdateRowsEvent
event) {
LOG.info("Tablet row count map is empty");
continue;
}
+ long changelogIndexId = olapTable.hasChangelog()
+ ? olapTable.getChangelogIndexMeta().getIndexId() : -1;
long rowsForAllReplica = 0;
+ long changelogRowsForAllReplica = 0;
for (Entry<Long, Long> entry : tabletRows.entrySet()) {
+ if (changelogIndexId > 0 &&
isChangelogTablet(entry.getKey(), changelogIndexId)) {
+ // The change rows of the load, which are neither the
rows loaded into the
+ // base index nor a lower bound of the table's rows:
see TableStatsMeta.
Review Comment:
[P2] Do not route unknown tablet IDs into the base counter. When a load's
partition is force-dropped before its transaction becomes visible,
DatabaseTransactionMgr removes the missing partition from the commit info but
still forwards the transaction's tablet delta map. The old changelog tablet
metadata has already been removed, so isChangelogTablet returns false and this
branch adds its rows to updatedRows. That permanently inflates the base
estimate and leaves the changelog counter missing. Carry the index identity
with the event or discard/handle missing tablet metadata explicitly instead of
treating it as a base tablet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
