englefly commented on PR #68282: URL: https://github.com/apache/doris/pull/68282#issuecomment-5808030370
New commit `4001d5d4532` fixes the P2 of the last review (inline `r4089657633`): the rows loaded after a truncation are now reported for a row-preserving rollup too, not only for the base index. **The case** (reproduced first, on the build of `f1359ea6196`) ``` CREATE TABLE t2 (k1 INT, k2 INT, v INT) DUPLICATE KEY(k1, k2) ...; ALTER TABLE t2 ADD ROLLUP r_dup (k1, v); -- drops a key column: the projection shape the analyzer accepts TRUNCATE TABLE t2; INSERT INTO t2 VALUES (1,1,1),(2,2,2),(3,3,3); EXPLAIN SELECT k1, v FROM t2 INDEX r_dup; -- cardinality=1, the base index scan reported 3 ``` The record kept by `TRUNCATE` only knew the base index: it was the only index seeded with an empty row count and the only one the loaded rows were added to. A `DUP_KEYS` rollup whose columns are all plain keeps one row per base row, so the loaded rows are its rows as well, and while its new tablets are unreported the planner clamped them to 1. **Fix** `TableStatsMeta.keepsOneRowPerBaseRow(table, indexId)` decides it once for both consumers: the base index, or a `DUP_KEYS` index without an aggregating column (key columns carry *no* aggregation type, value columns of a duplicate key index are `NONE`). Such an index is seeded empty by the reset and gets the delta in `getRowCountWithDeltaRows()` and in `StatsCalculator.computeDeltaRowCount()`. An aggregating index (or a unique key index, which merges rows) is unchanged and never gets the base delta. **Verification on this commit** - `EXPLAIN SELECT k1, v FROM t2 INDEX r_dup` after `TRUNCATE` + 3 rows: `cardinality=3` (was 1); `StatsCalculator.computeOlapScan` logs `tableRowCount is 3.0` (was `1.0`); `EXPLAIN SELECT * FROM t2` stays 3. - Unit tests: `TableStatsMetaTest` 11/11 (two new: a row-preserving rollup reports the loaded rows, an aggregating one does not), `AnalysisManagerTest` 18/18, `StatisticsRecordReplayTest` 2/2; checkstyle not skipped. - Regression: `test_row_count_after_truncate` (now with the projection-rollup case scanning `INDEX r_dup`) and `test_drop_stats_and_truncate`: 1 suite each, 0 failed. - The original end-to-end case is unchanged: `TRUNCATE TABLE t; INSERT (3 rows); EXPLAIN SELECT * FROM t;` reports `cardinality=3`. - One honest note from the process: my first version of the predicate compared the aggregation type against `NONE` only, and the key column of the rollup (`null` aggregation type) made it reject the rollup, so the live cardinality stayed 1. The unit tests did not catch it because I had hand-set `NONE` for the key column; both the predicate and the test mocks now use the real shapes. Earlier gaps remain as documented: analyze-completion journaling outside `idToTblStats`, the MTMV cache entry re-inserted after an invalidation, and creating the statistics record at CREATE TABLE time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
