yujun777 opened a new pull request, #70839: URL: https://github.com/apache/doris/pull/70839
### Background A merge-on-write incremental materialized view stores a signature of its normalized write plan. The signature protects row identity and aggregate state layout across refreshes. A partition refresh rebuilds a subset of the view from its base tables. ### What problem does this PR solve? Issue Number: None Related PR: #70785 (establishes the signature on the first successful write) PR Type: Bugfix Problem Summary: A partition refresh could write a different layout into part of a view while other partitions retained the stored layout. Compare the signature before each partition write. A strict refresh reports a mismatch and a request that allows fallback runs a whole-view COMPLETE. ```sql REFRESH MATERIALIZED VIEW ivm_part_sig_mv PARTITIONS; ``` | With a changed signature | Task result | | --- | --- | | Before | `SUCCESS` with a partial refresh | | After | `FAILED` with `IVM layout signature mismatch` | ### Details - **Frequency:** Always when a MOW IVM partition rebuild has work to do and generates a signature different from the stored one. - **Root cause:** `MTMVTask` used the full refresh rewrite mode for partition rebuilds. `IvmNormalizeUniqKeysMTMV` compared signatures for incremental writes, and the full mode only generated a candidate signature. - **Why existing tests missed it:** The normalizer unit tests and `test_ivm_plan_signature_fallback` checked signature changes on the incremental write path. The partition rebuild regressions kept the layout fixed. - **Fix:** Check the stored signature during normalization for MOW partition writes, covering explicit PARTITIONS and the dirty-partition rebuild during INCREMENTAL. Keep the existing stream reads and let the first successful write establish the signature when it is unset. - **Other paths and branches:** COMPLETE still checks its own batch signatures for consistency and can replace the baseline on success. Append-only DUP and ordinary materialized views keep their existing refresh paths. This change follows the signature lifecycle in #70785 on master. - **Behavior change:** Strict partition rebuilds fail before writing a mismatching batch. Fallback-enabled requests rebuild the whole view and publish its new signature on success. ### Testing - On the previous base, `test_ivm_partition_signature_fallback` reproduced a successful strict partition refresh with signature drift before the fix. The test passed with the fix and verified strict failures, unchanged rows, COMPLETE fallback, and a subsequent matching partition refresh. - Before the rebase, six local IVM regression suites and 20 targeted FE tests passed. - After rebasing onto upstream master, five targeted signature and complete-refresh FE tests passed. ### Release note None. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Ready -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
