yujun777 opened a new pull request, #70839:
URL: https://github.com/apache/doris/pull/70839

   ### Background
   
   A merge-on-write incremental materialized view stores a signature of its 
normalized write plan. The signature protects row identity and aggregate state 
layout across refreshes. A partition refresh rebuilds a subset of the view from 
its base tables.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: #70785 (establishes the signature on the first successful write)
   PR Type: Bugfix
   
   Problem Summary:
   
   A partition refresh could write a different layout into part of a view while 
other partitions retained the stored layout. Compare the signature before each 
partition write. A strict refresh reports a mismatch and a request that allows 
fallback runs a whole-view COMPLETE.
   
   ```sql
   REFRESH MATERIALIZED VIEW ivm_part_sig_mv PARTITIONS;
   ```
   
   | With a changed signature | Task result |
   | --- | --- |
   | Before | `SUCCESS` with a partial refresh |
   | After | `FAILED` with `IVM layout signature mismatch` |
   
   ### Details
   
   - **Frequency:** Always when a MOW IVM partition rebuild has work to do and 
generates a signature different from the stored one.
   - **Root cause:** `MTMVTask` used the full refresh rewrite mode for 
partition rebuilds. `IvmNormalizeUniqKeysMTMV` compared signatures for 
incremental writes, and the full mode only generated a candidate signature.
   - **Why existing tests missed it:** The normalizer unit tests and 
`test_ivm_plan_signature_fallback` checked signature changes on the incremental 
write path. The partition rebuild regressions kept the layout fixed.
   - **Fix:** Check the stored signature during normalization for MOW partition 
writes, covering explicit PARTITIONS and the dirty-partition rebuild during 
INCREMENTAL. Keep the existing stream reads and let the first successful write 
establish the signature when it is unset.
   - **Other paths and branches:** COMPLETE still checks its own batch 
signatures for consistency and can replace the baseline on success. Append-only 
DUP and ordinary materialized views keep their existing refresh paths. This 
change follows the signature lifecycle in #70785 on master.
   - **Behavior change:** Strict partition rebuilds fail before writing a 
mismatching batch. Fallback-enabled requests rebuild the whole view and publish 
its new signature on success.
   
   ### Testing
   
   - On the previous base, `test_ivm_partition_signature_fallback` reproduced a 
successful strict partition refresh with signature drift before the fix. The 
test passed with the fix and verified strict failures, unchanged rows, COMPLETE 
fallback, and a subsequent matching partition refresh.
   - Before the rebase, six local IVM regression suites and 20 targeted FE 
tests passed.
   - After rebasing onto upstream master, five targeted signature and 
complete-refresh FE tests passed.
   
   ### Release note
   
   None.
   
   ### Check List (For Author)
   
   - Behavior changed: Yes (see Details)
   - Does this need documentation: No
   - Pre-submission status: Ready
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to