yujun777 commented on PR #68648: URL: https://github.com/apache/doris/pull/68648#issuecomment-6394562237
The external regression in `mtmv_p0/test_iceberg_mtmv` was related to this PR, and is fixed in e77717d. Reproduced locally on the PR head: `test_ts_refresh_null` came back empty instead of `\N 8`, while the same suite passes on the merge base. Root cause: the base table's day-truncated null-key partition is one the MV partition mapping does not describe -- the generated roll-up descriptor and the MV partition's own descriptor differ -- so `MTMV#calculatePartitionMappings` matches nothing for that MV partition. That gap itself is pre-existing, and `MTMVTask`'s undescribed-partition guard is intentionally olap-only: an external partition the mapping cannot describe, such as a null partition key, is not an alignment lag the MV ever catches up with. Master read such an MV partition through its own key range, which reaches the null rows via `IS NULL`. The new base-partition scope turned the empty mapping into "this table feeds the partition nothing" and refreshed it as `FALSE`, so the refresh wrote no rows over the rows the partition already held. Fix: `MTMVPartitionUtil#mappedBasePartitions` now leaves the batch unscoped when it contains an MV partition no base partition describes, keeping the key-range read that partition had before partitions were scoped. A batch whose MV partitions are all described keeps the scoped read unchanged. Validated on a local cluster with the fix built from this branch: mtmv_p0 `test_iceberg_mtmv` (including `test_ts_refresh_null`), `test_mtmv_base_partition_read_scope`, `test_mtmv_external_pct_read_scope`, `test_mtmv_external_pct_compensation`, `test_multi_pct_null_partition_mtmv`, `test_null_partition_mtmv`, `test_iceberg_mtmv_with_partition_evolution`, `test_partition_refresh_mtmv`, `test_limit_partition_mtmv`, and `mtmv_p0/ivm/test_ivm_undescribed_partition_refresh` all pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
