yujun777 commented on code in PR #68170:
URL: https://github.com/apache/doris/pull/68170#discussion_r4062085905
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/mtmv/MTMVTask.java:
##########
@@ -855,6 +889,55 @@ private void
executePartitionBasedRefresh(MTMVRefreshContext context, RefreshMod
mtmv.getDatabase().getFullName(), mtmv.getName(), getTaskId());
}
+ /**
+ * Whether a base table the refresh reads has no stream that can be read,
which no attempt other
+ * than COMPLETE can work around.
+ *
+ * <p>A base table that cannot be resolved is skipped rather than judged:
it says nothing about the
+ * streams, and the refresh fails on it for its own reasons -- the attempt
that runs reports that,
+ * this one only decides which attempt that should be.
+ */
+ private boolean hasUnusableIvmStream() {
+ Database mvDb = (Database) mtmv.getDatabase();
+ if (mvDb == null) {
+ // Nothing to look the streams up in, so there is nothing to
decide here.
+ return false;
+ }
+ Set<TableNameInfo> excluded = mtmv.getExcludedTriggerTables();
+ // The tables in the plan, not the relation's closure: a chained MV is
created with a stream for
+ // every base table behind the MVs it reads, but no rewrite ever looks
those up -- the incremental
+ // rewriter and the full refresh take the streams of the plan's scans
-- so judging them would
+ // rebuild an MV whose refresh had nothing wrong with it.
+ for (BaseTableInfo baseTableInfo :
relation.getBaseTablesOneLevelAndFromView()) {
Review Comment:
Fixed in 0148150244c. You are right, and the mechanism is exactly as you
describe: `IvmFullRefreshMTMV.rewriteScan` returns the plain scan when a PCT
table is not in the reset mapping (`if (isPctTable ||
!nonPctReadMode.isPresent()) { return scan; }`), so `getIvmStream` is never
called for it.
The check now lives in the partition attempt instead of before it, because
that is the first point where the scope is known.
`executePartitionBasedRefresh` plans its partitions, then judges exactly the
set it will read -- the non-PCT tables of the plan plus the PCT tables
appearing in the refreshed partitions' mappings, which is the same set the
reset read mode is built from -- and falls back through the existing path (a
`false` return continues to COMPLETE) rather than failing. Two consequences
worth noting:
- it runs before the baseline barrier is written, so an attempt that cannot
succeed no longer leaves a partial guard behind, which was the other half of
the earlier thread;
- a request that may not fall back (`REFRESH ... PARTITIONS` without
FALLBACK, or an explicit partition list) still fails instead of quietly
refreshing less than it asked for.
`buildAttempts` keeps the check for the incremental attempt only, where the
plan's tables are exactly what the rewrite scans. It is scoped to requests
whose attempt list contains the incremental attempt, so a `PARTITIONS FALLBACK`
request no longer inherits a verdict taken from the whole plan.
Coverage:
`MTMVTaskTest.testPartitionRefreshChecksOnlyTheStreamsItsPartitionsRead` pins
both halves -- a PCT table that no refreshed partition's mapping names is not
judged, and as soon as one names it, its missing stream decides. The assertion
fails when the whole plan is judged (I checked that by reverting the scoping),
and
`regression-test/suites/mtmv_p0/ivm/test_ivm_partitions_fallback_stream_unusable`
still pins the end-to-end fallback through the new path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
