github-actions[bot] commented on code in PR #66187:
URL: https://github.com/apache/doris/pull/66187#discussion_r4408471226
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindRelation.java:
##########
@@ -800,6 +806,18 @@ private boolean
isScanAppendOnlyTableStream(OlapTableStream table) {
private LogicalPlan getLogicalPlan(TableIf table, UnboundRelation
unboundRelation,
List<String> qualifiedTableName,
CascadesContext cascadesContext) {
+ if (unboundRelation.isMaxVisiblePartition()) {
+ TableIf.TableType type = table.getType();
+ if (type != TableIf.TableType.OLAP && type !=
TableIf.TableType.MATERIALIZED_VIEW) {
+ throw new AnalysisException("max_visible_partition() is only
supported on OLAP table "
+ + "or materialized view, but table " + table.getName()
+ " is " + type);
+ }
+ if (unboundRelation.getTableSnapshot().isPresent()) {
+ throw new AnalysisException("max_visible_partition() cannot be
used with FOR VERSION/TIME AS OF");
+ }
+
cascadesContext.getStatementContext().setHasMaxVisiblePartition(true);
Review Comment:
[P1] Reject this modifier in MV definitions even on empty sources
For a partition-following async MV defined as `SELECT id,val FROM src
max_visible_partition()`, creation on an empty RANGE source currently succeeds:
`getPartitionIds()` returns `[]`, the scan has no
`manuallySpecifiedPartitions`, and `MTMVPlanUtil.analyzeQueryInternal()` /
`TableQueryOperatorChecker` therefore miss the prohibited scan operator. Create
it with deferred build and partition refresh, load p1 and refresh, then load
higher p2 and refresh again. PCT freshness compares each mapped source
partition separately, so unchanged mv.p1 is retained and only mv.p2 is
overwritten:
```text
Overwrite(mv.p2)
Filter(id in p2)
Scan(src, selected=[p2])
```
The MV now contains p1+p2, although its defining query selects only p2. This
is distinct from applying the clause while querying an ordinary MV, which the
added test covers. Please reject the clause in MV definitions using the
syntactic/statement flag regardless of current data, or track selection changes
so previously selected MV partitions are invalidated; add the empty-create ->
p1 refresh -> p2 refresh regression.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindRelation.java:
##########
@@ -1056,14 +1074,18 @@ private Plan parseAndAnalyzeView(TableIf view, String
ddlSql, CascadesContext pa
private List<Long> getPartitionIds(TableIf t, UnboundRelation
unboundRelation, List<String> qualifier) {
List<String> parts = unboundRelation.getPartNames();
- if (CollectionUtils.isEmpty(parts)) {
+ if (CollectionUtils.isEmpty(parts) &&
!unboundRelation.isMaxVisiblePartition()) {
return ImmutableList.of();
}
if (!t.isManagedTable()) {
throw new AnalysisException(String.format(
"Only OLAP table is support select by partition for now,"
+ "Table: %s is not OLAP table", t.getName()));
}
+ if (unboundRelation.isMaxVisiblePartition()) {
+ Partition partition = t.getMaxVisiblePartition();
+ return partition == null ? ImmutableList.of() :
ImmutableList.of(partition.getId());
Review Comment:
[P1] Keep an empty maximum distinct from an unrestricted scan
`[]` is the existing unrestricted-scan sentinel. On cloud FE A, prime empty
p1/p2 version entries, then load both through FE B while A's positive
partition-version TTL is still valid (disable version synchronization to make
this deterministic). For a changelog-enabled DUP table, `SELECT id FROM
t@incr() max_visible_partition()` gets `null` from the cached-empty maximum.
`makeOlapScan()` consequently gives the changelog wrapper all partition IDs.
Unlike ordinary empty pruning,
`OlapChangelogTableWrapper.selectNonEmptyPartitionIds()` forcibly refreshes
cached-empty versions, discovers both loads and retains both:
```text
ResultSink
Scan(changelog(t), selected=[p1,p2], @incr)
```
The query returns changes from both partitions. The original selection was
empty; a fresh maximum would be p2. Neither permits p1+p2. Please preserve an
explicit empty selection through normal scan validation, or select the maximum
from the same fresh version snapshot used by the change reader. Add a cloud
cached-empty-to-populated incremental-read regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
