github-actions[bot] commented on code in PR #66247:
URL: https://github.com/apache/doris/pull/66247#discussion_r3680287516
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/metacache/paimon/PaimonPartitionInfoLoader.java:
##########
@@ -46,7 +45,10 @@ public PaimonPartitionInfo load(NameMapping nameMapping,
Table paimonTable, List
return PaimonPartitionInfo.EMPTY;
}
try {
- List<Partition> paimonPartitions =
tableLoader.catalog(nameMapping).getPaimonPartitions(nameMapping);
+ // Catalog.listPartitions reloads the raw physical table and loses
Doris catalog/relation
+ // copies. Enumerate the already merged handle so validation and
planning see one table.
+ PaimonReaderOptions.validateEffectivePlanningTable(paimonTable);
Review Comment:
[P1] Preserve the relation override during pre-lock preload. Selector-free
Paimon `@options` are registered as a latest relation, but when preload is
enabled for a mixed internal/external query, `PreloadExternalMetadata` calls
`loadSnapshots(table, empty, empty)` before `BindRelation`. A partitioned
physical table with `scan.manifest.parallelism=0` therefore reaches this
validator and fails before its safe relation override of `1` is applied. Please
retain the scan params/effective handle in preload, or skip latest-snapshot
preload for these relations, and cover the mixed-table preload path.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/PaimonExternalTable.java:
##########
@@ -178,6 +178,19 @@ private PaimonSnapshotCacheValue
getPaimonSnapshotCacheValue(Optional<TableSnaps
"Failed to get Paimon snapshot: " + (e.getMessage() ==
null ? "unknown cause" : e.getMessage()),
e);
}
+ } else if (scanParams.isPresent() && scanParams.get().isOptions()) {
+ Table baseTable = getBasePaimonTable();
+ Map<String, String> resolvedOptions =
scanParams.get().getOrResolveMapParams(
+ options -> PaimonScanParams.resolveOptions(baseTable,
options));
+ Table effectiveTable = PaimonScanParams.applyOptions(baseTable,
resolvedOptions);
+ if (PaimonScanParams.hasOnlyReaderOptions(resolvedOptions)) {
+ // Reader tuning cannot change snapshot metadata. Reuse the
memoized projection so
+ // a per-query batch-size change does not enumerate every
partition again.
+ return PaimonUtils.getLatestSnapshotCacheValue(this);
+ }
+ // The shared latest cache was built from the catalog-scoped
handle. Relation options
+ // need their own projection so partition enumeration uses the
final safe table copy.
+ return PaimonUtils.loadSnapshotProjection(this, effectiveTable);
Review Comment:
[P2] Keep this option-bearing projection relation-scoped.
`StatementContext.loadSnapshots` has only one `MvccTableInfo` slot per
catalog/db/table, so if the first alias uses
`@options('scan.manifest.parallelism'='1')`, this snapshot retains `1` and a
later unannotated alias reuses it through `PaimonSource`; reversing alias order
instead rejects the raw physical `0`. That makes relation isolation and
validation traversal-order-dependent. Please keep the shared latest snapshot
neutral or key option projections by relation/scan parameters, and add a
two-alias order test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
