github-actions[bot] commented on code in PR #69343:
URL: https://github.com/apache/doris/pull/69343#discussion_r4335228948
##########
fe/pom.xml:
##########
@@ -437,7 +437,7 @@ under the License.
with the SAME paimon jar; a version mismatch silently breaks that
FE->BE
deserialization at runtime. These two MUST stay equal — do NOT
override
paimon.version per-module. -->
- <paimon.version>1.4.2</paimon.version>
+ <paimon.version>2.1.0</paimon.version>
Review Comment:
[P1] Keep catalog query checks for the new file_indexes table
The SDK bump automatically exposes `$file_indexes` through
`SystemTableLoader.SYSTEM_TABLES`, including on tables with
`query-auth.enabled=true`. Its FE scan only enumerates partition markers; the
catalog check happens later in `FileIndexesRead -> FilesSplit.splits ->
storeTable.newScan().plan()`. Doris drops the catalog loader before that BE
read, making `CatalogEnvironment.tableQueryAuth` a no-op, while
`buildBackendTable` transfers the deferred check only for `$files`.
Consequently, a catalog that allows metadata lookup but denies `authTableQuery`
still returns file paths and record counts through `t$file_indexes`. Reject
this system table for query-auth tables or preserve its query-authorization
contract before removing the loader, and add a denied-query case for it.
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonScanParams.java:
##########
@@ -127,11 +126,11 @@ public final class PaimonScanParams {
private static final Set<String> PAIMON_READER_SYSTEM_TABLES =
ImmutableSet.of(
"audit_log", "binlog", "row_tracking");
- // Not a capability the user selects but a property of where the snapshot
is resolved: these five
+ // Not a capability the user selects but a property of where the snapshot
is resolved: these tables
// pick it inside the BE reader rather than reading what the FE planned.
See
// resolvesSnapshotOnBackend for the per-table call chain.
private static final Set<String> BACKEND_SNAPSHOT_SYSTEM_TABLES =
ImmutableSet.of(
- "files", "partitions", "manifests", "statistics", "table_indexes");
+ "files", "file_indexes", "partitions", "manifests", "statistics",
"table_indexes");
Review Comment:
[P2] Preserve the file_indexes pin in the JNI source payload
Adding this entry pins the wrapper returned by `tableForBackend`, but the
normal scan also sends `doris.serialized-system-source` from
`prepareSystemSource` in `getScanNodeProperties`. That second source never
receives `pinCatalogSnapshot`.
`PaimonJniScanner.applyBackendManifestParallelism` always rebuilds from it when
present, replacing the pinned wrapper. With snapshots 1 and 2 on disk and a
REST catalog publishing only 1, `t$file_indexes` therefore enumerates snapshot
2's indexes. Pin the transported source as well, and exercise the actual
scan-properties/JNI reconstruction path; the new test reads `tableForBackend`
directly and misses this replacement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
