Gabriel39 commented on code in PR #67904:
URL: https://github.com/apache/doris/pull/67904#discussion_r3999755198
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnectorMetadata.java:
##########
@@ -483,33 +484,34 @@ public ConnectorTableSchema getTableSchema(
return getTableSchema(session, handle);
}
Table table = loadTable(session, iceHandle);
- Schema schema;
- if (table.currentSnapshot() == null) {
- // Empty table: legacy getSchema falls back to the latest schema
(NEWEST_SCHEMA_ID path).
- schema = table.schema();
- } else {
- schema = table.schemas().get((int) snapshot.getSchemaId());
- if (schema == null) {
- // Defensive: a pinned id absent from table.schemas() (legacy
would NPE) -> latest.
- // INVARIANT: this SLOT-schema fallback MUST stay identical to
the DICT-schema fallback in
- // IcebergScanPlanProvider.pinnedSchema (same getSchemaId()
lookup + same silent -> table.schema()).
- // If the two diverge, the field-id dict names and the BE
scan-slot names resolve DIFFERENT
- // schemas -> BE children.at() std::out_of_range-SIGABRT on a
schema-evolved time-travel read
- // (reverify #65185 L16). Do not harden ONE side to throw
without the other.
- schema = table.schema();
- }
- }
- return buildTableSchema(iceHandle.getTableName(), table, schema, true);
+ Schema schema = resolvePinnedSchema(table, snapshot);
+ String specId =
snapshot.getProperties().get(PARTITION_SPEC_ID_PROPERTY);
+ PartitionSpec spec = specId == null ? table.spec() :
table.specs().get(Integer.parseInt(specId));
Review Comment:
Fixed in a56a252a07.
This is in scope: this PR introduced the cached partition-spec property, and
the new warm-cache drop/recreate test reproduced the null dereference with spec
1 retained while the replacement table only has spec 0.
The pinned schema assembly now falls back to the reloaded table's current
spec when the cached spec ID is absent, matching the existing missing-schema
fallback. The test retains the production latest-pin cache across an actual
drop/recreate, verifies the cached IDs are absent from the replacement's
history, and checks columns, column handles, partition columns, and the SHOW
partition clause. Historical spec isolation when the ID is still present
remains outside this change.
Validation: 801 related tests passed, one connectivity test skipped; final
FE build, connector packages, and Checkstyle passed.
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonScanPlanProvider.java:
##########
@@ -1195,6 +1222,10 @@ OptionalInt backendManifestParallelism(PaimonTableHandle
handle, Table scanTable
*/
// Package-private for direct unit testing (PaimonBackendBoundTableTest).
Table tableForBackend(PaimonTableHandle handle, Table scanTable) {
+ return withBoundSchemaAuthentication(handle, () ->
buildBackendTable(handle, scanTable));
Review Comment:
Fixed in a56a252a07.
This is in scope because the bound-schema restoration introduced by this PR
can perform schema-file IO at these existing later call sites.
Both backend manifest-parallelism calculation and
DORIS_SERIALIZED_SYSTEM_SOURCE preparation now use a shared helper that
performs runtimeSafeSystemSource inside the bound-schema authentication/TCCL
scope. The guarded real-FileIO test now exercises the full
getScanNodeProperties JNI path for a warm read-optimized OPTIONS handle after a
schema rename and checks that the serialized system source is emitted. The new
test failed on the previous code at an unauthenticated schema read; it also
checks that the caller's authentication and classloader state are restored.
Scope note: enabling the native schema dictionary also exposes
unauthenticated reads in resolveCurrentSchemaFields/buildSchemaEvolutionParam.
Both methods are unchanged from pre-PR ee4a91720f. The regression forces JNI to
isolate the newly introduced bound-source restoration from that historical
dictionary issue; this PR does not expand into that separate path.
Validation: 801 related tests passed, one connectivity test skipped; final
FE build, connector packages, and Checkstyle passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
