github-actions[bot] commented on code in PR #66913:
URL: https://github.com/apache/doris/pull/66913#discussion_r3903765605
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergExternalMetaCache.java:
##########
@@ -133,10 +147,34 @@ IcebergSnapshotEntryKey.class,
IcebergSnapshotCacheValue.class, defaultEntryCach
public Table getIcebergTable(ExternalTable dorisTable) {
NameMapping nameMapping = dorisTable.getOrBuildNameMapping();
- return
tableEntry.get(nameMapping.getCtlId()).get(nameMapping).getIcebergTable();
+ IcebergTableCacheValue.Lease lease = statementLease(nameMapping);
+ if (lease != null) {
+ return lease.getIcebergTable();
+ }
+ throw new IllegalStateException("Iceberg table access outside a
statement must use a scoped borrower");
Review Comment:
[P1] Keep no-context full-schema projection within one lease
Without a `StatementContext`, `getSnapshotCache()` deliberately returns
`withoutRetainedTable()`. `getFullSchema()` then calls
`snapshotTable.orElseGet(this::getIcebergTable)` while appending row-lineage
columns, so `StatisticsUtil.findColumn()` -> `ExternalTable.getColumn()`
deterministically throws here. It also derives schema via a separate bounded
borrow after the snapshot borrow closes, allowing a reset to splice the
snapshot/schema ID with another table generation. Perform snapshot, schema
conversion, and row-lineage derivation under one bounded `withIcebergTable`
projection, and add no-`ConnectContext` `getFullSchema`/`getColumn` coverage.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/source/IcebergScanNode.java:
##########
@@ -307,12 +309,16 @@ protected void doInitialize() throws UserException {
// These tables are always readable regardless of format
version
formatVersion = MIN_DELETE_FILE_SUPPORT_VERSION;
}
- preExecutionAuthenticator =
source.getCatalog().getExecutionAuthenticator();
- storagePropertiesMap =
VendedCredentialsFactory.getStoragePropertiesMapWithVendedCredentials(
-
source.getCatalog().getCatalogProperty().getMetastoreProperties(),
-
source.getCatalog().getCatalogProperty().getStoragePropertiesMap(),
- icebergTable
- );
+ if (runtimeContext == null) {
+ preExecutionAuthenticator =
source.getCatalog().getExecutionAuthenticator();
+ storagePropertiesMap =
VendedCredentialsFactory.getStoragePropertiesMapWithVendedCredentials(
+
source.getCatalog().getCatalogProperty().getMetastoreProperties(),
+
source.getCatalog().getCatalogProperty().getStoragePropertiesMap(),
icebergTable);
+ } else {
+ preExecutionAuthenticator = runtimeContext.getAuthenticator();
Review Comment:
[P1] Keep mapping flags with the frozen scan generation
This branch restores the frozen generation's authenticator and storage
state, but it leaves `enable_mapping_varbinary` and
`enable_mapping_timestamp_tz` on the mutable live catalog. The same snapshot
already captured both flags, yet partition metadata, schema/default/delete
conversion, and the two FE-BE parameters below all call inherited
`getEnableMapping*()`. If ALTER/reset flips either flag after the relation pins
G1, the scan describes G1 fields and partitions using G2 type/transport
semantics. Copy the snapshot's mapping flags into scan-local runtime state (or
include them in `IcebergRuntimeContext`) and use those values for every
conversion and Thrift parameter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
