924060929 commented on code in PR #69806:
URL: https://github.com/apache/doris/pull/69806#discussion_r4420273084
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/RefreshManager.java:
##########
@@ -327,36 +327,40 @@ public void refreshTableInternal(ExternalDatabase db,
ExternalTable table, long
private void refreshTableInternal(ExternalDatabase db, ExternalTable
table, long updateTime,
boolean invalidateTableLookup) {
boolean invalidateDatabase = invalidateTableLookup &&
table.getCatalog().getLowerCaseTableNames() != 0;
- table.advanceMetadataRefreshGeneration();
- table.unsetObjectCreated();
- if (updateTime > 0) {
- table.setUpdateTime(updateTime);
- }
- // Connector metadata is the row-count source, so invalidate it before
engine and row-count caches.
+ table.beginMetadataRefresh();
Review Comment:
Fixed in b5e0824d289d. The database now owns a lookup-refresh generation and
active count. applyTableRefresh brackets both its hot-table and cold-table
branches, shared by commands and replay; observations on objects published
during this interval cannot authorize cleanup, and generation changes reject
reads crossing the interval. The outer finally releases the fence on failures,
and the count preserves it across overlapping refreshes. Ordinary metadata-only
writes do not acquire this database fence or reload names. Provider IO remains
outside catalog monitors. Deterministic tests pause after lookup invalidation,
publish a replacement, delay its absence, finish refresh and warm recreated
metadata, then verify that identity, schema and connector caches survive. Both
hot and cold table paths are covered, plus overlap and failure recovery.
Validation: 484 focused unit cases passed across 44 classes; FE build and
Checkstyle passed. Controlled providers, not a live external-service
integration run.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/plugin/PluginDrivenSysExternalTable.java:
##########
@@ -96,13 +96,17 @@ private static long generateSysTableId(long sourceTableId,
String sysName) {
@Override
public Optional<ConnectorTableHandle> resolveConnectorTableHandle(
ConnectorSession session, ConnectorMetadata metadata) {
- String dbName = db != null ? db.getRemoteName() : "";
- Optional<ConnectorTableHandle> baseHandle =
- metadata.getTableHandle(session, dbName,
sourceTable.getRemoteName());
+ MetadataRead read = observeConnectorMetadata(session, metadata);
+ Optional<ConnectorTableHandle> baseHandle =
sourceTable.resolveConnectorTableHandle(session, metadata);
if (!baseHandle.isPresent()) {
return Optional.empty();
}
- return metadata.getSysTableHandle(session, baseHandle.get(),
sysTableName);
+ return read.execute(() -> metadata.getSysTableHandle(session,
baseHandle.get(), sysTableName));
Review Comment:
Fixed the pinned-schema bypass in b5e0824d289d. schemaCacheValueAt now
captures the source MetadataRead before resolving the system handle and retains
it around getTableSchema(session, sysHandle, snapshot). Tests exercise
getFullSchemaAt with delayed source absence and with same-name recreation; only
the former cleans the shared source identity.
The lifecycle suggestion needs a distinction: this system relation borrows
the live statement scope, as documented by its
buildSchemaSession/closeSchemaSession overrides and
twoAliasSchemasBorrowOneLiveStatementScopeWithoutClosingIt. Closing it here
would invalidate handles/pins needed by later aliases and scans. The pinned
path now invokes those existing lifecycle hooks in try/finally, preserving
statement-owned closure rather than prematurely closing a borrowed scope.
Validation: 484 focused unit cases passed across 44 classes; FE build and
Checkstyle passed. Controlled providers, not a live external-service
integration run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
