github-actions[bot] commented on code in PR #66247:
URL: https://github.com/apache/doris/pull/66247#discussion_r3682529945
##########
fe/fe-connector/fe-connector-spi/src/main/java/org/apache/doris/connector/spi/ConnectorProvider.java:
##########
@@ -105,6 +106,18 @@ default void validateProperties(Map<String, String>
properties) {
// no-op by default
}
+ /**
+ * Validates an ALTER CATALOG candidate without publishing it to the live
catalog.
+ * Connectors with legacy-property compatibility rules may override this
method.
+ */
+ default void validatePropertiesForUpdate(
Review Comment:
[P1] Bump the connector plugin API major for this SPI addition
This adds a method to the explicitly frozen connector surface and updates
its baseline, but `connector.plugin.api.version` remains `1.0`.
`ConnectorPluginSurfaceTest` and the POM define every surface change—including
a default method—as a major change, while `ApiVersionGate` compares majors
only. As written, artifacts with the old and new surfaces are both admitted as
API 1, so an older Paimon provider can take this default ALTER validator
instead of the compatibility-aware override. Please bump the API to `2.0` and
update the version-gate expectations with this surface change.
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonScanPlanProvider.java:
##########
@@ -311,20 +312,45 @@ Table resolveTable(PaimonTableHandle paimonHandle) {
Table resolveScanTable(PaimonTableHandle paimonHandle) {
Table table = resolveTable(paimonHandle);
Map<String, String> scanOptions = paimonHandle.getScanOptions();
+ Table finalTable = table;
if (scanOptions != null && !scanOptions.isEmpty()) {
if (PaimonScanParams.isOptionsPin(scanOptions)) {
// An @options pin owns the whole scan-startup state:
applyOptions strips the internal
// markers and nulls out the absent members of paimon's
inherited read-state family, so a
// scan.mode / tag persisted on the base table cannot leak
into this relation's read.
- return PaimonScanParams.applyOptions(table, scanOptions);
+ finalTable = PaimonScanParams.applyOptions(table, scanOptions);
+ } else {
+ // FIX-INCR-SCAN-RESET: for an @incr read, reapply legacy's
null reset of
+ // scan.snapshot-id/scan.mode here (the single Table.copy
chokepoint shared by both the
+ // native/JNI scan path and the JNI serialized-table path) so
a stale persisted pin on the
+ // base table cannot hijack incremental-between.
Non-incremental pins pass through unchanged.
+ finalTable =
table.copy(PaimonIncrementalScanParams.applyResetsIfIncremental(scanOptions));
+ }
+ }
+ // This is the last common boundary before planning and serialization.
Validate only after
+ // relation and incremental copies establish relation > catalog >
physical precedence.
+ PaimonReaderOptions.validateEffectiveTable(finalTable);
+ validateHiddenSystemDataTable(paimonHandle, scanOptions);
+ return finalTable;
+ }
+
+ private void validateHiddenSystemDataTable(PaimonTableHandle handle,
Map<String, String> scanOptions) {
+ if (!handle.isSystemTable()) {
+ return;
+ }
+ try {
+ Table dataTable = catalogOps.getTable(
Review Comment:
[P1] Validate the data handle embedded in this system wrapper
This newly added guard reloads `db.table` and validates generation B, but
`getSysTableHandle` has already pinned the system wrapper to generation A and
the scan returns/plans that original wrapper. If the catalog cache changes
between those boundaries, unsafe A plus safe B passes (and the reverse falsely
rejects); the metadata statistics twin repeats the same reload. This is a
downstream connector reintroduction of the exact-handle invariant discussed in
r3679793476, after that thread fixed FE-core wrapper construction. Please
retain/expose the exact `FileStoreTable` child, apply relation options to it,
and validate that same child through scan and statistics, with
divergent-generation end-to-end tests.
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonConnectorMetadata.java:
##########
@@ -465,6 +467,12 @@ public Optional<ConnectorTableHandle>
getSysTableHandle(ConnectorSession session
Table sysTable;
try {
sysTable = context.executeAuthenticated(() -> {
+ Table source = base.getPaimonTable();
Review Comment:
[P1] Keep the `$ro` schema dictionary on the pinned generation
This now builds the read-optimized wrapper from source generation A, but the
handle retains only the wrapper. Later `getScanNodeProperties` serializes/plans
that wrapper while `resolveSchemaDictTable` calls `reloadBaseTable(handle)` and
builds field-ID history from a fresh generation B. After a cache refresh or
schema rename/drop, requested-name resolution can fail or the BE can receive
A's splits with B's schema dictionary. Please retain the exact source
`FileStoreTable` and use it for both wrapper construction and the `$ro`
dictionary, with a divergent-generation test through `getScanNodeProperties`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
