924060929 commented on code in PR #66007:
URL: https://github.com/apache/doris/pull/66007#discussion_r3687960029
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindSink.java:
##########
@@ -651,6 +651,27 @@ private Plan
bindTVFTableSink(MatchingContext<UnboundTVFTableSink<Plan>> ctx) {
Optional.empty(), Optional.empty(), projectWithCast);
}
+ /**
+ * Returns the schema of a write target without inheriting a snapshot
pinned by a source relation.
+ *
+ * <p>An INSERT may read an older version of the same connector table. The
no-arg external-table schema
+ * lookup consults that statement-level ambient pin, but a sink is not
that source reference and must bind
+ * against the latest write schema. Non-connector tables retain their
existing lookup.</p>
+ */
+ private static List<Column> sinkTargetFullSchema(TableIf table) {
+ if (table instanceof PluginDrivenExternalTable) {
+ return ((PluginDrivenExternalTable)
table).getFullSchema(Optional.empty());
+ }
+ return table.getFullSchema();
+ }
+
+ private static Column connectorSinkTargetColumn(PluginDrivenExternalTable
table, String name) {
+ return sinkTargetFullSchema(table).stream()
Review Comment:
**[P2] Resolve the latest target schema only once per sink bind**
`connectorSinkTargetColumn` calls `sinkTargetFullSchema(table)` for every
user column and every static-partition name, then linearly scans the returned
list. An explicit list of M columns over an N-column table therefore becomes
O(M×N), replacing the previous map lookup.
The more expensive case is row-level DML:
`PluginDrivenExternalTable.getFullSchema(Optional.empty())` passes through
`appendSyntheticWriteColumns`; while the hidden-column gate is open, that calls
`fetchSyntheticWriteColumns`, resolves the connector handle again, and the
Iceberg `getTableHandle` implementation executes `catalogOps.tableExists`. A
wide DML statement can consequently repeat a remote catalog existence probe per
target column.
Please load the latest target schema once at the start of sink binding,
build one case-insensitive name map, and pass that schema/map through
canonicalization, explicit-column selection, and static-partition
materialization. An invocation-count test should verify that one bind calls
`getFullSchema(Optional.empty())` only once regardless of the number of target
columns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
