englefly commented on code in PR #67518:
URL: https://github.com/apache/doris/pull/67518#discussion_r4032527454
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisExternalCatalog.java:
##########
@@ -168,6 +178,72 @@ public boolean useArrowFlight() {
"true"));
}
+ /**
+ * Returns the remote olap table behind the given table, or null if the
table is not a
+ * remote doris table bound in the virtual cluster mode
(use_arrow_flight=false), which
+ * binds a RemoteOlapTable directly. The arrow flight mode binds a
+ * RemoteDorisExternalTable, which is rejected by MaterializeProbeVisitor,
so topn lazy
+ * materialization never runs for it.
+ */
+ public static RemoteOlapTable getRemoteOlapTable(TableIf table) {
+ return table instanceof RemoteOlapTable ? (RemoteOlapTable) table :
null;
+ }
+
+ /**
+ * Whether any remote backend id collides with a local backend id, or two
remote tables
+ * (e.g. from different remote catalogs) collide with each other. Backend
ids of clusters
+ * are independently allocated; on collision the second phase fetch cannot
distinguish the
+ * id spaces and would route rows to a wrong backend, so topn lazy
materialization must
+ * be skipped.
+ *
+ * <p>Callers must pass one table per remote catalog (see
LazyMaterializeTopN): tables of
+ * the same catalog share the same backend map and would be falsely
reported as a
+ * remote-vs-remote conflict.
+ */
+ public static boolean
hasRemoteBackendIdConflict(Collection<RemoteOlapTable> remoteTables) {
+ // Align with the address book domain of
MaterializationNode.initNodeInfo: only alive,
+ // query-available backends of the selected compute group are routed
to, so ids outside
+ // that domain cannot collide at runtime and must not disable the
optimization.
+ // Fall back to the whole cluster's ids when the compute group cannot
be resolved
+ // (e.g. unit tests without a session); the wider set only skips the
optimization
+ // more often and never breaks correctness.
+ Set<Long> localBackendIds;
+ try {
+ BeSelectionPolicy policy = new BeSelectionPolicy.Builder()
+ .needQueryAvailable()
Review Comment:
如果本地集群 的一个BE id 是123, 但当时的状态不是queryAvaliable, 需要查询的remote be id也是123,
那么也会出现第二阶段获取数据失败
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
