englefly commented on code in PR #67518:
URL: https://github.com/apache/doris/pull/67518#discussion_r3977007401
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java:
##########
@@ -2758,14 +2758,34 @@ public PlanFragment
visitPhysicalWindow(PhysicalWindow<? extends Plan> physicalW
return inputPlanFragment;
}
+ /**
+ * Collects backends of the remote doris clusters referenced by the lazy
materialized
+ * relations. Rowids of remote tables are generated by the remote
cluster's BEs, so the
+ * second phase fetch must be able to reach them. Duplicate relations may
resolve to the
+ * same remote table (e.g. self join), so merge by backend id.
+ */
+ private List<Backend> collectRemoteBackends(PhysicalLazyMaterialize<?
extends Plan> materialize) {
+ Map<Long, Backend> mergedBackends = Maps.newHashMap();
Review Comment:
1. 为什么叫 mergedBackends, 没有看出这段代码有merge的语义
2. 内表是不是也可以通过catalog 拿到对应的backends? 如果可以, 那么这里可以拿到所有内外表的backends.
这样LazyMaterializeTopN.initNodeInfo 时就不用把所有的backends 都加入候选.
##########
fe/fe-core/src/main/java/org/apache/doris/planner/MaterializationNode.java:
##########
@@ -157,6 +169,16 @@ public void initNodeInfo() {
for (Backend backend :
policy.getCandidateBackends(computeGroup.getBackendList())) {
nodesInfo.addToNodes(new TNodeInfo(backend.getId(), 0,
backend.getHost(), backend.getBrpcPort()));
}
+ // Remote doris catalog backends. Skip the dead ones, as the local
policy above does:
+ // the remote meta cache may still advertise them and an unreachable
entry makes the
+ // second phase fetch fail eagerly. Id conflicts are rejected before
the plan rewrite
+ // (LazyMaterializeTopN), so no check here.
+ for (Backend backend : remoteBackends) {
+ if (!backend.isAlive()) {
Review Comment:
这里应该是queryAvialable.
##########
fe/fe-core/src/main/java/org/apache/doris/planner/MaterializationNode.java:
##########
@@ -157,6 +169,16 @@ public void initNodeInfo() {
for (Backend backend :
policy.getCandidateBackends(computeGroup.getBackendList())) {
Review Comment:
我理解这个pr的目的就是将remoteBackends 也加入到nodesInfo里.
所以我建议
"for (Backend backend :
policy.getCandidateBackends(computeGroup.getBackendList()))"
改为(伪代码)
for (Backend backend :
policy.getCandidateBackends(computeGroup.getBackendList()
+collectRemoteBackends(materialize))).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
