walterddr commented on code in PR #10756:
URL: https://github.com/apache/pinot/pull/10756#discussion_r1195433083
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/service/dispatch/QueryDispatcher.java:
##########
@@ -212,7 +212,8 @@ public static DistributedStagePlan
constructDistributedStagePlan(DispatchableSub
int stageId, VirtualServerAddress serverAddress) {
return new DistributedStagePlan(stageId, serverAddress,
dispatchableSubPlan.getQueryStageList().get(stageId).getPlanFragment().getFragmentRoot(),
-
dispatchableSubPlan.getQueryStageList().get(stageId).toStageMetadata());
+
StageMetadata.from(dispatchableSubPlan.getQueryStageList().get(stageId)),
Review Comment:
e.g. intead of doing this in Dispatcher
```
_executorService.submit(() ->
client.submit(Worker.QueryRequest.newBuilder().setStagePlan(
QueryPlanSerDeUtils.serialize(
constructDistributedStagePlan(dispatchableSubPlan, finalStageId,
virtualServerAddress)))
.putMetadata(QueryConfig.KEY_OF_BROKER_REQUEST_ID,
String.valueOf(requestId))
.putMetadata(QueryConfig.KEY_OF_BROKER_REQUEST_TIMEOUT_MS,
String.valueOf(timeoutMs))
.putAllMetadata(queryOptions).build(), finalStageId,
queryServerInstance, deadline,
dispatchCallbacks::offer));
```
1. directly serialized the dispatchableSubPlan to proto format:
```
QueryPlanSerDeUtils.serialize(dispatchableSubPlan, finalStageId,
virtualServerAddress)
```
2. when deserialize, deserialize it into multiple distributed plan
```
List<DistributedStagePlan> QueryPlanSerDeUtils.deserialize(Worker.StagerPlan
stagePlan)
```
##########
pinot-common/src/main/proto/worker.proto:
##########
@@ -73,10 +73,10 @@ message StagePlan {
string virtualAddress = 2;
StageNode stageRoot = 3;
StageMetadata stageMetadata = 4;
+ WorkerMetadata workerMetadata = 5;
Review Comment:
this might not be the right abstraction for proto.
we want to send a single proto request from broker to server to launch all
workers, not sending 1 grpc request per worker which will be inefficient. let's
still keep the wire protocol as a list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
