walterddr commented on code in PR #10791:
URL: https://github.com/apache/pinot/pull/10791#discussion_r1201465552
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/serde/QueryPlanSerDeUtils.java:
##########
@@ -51,12 +53,23 @@ public static DistributedStagePlan
deserialize(Worker.StagePlan stagePlan) {
return distributedStagePlan;
}
- public static Worker.StagePlan serialize(DistributedStagePlan
distributedStagePlan) {
+ public static List<DistributedStagePlan> deserialize(Worker.QueryRequest
request) {
+ List<DistributedStagePlan> distributedStagePlans = new ArrayList<>();
+ for (Worker.StagePlan stagePlan : request.getStagePlanList()) {
+ distributedStagePlans.add(deserialize(stagePlan));
+ }
+ return distributedStagePlans;
+ }
+
+ public static Worker.StagePlan serialize(DispatchableSubPlan
dispatchableSubPlan, int stageId,
+ VirtualServerAddress serverAddress) {
Review Comment:
@ankitsultana i remember you mentioned to me that dispatching 1-worker per
request is extremely heavy in GRPC payload gen.
does this approach help?
e.g.
1. the dispatchable plan with the planFragment(stage) + all the worker
metadata will be **serialized** together in 1 GRPC request (1 request for all
worker)
2. the request will be **deserialized** into a list of distributed stage
plans (e.g. 1 plan per worker)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
