suneet-s commented on a change in pull request #12228:
URL: https://github.com/apache/druid/pull/12228#discussion_r800222182
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/overlord/http/OverlordResource.java
##########
@@ -422,6 +428,69 @@ public Response getWorkerConfig()
return Response.ok(workerConfigRef.get()).build();
}
+ /**
+ * Gets the total worker capacity of varies states of the cluster.
+ */
+ @GET
+ @Path("/totalWorkerCapacity")
+ @Produces(MediaType.APPLICATION_JSON)
+ @ResourceFilters(ConfigResourceFilter.class)
+ public Response getTotalWorkerCapacity()
+ {
+ // Calculate current cluster capacity
+ int currentCapacity;
+ Optional<TaskRunner> taskRunnerOptional = taskMaster.getTaskRunner();
+ if (!taskRunnerOptional.isPresent()) {
+ // Cannot serve call as not leader
+ return Response.status(Response.Status.SERVICE_UNAVAILABLE).build();
Review comment:
nit: log line to help with debugging
```suggestion
log.info("Can not calculate total worker capacity as TaskRunner is not
present.");
return Response.status(Response.Status.SERVICE_UNAVAILABLE).build();
```
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/overlord/http/OverlordResource.java
##########
@@ -422,6 +428,69 @@ public Response getWorkerConfig()
return Response.ok(workerConfigRef.get()).build();
}
+ /**
+ * Gets the total worker capacity of varies states of the cluster.
+ */
+ @GET
+ @Path("/totalWorkerCapacity")
+ @Produces(MediaType.APPLICATION_JSON)
+ @ResourceFilters(ConfigResourceFilter.class)
+ public Response getTotalWorkerCapacity()
+ {
+ // Calculate current cluster capacity
+ int currentCapacity;
+ Optional<TaskRunner> taskRunnerOptional = taskMaster.getTaskRunner();
+ if (!taskRunnerOptional.isPresent()) {
+ // Cannot serve call as not leader
+ return Response.status(Response.Status.SERVICE_UNAVAILABLE).build();
+ }
+ TaskRunner taskRunner = taskRunnerOptional.get();
+ Collection<ImmutableWorkerInfo> workers;
+ if (taskRunner instanceof WorkerTaskRunner) {
+ workers = ((WorkerTaskRunner) taskRunner).getWorkers();
+ currentCapacity = workers.stream().mapToInt(workerInfo ->
workerInfo.getWorker().getCapacity()).sum();
+ } else {
+ log.debug(
+ "Cannot calculate capacity as task runner [%s] of type [%s] does not
support listing workers",
+ taskRunner,
+ taskRunner.getClass().getName()
+ );
+ workers = ImmutableList.of();
+ currentCapacity = -1;
+ }
+
+ // Calculate maximum capacity with auto scale
+ int maximumCapacity;
+ if (workerConfigRef == null) {
+ workerConfigRef = configManager.watch(WorkerBehaviorConfig.CONFIG_KEY,
WorkerBehaviorConfig.class);
+ }
+ WorkerBehaviorConfig workerBehaviorConfig = workerConfigRef.get();
+ if (workerBehaviorConfig == null) {
+ // Auto scale not setup
+ log.debug("Cannot calculate maximum worker capacity as worker behavior
config is not configured");
Review comment:
nit: similar comment to line 453 and other debug statements in this
funciton.
```suggestion
log.info("Cannot calculate maximum worker capacity as worker behavior
config is not configured");
```
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/overlord/http/OverlordResource.java
##########
@@ -422,6 +428,69 @@ public Response getWorkerConfig()
return Response.ok(workerConfigRef.get()).build();
}
+ /**
+ * Gets the total worker capacity of varies states of the cluster.
+ */
+ @GET
+ @Path("/totalWorkerCapacity")
+ @Produces(MediaType.APPLICATION_JSON)
+ @ResourceFilters(ConfigResourceFilter.class)
+ public Response getTotalWorkerCapacity()
+ {
+ // Calculate current cluster capacity
+ int currentCapacity;
+ Optional<TaskRunner> taskRunnerOptional = taskMaster.getTaskRunner();
+ if (!taskRunnerOptional.isPresent()) {
+ // Cannot serve call as not leader
+ return Response.status(Response.Status.SERVICE_UNAVAILABLE).build();
+ }
+ TaskRunner taskRunner = taskRunnerOptional.get();
+ Collection<ImmutableWorkerInfo> workers;
+ if (taskRunner instanceof WorkerTaskRunner) {
+ workers = ((WorkerTaskRunner) taskRunner).getWorkers();
+ currentCapacity = workers.stream().mapToInt(workerInfo ->
workerInfo.getWorker().getCapacity()).sum();
+ } else {
+ log.debug(
Review comment:
Why not info - this seems like useful information
```suggestion
log.info(
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
