cryptoe commented on code in PR #18033:
URL: https://github.com/apache/druid/pull/18033#discussion_r2542044293
##########
services/src/main/java/org/apache/druid/cli/CliOverlord.java:
##########
@@ -458,6 +455,47 @@ private void configureOverlordHelpers(Binder binder)
dutyBinder.addBinding().to(TaskLogAutoCleaner.class);
dutyBinder.addBinding().to(UnusedSegmentsKiller.class).in(LazySingleton.class);
}
+
+ /**
+ * Configures Overlord-specific web resources and QoS filtering.
+ * This method performs two main tasks:
+ * <ol>
+ * <li>Registers Jersey resources for Overlord REST endpoints</li>
+ * <li>Configures QoS (Quality of Service) filtering for request
limiting</li>
+ * </ol>
+ * <p>
+ * The Jersey resources handle the following endpoint paths:
+ * <ul>
+ * <li>/druid/indexer/v1 - Main indexing and task management
endpoints</li>
+ * <li>/druid-internal/v1 - Internal Overlord management
endpoints</li>
+ * </ul>
+ * Note to developers:
+ * Whenever adding new resources, please check if the root paths are
added in the QOS filtering.
+ */
+ private void configureOverlordWebResources(Binder binder)
+ {
+ Jerseys.addResource(binder, OverlordResource.class);
+ Jerseys.addResource(binder, SupervisorResource.class);
+ Jerseys.addResource(binder, HttpRemoteTaskRunnerResource.class);
+ Jerseys.addResource(binder, OverlordCompactionResource.class);
+ Jerseys.addResource(binder, OverlordDataSourcesResource.class);
+
+ // Add QoS filtering for overlord-specific endpoints if we have
enough threads
+ final int serverHttpNumThreads =
properties.containsKey("druid.server.http.numThreads")
+ ?
Integer.parseInt(properties.getProperty("druid.server.http.numThreads"))
+ :
ServerConfig.getDefaultNumThreads();
+
+ final int threadsForOverlordWork = serverHttpNumThreads -
THREADS_RESERVED_FOR_HEALTH_CHECK;
+
+ if (threadsForOverlordWork >=
ServerConfig.DEFAULT_MIN_QOS_THRESHOLD) {
Review Comment:
I was optimizing the patch for larger clusters, because that was the use
case I had at that time, but thinking more you are correct it should happen
across the board.
Added a new config:
"druid.indexer.server.maxConcurrentActions"
Went with "max(1, max(serverHttpNumThreads - 4, serverHttpNumThreads * 0.8))
so that large clusters are not impacted much.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
