gianm commented on code in PR #13271:
URL: https://github.com/apache/druid/pull/13271#discussion_r1010929285
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/indexing/IndexerWorkerContext.java:
##########
@@ -276,6 +268,62 @@ public Bouncer processorBouncer()
return injector.getInstance(Bouncer.class);
}
+ /**
+ * Number of workers that may run in the current JVM, including the current
worker.
+ */
+ private int computeNumWorkersInJvm()
+ {
+ if (toolbox.getAppenderatorsManager() instanceof
UnifiedIndexerAppenderatorsManager) {
+ // CliIndexer
+ return injector.getInstance(WorkerConfig.class).getCapacity();
+ } else {
+ // CliPeon
+ return 1;
+ }
+ }
+
+ /**
+ * Amount of memory available for our usage.
+ */
+ private long computeAvailableHeapMemory()
+ {
+ return Runtime.getRuntime().maxMemory() - computeTotalLookupFootprint();
+ }
+
+ /**
+ * Total estimated lookup footprint. Obtained by calling {@link
LookupExtractor#estimateHeapFootprint()} on
+ * all available lookups.
+ */
+ private long computeTotalLookupFootprint()
+ {
+ // Subtract memory taken up by lookups. Correctness of this operation
depends on lookups being loaded *before*
+ // we create this instance. Luckily, this is the typical mode of
operation, since by default
+ // druid.lookup.enableLookupSyncOnStartup = true.
Review Comment:
The Coordinator doesn't know how big the lookups are: it has the
definitions, but nothing more. So I think we are doing about the best we can
here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
