kfaraz commented on code in PR #16889:
URL: https://github.com/apache/druid/pull/16889#discussion_r1770952040
##########
indexing-service/src/main/java/org/apache/druid/indexing/overlord/ImmutableWorkerInfo.java:
##########
@@ -56,7 +60,7 @@ public class ImmutableWorkerInfo
public ImmutableWorkerInfo(
@JsonProperty("worker") Worker worker,
@JsonProperty("currCapacityUsed") int currCapacityUsed,
- @JsonProperty("currParallelIndexCapacityUsed") int
currParallelIndexCapacityUsed,
Review Comment:
Please do not remove existing fields.
##########
indexing-service/src/main/java/org/apache/druid/indexing/overlord/config/WorkerTaskRunnerConfig.java:
##########
@@ -20,32 +20,66 @@
package org.apache.druid.indexing.overlord.config;
import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import org.apache.druid.indexing.overlord.util.CustomJobTypeLimitsDeserializer;
+
+import java.util.HashMap;
+import java.util.Map;
public class WorkerTaskRunnerConfig
{
@JsonProperty
private String minWorkerVersion = "0";
@JsonProperty
- private double parallelIndexTaskSlotRatio = 1;
+ @JsonDeserialize(using = CustomJobTypeLimitsDeserializer.class)
+ private Map<String, Number> customJobTypeLimits = new HashMap<>();
Review Comment:
Why do we need a custom deserializer?
Can't we just have a map from String to Double?
##########
docs/configuration/index.md:
##########
@@ -1134,7 +1134,7 @@ The following configs only apply if the Overlord is
running in remote mode. For
|--------|-----------|-------|
|`druid.indexer.runner.taskAssignmentTimeout`|How long to wait after a task
has been assigned to a Middle Manager before throwing an error.|`PT5M`|
|`druid.indexer.runner.minWorkerVersion`|The minimum Middle Manager version to
send tasks to. The version number is a string. This affects the expected
behavior during certain operations like comparison against
`druid.worker.version`. Specifically, the version comparison follows dictionary
order. Use ISO8601 date format for the version to accommodate date comparisons.
|"0"|
-| `druid.indexer.runner.parallelIndexTaskSlotRatio`| The ratio of task slots
available for parallel indexing supervisor tasks per worker. The specified
value must be in the range `[0, 1]`. |1|
Review Comment:
We cannot change the existing property as there are already people using it.
Please add a new property that can take precedence over the existing
property.
You can call it `taskTypeToSlotRatio` or simply `taskSlotRatios`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
