abhishekrb19 commented on code in PR #19203:
URL: https://github.com/apache/druid/pull/19203#discussion_r2996045926
##########
indexing-service/src/test/java/org/apache/druid/indexing/overlord/TaskQueueTest.java:
##########
@@ -751,6 +753,47 @@ public void
testTaskWaitingTimeMetricEmittedForMultipleTasks() throws Exception
serviceEmitter.verifyEmitted("task/run/time", 3);
}
+ @Test
+ public void testTaskSubmissionToTaskRunnerBasedOnPriority() throws Exception
+ {
+ final RecordingTaskRunner recordingRunner = new
RecordingTaskRunner(serviceEmitter);
+ final TaskQueue priorityQueue = new TaskQueue(
+ new TaskLockConfig(),
+ new TaskQueueConfig(10, null, null, null, null, null),
+ new DefaultTaskConfig(),
+ getTaskStorage(),
+ recordingRunner,
+ actionClientFactory,
+ getLockbox(),
+ serviceEmitter,
+ getObjectMapper(),
+ new NoopTaskContextEnricher()
+ );
+ priorityQueue.setActive(true);
+
+ final NoopTask lowPriority1 = NoopTask.ofPriority(1);
+ final NoopTask lowPriority2 = NoopTask.ofPriority(10);
+ final NoopTask medPriority = NoopTask.ofPriority(50);
+ final NoopTask highPriority1 = NoopTask.ofPriority(90);
+ final NoopTask highPriority2 = NoopTask.ofPriority(100);
+
+ priorityQueue.add(lowPriority1);
+ priorityQueue.add(medPriority);
+ priorityQueue.add(lowPriority2);
+ priorityQueue.add(highPriority2);
+ priorityQueue.add(highPriority1);
+
+ priorityQueue.manageQueuedTasks();
+
+ final List<String> submitted = recordingRunner.getSubmittedTaskIds();
+ Assert.assertEquals(5, submitted.size());
+ Assert.assertEquals(highPriority2.getId(), submitted.get(0));
+ Assert.assertEquals(highPriority1.getId(), submitted.get(1));
Review Comment:
I think this would introduce test flakiness since iteration over
`ConcurrentHashMap` does not preserve insertion order and the comparator
doesn't have a tie breaker condition. Should we relax this check to assert that
the set contains either highPriority2 or highPriority1? Ditto for the
low-priority ones.
##########
indexing-service/src/main/java/org/apache/druid/indexing/overlord/TaskQueue.java:
##########
@@ -417,8 +417,14 @@ private void startPendingTasksOnRunner()
log.info("Notified task runner to clean up [%,d] tasks with IDs[%s].",
unknownTaskIds.size(), unknownTaskIds);
// Attain futures for all active tasks (assuming they are ready to run).
- // Copy tasks list, as notifyStatus may modify it.
- for (final String queuedTaskId : List.copyOf(activeTasks.keySet())) {
+ // Copy tasks list, as notifyStatus may modify it. Sort by priority
(highest first) so that
Review Comment:
The comment needs an update since there's no more copy?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
