samarthjain commented on a change in pull request #12097:
URL: https://github.com/apache/druid/pull/12097#discussion_r818168065
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/overlord/TaskRunner.java
##########
@@ -88,11 +89,19 @@
*/
void shutdown(String taskid, String reason);
- default void shutdown(String taskid, String reasonFormat, Object... args)
+ default void shutdown(String taskid, String reasonFormat, Object...
reasonArgs)
{
- shutdown(taskid, StringUtils.format(reasonFormat, args));
+ // only calculate the 'reason' string for debug level logging
+ // in large clusters the 'reasonArgs' may be very large / expensive if it
includes a list of tasks
+ String reason = (getLogger().isDebugEnabled()) ?
StringUtils.format(reasonFormat, reasonArgs) : "debug log disabled";
Review comment:
Looking at the callers, I see only TaskQueue#shutdown() that is passing
in the task ids. So, I propose we modify code in TaskQueue#manageInternal() to
something like this:
```
// Kill tasks that shouldn't be running
final Set<String> knownTaskIds = tasks
.stream()
.map(Task::getId)
.collect(Collectors.toSet());
final Set<String> tasksToKill =
Sets.difference(runnerTaskFutures.keySet(), knownTaskIds);
if (!tasksToKill.isEmpty()) {
log.info("Asking taskRunner to clean up %,d tasks.",
tasksToKill.size());
// On large installations running several thousands of tasks,
// concatenating the list of known task ids can be compupationally
expensive.
boolean logKnownTaskIds = log.isDebugEnabled();
String reason = logKnownTaskIds
? String.format("Task is not in knownTaskIds[%s]",
knownTaskIds)
: "Task is not in knownTaskIds";
for (final String taskId : tasksToKill) {
try {
taskRunner.shutdown(
taskId, reason
);
}
catch (Exception e) {
log.warn(e, "TaskRunner failed to clean up task: %s", taskId);
}
}
}
```
If we do this, then you won't have to add a new `getLogger()` method in
`TaskRunner` interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
