Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/19194#discussion_r141475205
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -512,6 +535,9 @@ private[spark] class TaskSetManager(
serializedTask)
}
} else {
+ if (runningTasks >= maxConcurrentTasks) {
+ logDebug("Already running max. no. of concurrent tasks.")
--- End diff --
can you update this msg to include the jobGroup and full config name?
also I think it would be good to have this log at the info level one time
(not on every `resourceOffer`, just the first time we hit the limit for this
TaskSet). I just see users hitting this inadvertently and so it would be good
to make it clear.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
