viirya commented on issue #26696: [WIP][SPARK-18886][CORE] Only reset
scheduling delay timer if allocated slots are fully utilized
URL: https://github.com/apache/spark/pull/26696#issuecomment-561052772
> AFAIK, the issue of delay scheduling is: it has a timer per task set
manager, and the timer gets reset as soon as there is one task from this task
set manager gets scheduled on a preferred location.
> A stage may keep waiting for locality and not leverage available nodes in
the cluster, if its task duration is shorter than the locality wait time (3
seconds by default).
By adjusting the locality wait time, can't users make better trade off
between locality and cluster utilization? In this case, users can set a smaller
locality wait time.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
