ulysses-you commented on code in PR #57329: URL: https://github.com/apache/spark/pull/57329#discussion_r3655478609
########## docs/configuration.md: ########## @@ -3181,6 +3181,40 @@ Apart from these, the following properties are also available, and may be useful </td> <td>0.5.0</td> </tr> +<tr> + <td><code>spark.task.oomRetryCpusIncrement</code></td> + <td>0</td> + <td> + Number of additional CPUs to allocate for each retry of a task that failed due to + out-of-memory. Each OOM retry of a task gets + <code>spark.task.cpus + spark.task.oomRetryCpusIncrement * N</code> CPUs (N = number of Review Comment: Fixed. The doc now describes the base as the task's actual CPU request rather than always `spark.task.cpus`. `TaskSchedulerImpl` passes `ResourceProfile.getTaskCpusOrDefaultForProfile(prof, conf)` into `resourceOffer`, so the formula is now documented as `T + spark.task.oomRetryCpusIncrement * N`, where `T` is the ResourceProfile's task cpus (falling back to `spark.task.cpus` when the profile supplies no override). Updated both the `OOM_RETRY_CPUS_INCREMENT` config doc and `configuration.md`, and added a test with a stage task-profile of 4 cpus asserting the first retry needs `4 + 1 = 5`, not `2`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
