dongjoon-hyun commented on a change in pull request #35114:
URL: https://github.com/apache/spark/pull/35114#discussion_r779337519
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1486,7 +1486,46 @@ See the [configuration page](configuration.html) for
information on Spark config
</td>
<td>3.3.0</td>
</tr>
-
+<tr>
+ <td><code>spark.kubernetes.executor.rollInterval</code></td>
+ <td><code>0s</code></td>
+ <td>
+ Interval between executor roll operations. It's disabled by default with
`0s`.
+ </td>
+ <td>3.3.0</td>
+</tr>
+<tr>
+
<td><code>spark.kubernetes.executor.minTasksPerExecutorBeforeRolling</code></td>
+ <td><code>0</code></td>
+ <td>
+ The minimum number of tasks per executor before rolling.
+ Spark will not roll executors whose total number of tasks is smaller
+ than this configuration. The default value is zero.
+ </td>
+ <td>3.3.0</td>
+</tr>
+<tr>
+ <td><code>spark.kubernetes.executor.rollPolicy</code></td>
+ <td><code>OUTLIER</code></td>
+ <td>
+ Executor roll policy: Valid values are ID, ADD_TIME, TOTAL_GC_TIME,
+ TOTAL_DURATION, FAILED_TASKS, and OUTLIER (default).
+ When executor roll happens, Spark uses this policy to choose
+ an executor and decommission it. The built-in policies are based on
executor summary
+ and newly started executors are protected by
spark.kubernetes.executor.minTasksPerExecutorBeforeRolling.
+ ID policy chooses an executor with the smallest executor ID.
+ ADD_TIME policy chooses an executor with the smallest add-time.
Review comment:
It's the time when a executor was added~
https://github.com/apache/spark/blob/3e4de180ba1cb60e4527f5bd10a88eac414bf212/core/src/main/scala/org/apache/spark/status/api/v1/api.scala#L120
Spark creates executor pods in an order of sequential ID, but the pod
creation times depends on the node status.
Some nodes need to download the pod image. So, ID order and ADD_TIME order
is different in the production.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
