HyukjinKwon commented on code in PR #59187:
URL: https://github.com/apache/spark/pull/59187#discussion_r4156355559
##########
docs/running-on-kubernetes.md:
##########
@@ -2118,6 +2124,8 @@ See the [configuration page](configuration.html) for
information on Spark config
at least two standard deviation from the mean in average task time,
total task time, total task GC time, and the number of failed tasks if
exists.
If there is no outlier, it works like TOTAL_DURATION policy.
+ OUTLIER_NO_FALLBACK policy picks an outlier using the OUTLIER policy above.
Review Comment:
**Nit (P3):** This entry defines `OUTLIER_NO_FALLBACK` by pointing to the
OUTLIER description above. That description (here and in the `Config.scala` doc
string) still lists only average task time, total task time, total task GC
time, and failed tasks. `ExecutorRollPlugin.outliersFromMultipleDimensions`
also checks peak JVM on-heap memory, peak JVM off-heap memory, total shuffle
write, and disk used. As written, users would expect an executor that is an
outlier only in, say, disk used never to be rolled under `OUTLIER_NO_FALLBACK`,
but it will be. Since this PR is syncing these docs with the enum, could you
extend the OUTLIER dimension list in both files to all eight metrics?
**Verification:**
- **Inspection:** The documented OUTLIER dimensions in both files match the
outliers(...) calls in ExecutorRollPlugin.outliersFromMultipleDimensions.
- **Compile:** The kubernetes core module still compiles, and the Scala
source lines stay within the 100-character limit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
