ForVic commented on code in PR #52068:
URL: https://github.com/apache/spark/pull/52068#discussion_r2437512667
##########
core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:
##########
@@ -1058,6 +1062,26 @@ private[spark] class SparkSubmit extends Logging {
/** Throw a SparkException with the given error message. */
private def error(msg: String): Unit = throw new SparkException(msg)
+ /**
+ * Store the diagnostics using the SparkDiagnosticsSetter.
+ */
+ private def storeDiagnostics(args: SparkSubmitArguments, sparkConf:
SparkConf,
+ t: Throwable): Unit = {
+ // Swallow exceptions when storing diagnostics, this shouldn't fail the
application.
+ try {
+ if (args.master.startsWith("k8s") && !isShell(args.primaryResource)
+ && !isSqlShell(args.mainClass) && !isThriftServer(args.mainClass)) {
+ val diagnostics = SparkStringUtils.abbreviate(
+ org.apache.hadoop.util.StringUtils.stringifyException(t),
+ KUBERNETES_DIAGNOSTICS_MESSAGE_LIMIT_BYTES)
+ SparkSubmitUtils.
Review Comment:
Updated a little, but there's really not that much earlier we could check
this, since we won't need to set diagnostics for shell/sqlShell/thrift jobs and
I don't think we should change the API of `.supports()` to take
`SparkSubmitArguments` if we do want to check that logic in there since then
we'll have to change the visibility of the class just for this.
##########
core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:
##########
@@ -1058,6 +1062,26 @@ private[spark] class SparkSubmit extends Logging {
/** Throw a SparkException with the given error message. */
private def error(msg: String): Unit = throw new SparkException(msg)
+ /**
+ * Store the diagnostics using the SparkDiagnosticsSetter.
+ */
+ private def storeDiagnostics(args: SparkSubmitArguments, sparkConf:
SparkConf,
+ t: Throwable): Unit = {
+ // Swallow exceptions when storing diagnostics, this shouldn't fail the
application.
+ try {
+ if (args.master.startsWith("k8s") && !isShell(args.primaryResource)
+ && !isSqlShell(args.mainClass) && !isThriftServer(args.mainClass)) {
+ val diagnostics = SparkStringUtils.abbreviate(
Review Comment:
updated the diagnostics abbreviation and stringify, but I don't think we
should be pulling out the calls in the if statement, see my other comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
