vanzin commented on a change in pull request #25648: [SPARK-28947][K8S] Status
logging not happens at an interval for liveness
URL: https://github.com/apache/spark/pull/25648#discussion_r329204734
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/LoggingPodStatusWatcher.scala
##########
@@ -92,13 +86,22 @@ private[k8s] class LoggingPodStatusWatcherImpl(
private def closeWatch(): Unit = {
podCompletedFuture.countDown()
- scheduler.shutdown()
+ maybeLoggingService.foreach(_.shutdown())
}
- override def awaitCompletion(): Unit = {
- podCompletedFuture.await()
- logInfo(pod.map { p =>
- s"Container final statuses:\n\n${containersDescription(p)}"
- }.getOrElse("No containers were found in the driver pod."))
+ override def watchOrStop(sId: String): Unit = maybeLoggingService match {
+ case Some(service) =>
+ logInfo(s"Waiting for application ${conf.appName} with submission ID
$sId to finish...")
+ val interval = conf.get(REPORT_INTERVAL)
+ val logShortStatus: Runnable = () => logInfo(s"Application status for
$appId (phase: $phase)")
+ service.scheduleAtFixedRate(logShortStatus, 0, interval,
TimeUnit.MILLISECONDS)
+ podCompletedFuture.await()
+ logInfo(pod.map { p =>
+ s"Container final statuses:\n\n${containersDescription(p)}"
Review comment:
since you touched this, the style here is a bit weird. you could do:
```
logInfo(
pod.map { p => s"Container final statuses:\n\n${containersDescription(p)}"
}
.getOrElse("No containers were found in the driver pod."))
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
