Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r226456102
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2421,11 +2425,13 @@ class SparkContext(config: SparkConf) extends
Logging {
// the cluster manager to get an application ID (in case the cluster
manager provides one).
listenerBus.post(SparkListenerApplicationStart(appName,
Some(applicationId),
startTime, sparkUser, applicationAttemptId,
schedulerBackend.getDriverLogUrls))
+ _driverLogger.foreach(_.startSync(_hadoopConfiguration))
}
/** Post the application end event */
private def postApplicationEnd() {
listenerBus.post(SparkListenerApplicationEnd(System.currentTimeMillis))
+ _driverLogger.foreach(_.stop())
--- End diff --
I think it may be better call this in `stop()`. Stopping that logger is not
part of posting an event to the listener bus.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
