Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r228669149
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -274,11 +275,20 @@ private[history] class FsHistoryProvider(conf:
SparkConf, clock: Clock)
pool.scheduleWithFixedDelay(
getRunner(() => checkForLogs()), 0, UPDATE_INTERVAL_S,
TimeUnit.SECONDS)
- if (conf.getBoolean("spark.history.fs.cleaner.enabled", false)) {
+ if (conf.get(CLEANER_ENABLED)) {
// A task that periodically cleans event logs on disk.
pool.scheduleWithFixedDelay(
getRunner(() => cleanLogs()), 0, CLEAN_INTERVAL_S,
TimeUnit.SECONDS)
}
+
+ conf.get(DRIVER_LOG_DFS_DIR).foreach { _ =>
--- End diff --
`if (conf.contains(DRIVER_LOG_DFS_DIR) &&
conf.get(DRIVER_LOG_CLEANER_ENABLED))`
Don't use `foreach` as a boolean check.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
