Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18887#discussion_r137940633
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -117,17 +122,37 @@ private[history] class FsHistoryProvider(conf:
SparkConf, clock: Clock)
// used for logging msgs (logs are re-scanned based on file size, rather
than modtime)
private val lastScanTime = new java.util.concurrent.atomic.AtomicLong(-1)
- // Mapping of application IDs to their metadata, in descending end time
order. Apps are inserted
- // into the map in order, so the LinkedHashMap maintains the correct
ordering.
- @volatile private var applications: mutable.LinkedHashMap[String,
FsApplicationHistoryInfo]
- = new mutable.LinkedHashMap()
+ private val pendingReplayTasksCount = new
java.util.concurrent.atomic.AtomicInteger(0)
- val fileToAppInfo = new ConcurrentHashMap[Path,
FsApplicationAttemptInfo]()
+ private val storePath = conf.get(LOCAL_STORE_DIR)
--- End diff --
Need a description on `storePath` or `LOCAL_STORE_DIR`, although we have
the one in `monitoring.md`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
