Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/20138#discussion_r161099310
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -834,6 +906,9 @@ private[history] case class FsHistoryProviderMetadata(
private[history] case class LogInfo(
@KVIndexParam logPath: String,
+ @KVIndexParam("lastProcessed") lastProcessed: Long,
+ appId: Option[String],
--- End diff --
also a comment here explaining why appId is an Option, as that is unexpected
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
