Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r230464353
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -1033,6 +1102,7 @@ private[history] case class FsHistoryProviderMetadata(
private[history] case class LogInfo(
@KVIndexParam logPath: String,
@KVIndexParam("lastProcessed") lastProcessed: Long,
+ logType: LogType.Value,
--- End diff --
What happens here when you have an existing listing database where this
field is not recorded?
Depending on what happens, you may need a default value here, or maybe
handling nulls somewhere else, or even changing the DB version so old data is
invalidated.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
