Ngone51 commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-525982535 > We should just initialize all the time for startup of AppStatusListener. At least for now, KVStore is always empty when AppStatusListener startup, right ? Though, it would be fast to init entities from empty KVStore, but the logic itself doesn't make sense to me. I mean, what do we expect to get from the KVStore since we've already know it's empty ? The roadmap looks good to me. But some details about step 3 may still need full discussion. e.g. Do we only need snapshot for completed apps in SHS since snapshot for in-completed apps could be generated by live AppStatusListener ?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
