HeartSaVioR commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-525987396 > That's true, but KVStore still lack some necessary info to recover live entities due to live entities don't always write all fields out. Good point, thanks for pointing out. Once we want to reconstruct AppStatusListener via restoring KVStore, there should not be any difference between live entities and stored entities. If there's any difference, it should be fixed. Ideally AppStatusListener should not be stateful except entities in underlying KVStore. We could leverage some caching in performance perspective, but source of truth should be KVStore, not AppStatusListener.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
