Ngone51 commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-533847127 > It's not even verified via POC, so I can't see any reason to adopt it right now. Oh, sorry. I was just discussing the possible integrate way in the following design and not ask you to make the change right now. > The thing is, caller logic should be different based on the support of sync/async snapshot. We'll have dump()/dumpAsync() and supportAsyncDump() and let caller deal with this. Sorry, I don't know this part well. Why do we need to implement sync and async dump separately ? And what's factor should caller depends on to decide which way it should choose ? If we have both sync and async way to dump KVStore, why don't we just choose async way to reduce latency ? I always think that KVStore should just implement dump() and hide the details from the caller, e.g. whether it's sync or async. And KVStore should always expose its interface to caller as a single view without considering it supports sync dump or async dump while using it.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
