HeartSaVioR commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-654866771
That said, probably we also haven't check how much hybrid KV store will help on the smaller app event log (like 100mb or even smaller in uncompressed). If that turns out to not help enough (either hybrid KV store only reduces a couple of seconds or replaying is fast enough without hybrid KV store), we can even exclude them on hybrid KV store path via having a lower bound (considering compression), as I assume the overhead of hybrid KV store is not that tiny. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
