HeartSaVioR commented on issue #26416: [WIP][SPARK-29779][CORE] Compact old event log files and cleanup URL: https://github.com/apache/spark/pull/26416#issuecomment-551340326 So the patch implements "alternative" of design on second part, since we've found some major issues on snapshotting. (all models being participated in snapshot should guarantee compatibility, which heavily restricts (SQL)AppStatusListener' modification) We've discussed to find an answer in this doc but anything doesn't seem to be easy. https://docs.google.com/document/d/1LbPflpEyrh9jDunwmDnrTjtMybAXcGimy36AWcUQsq8/edit And I found a new idea with minor (I hope) downside - this is the implementation of the idea. The idea is described on description of this PR. If you're referring design doc, please consider this patch just overwrites the second part (compaction via snapshotting and cleanup event log files).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
