HeartSaVioR commented on issue #25943: [WIP][SPARK-29261][SQL][CORE] Support recover live entities from KVStore for (SQL)AppStatusListener URL: https://github.com/apache/spark/pull/25943#issuecomment-545668898 @Ngone51 Sorry I revisit too late. We've had some sort of a stuck moment since compatibility was raised as a major problem. I've discussed with @vanzin and @squito offline, and we all agreed about allowing no-compatibility on live entities/snapshots for now. (The goal of SPARK-28870 is changed to not allow deleting old event logs.) Based on the agreement, I feel we could just try to serialize live entities by itself (although we should have to recover references) instead of recreating them from KVStore entities which seem to be too complicated. Could you update your patch to simplify based on this agreement? Actually I have a POC commit to deal with this, and got a feedback, too verbose (like let jackson works for me). https://github.com/HeartSaVioR/spark/commit/946bca40f66aa5a1709f04e67a5d61b6b7351a6b I'd be really appreciate if you could take this up and simplify the code change. No need to retain authorship - it's yours if you are willing to take up. Apologize of the change of direction. I'm really sorry about this for leading wrong direction. Please let me know if you have any question or uncertain. Thanks in advance.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
