HeartSaVioR commented on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query URL: https://github.com/apache/spark/pull/22952#issuecomment-466877530 Just rolled back the change from 9e45876 to avoid initiating more discussion on removing archived entities. We can have this as follow-up issue, and once we agree upon removing archived entities from metadata as well as SeenFilesMap, we can leverage new branch (`SPARK-20568-FOLLOWUP-remove-obsolete-entries-from-metadata-and-seenfilesmap`). 9e45876 is the only difference for now. https://github.com/apache/spark/compare/master...HeartSaVioR:SPARK-20568-FOLLOWUP-remove-obsolete-entries-from-metadata-and-seenfilesmap
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
