HeartSaVioR edited a comment on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query URL: https://github.com/apache/spark/pull/22952#issuecomment-466280389 I'll remove them from SeenFiles so that it can work consistently between running query and restarted query. Then we still don't worry much about timestamp in SeenFiles. Btw after change I'll squash the latest change into one commit and put them to separate branch (I'll comment here): I just realized adding more features doesn't help getting this reviewed and finally merged. That can be handled in separate JIRA issue.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
