HeartSaVioR commented on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query URL: https://github.com/apache/spark/pull/22952#issuecomment-466214463 I may need to play with the case a bit, but my guess is if the new file has newer timestamp the file gets processed and overwrite old archived file when new file gets archived. Maybe we could simulate it in UT? This case I'm not sure which behavior we would want to have, and can exclude 5dcece0 if we are not happy with new behavior.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
