uncleGen edited a comment on issue #24128: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files URL: https://github.com/apache/spark/pull/24128#issuecomment-558077684 There are many end-users facing this problem. So I give +1 to resolve this problem. But the ttl option can just resolve the problems of some scenarios. This dose not resolve scenarios where end-users does not set ttl, or just want to read files from earliest offset with ttl set.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
