Re: Production Readiness of File Source

2021-03-22 Thread Arvid Heise
Hi Chirag, we are also happy to take contributions to add features like the file deletion. Since the file source is already running well in our deployments, I'd also recommend it to take it as a base for your custom implementation (getting a source right is not trivial). On Thu, Mar 18, 2021 at

Re: Production Readiness of File Source

2021-03-18 Thread Dawid Wysakowicz
Hi, As for the issue of production readiness of the File Source(and other components) I'd recommend having a look at the PR, which is close to being merged where we express our opinion how we see certain components: https://github.com/apache/flink-web/pull/426 I am also cc'ing Stephan who wrote t

Production Readiness of File Source

2021-03-16 Thread Chirag Dewan
Hi, I am intending to use the File source for a production use case. I have a few use cases that are currently not supported like deleting a file once it's processed.  So I was wondering if we can use this in production or write my own implementation? Is there any recommendations around this? Th