Re: Filesystem as a stream source in Table/SQL API

2020-11-22 Thread Jark Wu
Sure. I have created one: https://issues.apache.org/jira/browse/FLINK-20286 On Mon, 23 Nov 2020 at 12:19, eef hhj wrote: > Hi Jark, > > Thank you for your helpful and quick response, I will try the hive > connector solution. It will be great if you can point out the link tracking > the filesyste

Re: Filesystem as a stream source in Table/SQL API

2020-11-22 Thread eef hhj
Hi Jark, Thank you for your helpful and quick response, I will try the hive connector solution. It will be great if you can point out the link tracking the filesystem streaming data source. On Mon, Nov 23, 2020 at 10:33 AM Jark Wu wrote: > Hi Kai, > > Streaming filesystem source is not supporte

Re: Filesystem as a stream source in Table/SQL API

2020-11-22 Thread Jark Wu
Hi Kai, Streaming filesystem source is not supported yet in TableAPI/SQL. This is on the roadmap and there are some problems that need to be fixed. As a workaround, you can use Hive connector to reading files continuously on filesystems [1]. Best, Jark [1]: https://ci.apache.org/projects/flink/f

Re: Filesystem as a stream source in Table/SQL API

2020-11-22 Thread Xingbo Huang
Hi Kai, I took a look at the implementation of the filesystem connector. It will decide which files to read at startup and won't change during running. If you want to need this function, you may need to customize a new connector. Best, Xingbo eef hhj 于2020年11月21日周六 下午2:38写道: > Hi, > > I'm faci