Re: Flink Custom SourceFunction and SinkFunction

2019-03-04 Thread Piotr Nowojski
Hi, I couldn’t find any references to your question neither I haven’t seen such use case, but: Re 1. It looks like it could work Re 2. It should work as well, but just try to use StreamingFileSink Re 3. For custom source/sink function, if you do not care data processing guarantees it’s

Flink Custom SourceFunction and SinkFunction

2019-03-01 Thread Siew Wai Yow
Hi guys, I have question regarding to the title that need your expertise, 1. I need to build a SFTP SourceFunction, may I know if hadoop SFTPFileSystem suitable? 2. I need to build a SFTP SinkFunction as well, may I know if per-defined HDFS rolling file sink accept SFTP connection since