I think that could be an interesting source. Two quick questions to move forward
- To keep the Flink code base from becoming too big (hard to maintain and test) we started working with Apache Bahir as a project dedicated to streaming connectors. Would that be a good target for the connector? - What are your thoughts on fault tolerance for that connector? On Mon, Sep 26, 2016 at 3:01 PM, shijinkui <shijin...@huawei.com> wrote: > Hi, all > > 1. In order to support end-to-end pushed source, I create FLINK-4630< > https://issues.apache.org/jira/browse/FLINK-4630>. I want to know whether > is this idea worth? > > ------------------- > When source stream get start, listen a provided tcp port, receive stream > data from user data source. > This netty tcp source is keepping alive and end-to-end, that is from > business system to flink worker directly. > > user app push -----> netty server source of Flink > > describe the source in detail below: > > 1. source run as a netty tcp server > 2. user provide a tcp port, if the port is in used, increace the port > number between 1024 to 65535. Source can parallel. > 3. callback the provided url to report the real port to listen > 4. user push streaming data to netty server, then collect the data to > flink > > > Thanks > > Jinkui Shi > >
