Hello, We are writing TCP server socket custom source function in which TCP server socket listener will accept connections and read data. Single Custom source server socket function – ServerSocket serversocket = new ServerSocket(9999); Now using thread pool accept multiple connections in separate threads = new Runnable () -> serversocket.accept(); So client socket will be accepted and given to separate thread for read data from TCP stream. Rgds, Kamal From: Alexander Fedulov <alexander.fedu...@gmail.com> Sent: 31 October 2023 04:03 PM To: Kamal Mittal <kamal.mit...@ericsson.com> Cc: user@flink.apache.org Subject: Re: Flink custom parallel data source
Please note that SourceFunction API is deprecated and is due to be removed, possibly in the next major version of Flink. Ideally you should not be manually spawning threads in your Flink applications. Typically you would only perform data fetching in the sources and do processing in the subsequent operators which you can scale independently from the source parallelism. Can you describe what you are trying to achieve? Best, Alexander Fedulov On Tue, 31 Oct 2023 at 07:22, Kamal Mittal via user <user@flink.apache.org<mailto:user@flink.apache.org>> wrote: Hello Community, I need to have a custom parallel data source (Flink ParallelSourceFunction) for fetching data based on some custom logic. In this source function, opening multiple threads via java thread pool to distribute work further. These threads share Flink provided ‘SourceContext’ and collect records via source_context.collect() method. Is it ok to share source context in separate threads and get data? Is there any issue for downstream operators due to above design? Rgds, Kamal
