Hi, Flink's ProcessFunction has a close() method, which is executed on shutdown of the workers. (You could also use any of the Rich* functions for that purpose). If you add a ProcessFunction with the same parallelism before the KafkaSink, it'll be executed on the same machines as the Kafka producer.
Afaik, the close() call should not take forever, as the system might interrupt your thread if it doesn't finish closing on time (30s is the default for "cluster.services.shutdown-timeout") Best, Robert On Tue, Jan 21, 2020 at 10:02 AM Dominique De Vito <ddv36...@gmail.com> wrote: > Hi, > > For a Flink batch job, some value are writing to Kafka through a Producer. > > I want to register a hook for closing (at the end) the Kafka producer a > worker is using.... hook to be executed, of course, on worker side. > > Is there a way to do so ? > > Thanks. > > Regards, > Dominique > > > >
