Re: Continuous Reading of File using FileSource does not process the existing files in version 1.17

2024-01-08 Thread David Anderson
While the readFile method would monitor changes to existing files, it would completely re-ingest each changed file after every change. This behavior wasn't very user friendly. David On Fri, Jan 5, 2024 at 2:22 AM Martijn Visser wrote: > Hi Prasanna, > > I think this is as expected. There is no

Flink pending record metric weired after autoscaler rescaling

2024-01-08 Thread Yang LI
Dear Flink Community, I've encountered an issue during the testing of my Flink autoscaler. It appears that Flink is losing track of specific Kafka partitions, leading to a persistent increase in lag on these partitions. The logs indicate a 'kafka connector metricGroup name collision exception.'

Rabbitmq connector for Flink v1.18

2024-01-08 Thread Charlotta Westberg via user
Hi, We are using rabbitmq sources and sinks, and wanted to upgrade to flink 1.18, but noticed the documentation on RabbitMQ Connector mentioned There is no connector (yet) available for Flink version 1.18 I tried to find a JIRA issue for the connector to support flink 1.18 but was unable to.