Reporters do not have access to libraries provided with user-jars.
They are instantiated when JM/TM starts, i.e. before any user-code is even accessible.

My recommendation would be to either put the kafka dependencies in the /lib folder or try to relocate the kafka code in the reporter.

On 11.07.2018 14:59, Gyula Fóra wrote:
Hi all,

I have ran into the following problem and I want to double check wether this is intended behaviour.

I have a custom metrics reporter that pushes things to Kafka (so it creates a KafkaProducer in the open method etc.etc.) for my streaming job.

Naturally as my Flink job consumes from Kafka so it has the kafka connector dependencies I set the Kafka dependencies to provided in my metric reporter project and I put the built kafkaReporter.jar into the Flink lib. However it seems that the metrics reporter is instantiated without the user code classes since I get a NoClassdefFound error for KafkaProducer even though my streaming job starts successfully reading/writing kafka.

Any ideas why this happens and how to solve it? I am slightly against putting the kafka dependencies twice on the classpath as it has only caused problems in the past...


Reply via email to