Re: Akka Http used in custom RichSourceFunction

2018-05-25 Thread Piotr Nowojski
Thanks for getting back and I’m glad that you were able to resolve your issue :) Piotrek > On 25 May 2018, at 11:25, Niels van Kaam wrote: > > Hi, > > It was indeed the problem, and shading my akka dependency has solved the > problem. Thank you for pointing that out! > >

Re: Akka Http used in custom RichSourceFunction

2018-05-25 Thread Niels van Kaam
Hi, It was indeed the problem, and shading my akka dependency has solved the problem. Thank you for pointing that out! For references: When shading akka you also need to merge the reference.conf files from akka, or it will fail. This page contains useful documentation on how to shade akka:

Re: Akka Http used in custom RichSourceFunction

2018-05-25 Thread Piotr Nowojski
Hi, Yes, this might be the cause of the issue, because indeed it looks like your akka’s version is leaking to Flink’s classloader. Piotrek > On 25 May 2018, at 09:40, Niels van Kaam wrote: > > Hi Piotrek, > > Thank you for your response! > > I am currently just testing

Re: Akka Http used in custom RichSourceFunction

2018-05-25 Thread Niels van Kaam
Hi Piotrek, Thank you for your response! I am currently just testing the job in a local environment. I think that means all classes are in the Java classpath, which might also be the issue then. If I am correct that means I am currently not using dynamic classloading and just overwriting the

Re: Akka Http used in custom RichSourceFunction

2018-05-24 Thread Piotr Nowojski
Hi, Please take a look on https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/debugging_classloading.html . Especially check if you are using child-first class loading

Akka Http used in custom RichSourceFunction

2018-05-24 Thread Niels van Kaam
Hi All, I wrote a custom source function (RichSourceFunction) which connects to a web socket using the Akka Http Library. The job using this source runs fine on a local environment until upon shutdown I see the following error in the log: "Exception in thread "main" java.lang.NoSuchMethodError: