Hi,

I have run into a classpath issue when running Flink streaming job in YARN
session. I package my app into a fat jar with all the dependencies needed.
One of them is Google Guava. I then submit the jar to the session. The task
managers pre-created by the session build their classpath from the
FLINK_LIB_DIR and Hadoop / YARN lib directories. Unfortunately, there is a
dated Guava version pulled along with Hadoop dependencies which conflicts
with the version my app needs. Even worse, the Flink lib dir and Hadoop
libraries take precedence over my jar.

If I remember correctly, in Spark there is an option meaning "user
classpath goes first when looking for classes". I couldn't find anything
similar for Flink. I tried "flink run -C file:///path/to/my/libraries" in
the hope to extend the classpath but the old Guava version takes precedence
anyway.

How else can I bump "priority" of my app jar during the load process?

Thanks,
Yury

Reply via email to