Could you try setting set that env.java.opts in the Flink configuration?
I remember a similar thread; we have a sort of bootstrap application which runs with a very limited classpath for some configuration stuff, and by setting the log4j configuration you are also affecting said application, which then needs jackson but it doesn't use lib/.
On 14/10/2021 14:30, Yuval Itzchakov wrote:
Yes I am On Thu, Oct 14, 2021, 13:32 Chesnay Schepler <ches...@apache.org> wrote: Are you by chance explicitly setting the -Dlog4j.configurationFile option (or however it is called?) On 14/10/2021 11:59, Yuval Itzchakov wrote:Just tried adding the jars to lib/, I still receive the same error message. On Thu, Oct 14, 2021 at 12:53 PM Yuval Itzchakov <yuva...@gmail.com> wrote: I have an UBER jar that contains all the dependencies I require for the execution, I find it weird that I need to add an external library to lib to support this. On Thu, Oct 14, 2021 at 12:38 PM Chesnay Schepler <ches...@apache.org> wrote: You need to add the jars to the lib directory. This is independent of how Flink works; the JSONLayout needs jackson, so you need to make sure it is on the classpath. On 14/10/2021 11:34, Yuval Itzchakov wrote:Scala 2.12 Java 11 Flink 1.13.2 Running in Kubernetes Hi, While trying to use a configuration with JSONLayout in log4j2 with XML format, we receive the following error when bootstrapping the JM / TM: flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ser/FilterProvider flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.layout.JsonLayout.<init>(JsonLayout.java:158) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.layout.JsonLayout.<init>(JsonLayout.java:69) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.layout.JsonLayout$Builder.build(JsonLayout.java:102) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.layout.JsonLayout$Builder.build(JsonLayout.java:77) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:122) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:1002) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:942) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:934) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:934) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:552) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:241) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:288) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:579) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:651) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:668) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:253) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:153) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.LogManager.getContext(LogManager.java:155) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:45) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:48) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:30) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:329) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:349) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at org.apache.flink.runtime.util.bash.BashJavaUtils.<clinit>(BashJavaUtils.java:45) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.ser.FilterProvider flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) flink-adte-f7925524a2d37c92873ff0020dcd46c3-22b605b8-jm-946v7qt jobmanager at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) According to https://stackoverflow.com/questions/65062081/java-lang-classnotfoundexception-com-fasterxml-jackson-databind-ser-filterprovi, this happens due to the fact the JM / TM start with a modified classpath that doesn't include jackson-databind. While I could add the jars explicitly to the `lib/` directory in the Dockerfile, this is pretty annoying to maintain as versions evolve. Is there any other workaround for this? or an open issue?-- Best Regards,Yuval Itzchakov.-- Best Regards,Yuval Itzchakov.-- Best Regards,Yuval Itzchakov.
