Correct! Building a custom image for the deployment and then copying over the
jar to a specific directory for the FlinkDeployment to use (as the image
contains the legacy Flink jobs/jars as well as those newer ones for the
operator).
> On Feb 22, 2024, at 6:18 AM, dominik.buen...@swisscom.com wrote:
>
>
> Hi Rion
>
> I guess you’re building your own docker image for the deployment right?
>
> For switching to Logback I’m doing the following command (sbt-docker) when
> building the image.
>
> val eclasspath = (Compile / externalDependencyClasspath).value
> val logbackClassicJar = eclasspath.files.find(file =>
> file.getName.contains("logback-classic"))
> logbackClassicJar.foreach(logback => add(logback, «opt/flink/lib»))
>
> Given the error message that you provided I think the dependency is missing
> in the lib folder (don’t confuse this with the usrlib folder).
>
> Kind Regards
> Dominik
> From: Rion Williams <rionmons...@gmail.com>
> Date: Thursday, 22 February 2024 at 13:09
> To: Bünzli Dominik, INI-DNA-INF <dominik.buen...@swisscom.com>
> Cc: user@flink.apache.org <user@flink.apache.org>
> Subject: Re: Using Custom JSON Formatting with Flink Operator
>
> Be aware: This is an external email.
>
> Hi Dominick,
>
> In this case the jobs are running using application-mode. All of these were
> previously working as expected for the legacy jobs using the same
> configuration (however those were running via Ververica Platform and
> targeting Flink 1.15.2). I had somewhat expected similar behaviors but it
> seems there’s something that is missing.
>
> Thanks,
>
> Rion
>
>
> On Feb 22, 2024, at 1:15 AM, dominik.buen...@swisscom.com wrote:
>
>
> Good morning Rion,
>
> Are you in session job mode or application mode? I’ve had some similar issues
> (logback) lately and it turned out that I also needed to add the additional
> dependencies (I guess JsonTemplateLayout is one of them) to the lib folder of
> the deployment.
>
> Kind regards
> Dominik
>
> From: Rion Williams <rionmons...@gmail.com>
> Date: Thursday, 22 February 2024 at 00:46
> To: Flink User List <user@flink.apache.org>
> Subject: Using Custom JSON Formatting with Flink Operator
>
>
> Be aware: This is an external email.
>
>
>
> Hey Flinkers,
>
> Recently I’ve been in the process of migrating a series of older Flink jobs
> to use the official operator and have run into a snag on the logging front.
>
> I’ve attempted to use the following configuration for the job:
>
> ```
> logConfiguration:
> log4j-console.properties: |+
> rootLogger.level = INFO
> rootLogger.appenderRef.console.ref = ConsoleAppender
> rootLogger.appenderRef.rolling.ref = RollingFileAppender
> ...
> appender.console.name = ConsoleAppender
> appender.console.type = Console
> appender.console.layout.type = JsonTemplateLayout
> appender.console.layout.eventTemplateUri = classpath:GcpLayout.json
> ```
>
> However once the job begins running, I’m met with the following errors in the
> logs:
>
> ```
> ERROR Unable to locate plugin type for JsonTemplateLayout
> ERROR Unable to locate plugin for JsonTemplateLayout
> ERROR Could not create plugin of type class
> org.apache.logging.log4j.core.appender.ConsoleAppender for element Console:
> java.lang.NullPointerException java.lang.NullPointerException
> ```
>
> I believe that all of the appropriate references are correct in the actual
> shaded jar itself as I can see things like the JsonTemplateLayout inside of
> it (under org.apache.logging.log4j.template.json.JsonTemplateLayout ) as well
> as the GcpLayout that I’m targeting in the root of the shaded jar as well
> (including trying several adjustments to shade exceptions, adding a log4j
> specific shade transformer, etc.)
>
> I’ve tried adjusting several different knobs/configurations but I’m still
> continually getting this same error. I’d be happy to share any additional
> configuration for the job any/or the FlinkDeployment where applicable.
>
> Just a bit stumped here on something that feels like it should just work.
