Hi Peng What kind of deployment of K8s did you try in flink-doc[1], if using session mode, you can control your log4j configuration via configmap [2]. From my experience, this could control the log4j well.
If you did not override the command of flink docker, it will start-foreground the taskmanager. [1] https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/kubernetes.html [2] https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/kubernetes.html#session-cluster-resource-definitions Best Yun Tang From: Li Peng <[email protected]> Date: Tuesday, December 10, 2019 at 10:09 AM To: user <[email protected]> Subject: Flink on Kubernetes seems to ignore log4j.properties Hey folks, I noticed that my kubernetes flink logs (reached via kubectl logs <pod-name>) completely ignore any of the configurations I put into /flink/conf/. I set the logger level to WARN, yet I still see INFO level logging from flink loggers like org.apache.flink.runtime.checkpoint.CheckpointCoordinator. I even made copied the same properties to /flink/conf/log4j-console.properties and log4j-cli.properties. From what I can tell, kubernetes just listens to stdout and stderr, so shouldn't the log4j.properties control output to them? Anyone seen this issue before? Here is my log4j.properties: # This affects logging for both user code and Flink log4j.rootLogger=WARN, file, console, stdout # Uncomment this if you want to _only_ change Flink's logging log4j.logger.org.apache.flink=WARN # The following lines keep the log level of common libraries/connectors on # log level INFO. The root logger does not override this. You have to manually # change the log levels here. log4j.logger.akka=INFO log4j.logger.org.apache.kafka=INFO log4j.logger.org.apache.hadoop=INFO log4j.logger.org.apache.zookeeper=INFO # Log all infos in the given file log4j.appender.file=org.apache.log4j.FileAppender log4j.appender.file.file=${log.file} log4j.appender.file.append=false log4j.appender.file.layout=org.apache.log4j.PatternLayout log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n # Log all infos to the console log4j.appender.console=org.apache.log4j.ConsoleAppender log4j.appender.console.layout=org.apache.log4j.PatternLayout log4j.appender.console.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n # Suppress the irrelevant (wrong) warnings from the Netty channel handler log4j.logger.org.apache.flink.shaded.akka.org.jboss.netty.channel.DefaultChannelPipeline=ERROR, file, console log4j.logger.org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction=WARN log4j.logger.org.apache.flink.runtime.checkpoint=WARN Thanks, Li
