zentol opened a new pull request #11073: [FLINK-15672][build] Migrate to log4j2 URL: https://github.com/apache/flink/pull/11073 Incorporates #10984. This PR migrates the entire project to log4j2. Let me walk you through all the changes that were necessary for this to work: ### Dependencies Previously we relied on `log4j:log4j` (the logging framework) and `org.slf4j:slf4j-log4j12` (adapter from log4j to slf4j); collectively referred to as `log4j1 dependencies" from now on. We now rely on `org.apache.logging.log4j:log4j-api` (logging framework API, somewhat similar to what slf4j does), `org.apache.logging.log4j:log4j-core` (logging framework implementation), `org.apache.logging.log4j:log4j-slf4j-impl` (adapter from log4j to slf4j) and `org.apache.logging.log4j:log4j-1.2-api` (API bridge between log4j1 and 2). The version of all new dependencies is `2.12.1`, which was released August 2019. These dependencies are all on the test classpath. To prevent multiple log4j bindings from existing on the classpath, all traces of log4j 1 dependencies have been removed from the dependency tree through the gratuitous use of exclusions. An enforcer rule was additionally put into place to prevent these from slipping in again. This should not affect any transitive dependencies relying on log4j1 due to the log4j API bridge also being on the test and runtime classpath. The transition also allowed us to simplify the test logging setup for the ES connectors, as they no longer have to manually swap out log4j 1. ### Quickstarts/Walkthroughs The archetypes were naturally adjusted to use log4j2 as well, and the log4j version is now derived from the parent project instead of being hard-coded. ### Packaging The new log4j2 dependencies are bundled in flink-dist just like the old dependencies; i.e. under lib. Now however there will be 4 jars instead of 2. Modules that were previously explicitly bundling log4j2 (ES5-7) now longer do so. ### Licensing With the removal of `org.slf4j:slf4j-log4j12` we no longer need the dedicated slf4j NOTICE preamble nor license in the binary licensing. The ES5-7 modules no longer bundle log4j2 separately and hence had their relevant NOTICE entries removed. ### API changes Some tests are directly working against the log4j API (for various, and partially justified, reasons). These had to be migrated to new APIs. To unify this a new `TestLoggerResource` was introduced, and relevant tests adjusted. ### Configuration #### file property Log4j can automatically pick of configurations from the classpath, but traditionally we opted to explicitly configure the file location through a property instead for actual deployments. For log4j1 this property was `log4j.configuration`, whereas for log4j2 it is `log4j.configurationFile`. The latter aligns log4j with logback. **Note that this change currently prevents users from using log4j1 as their logging framework.** #### *.properties file names Log4j2 by default looks for `log4j2[-test].properties`.Accordingly, all `log4j-test.properties` were renamed to `log4j2-test.properties`, all `log4j.properties` used for examples were renamed to `log4j2.properties`. This should result in them automatically being picked up in the IDE. However, all *.properties files that are exposed to users have retained their old name (like `log4j-cli.properties`), in an attempt to prevent setups from being broken that do _some_ move/copy operations on them. #### *.properties file contents All existing log4j properties files have been migrated to the new syntax that log4j2 uses, as have all instances where we generated/modified the configuration programmatically. The old syntax is _no longer supported_, and will either be ignored or cause errors. The new syntax is a bit of a mixed bag, so let's dive into an example: ``` rootLogger.level = INFO rootLogger.appenderRef.name_1.ref = name_4 logger.name_2.name = akka logger.name_2.level = INFO appendername_3.name = name_4 appender.name_3.type = File appender.name_3.append = false appender.name_3.fileName = ${sys:log.file} appender.name_3.layout.type = PatternLayout appender.name_3.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n ``` ${sys:log.file} refers to en JVm system property. The most confusing thing in this syntax is the concept of "names" since the semantics vary significantly. All loggers and appenders have an identifier (e.g., name_2/3) that identifies it for options configured on that object; this conceptually works just like our reporters do. I.e., all options with the same identifier are applied to the same appender/logger. Loggers also have a `name` parameter that controls the package/class for which it is used. Appenders also have a `name` parameter, but this one instead is a reference that loggers can use as an `appenderRef`, which controls what appenders should be used for this logger. And while we're on the topic of `appenderRef`; these always take the form `logger.<logger_idenfitier>.appenderRef.<some_name>.<appender_name_ref>`, where `some_name` is literally arbitrary, the only important thing being that you don't have 2 refs on the same logger with the same name.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
