This is an automated email from the ASF dual-hosted git repository.
yong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git
The following commit(s) were added to refs/heads/master by this push:
new a522fa3 Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571
(#2816)
a522fa3 is described below
commit a522fa33b31e2405830a33cd2120be60d3174cd5
Author: Raúl Gracia <[email protected]>
AuthorDate: Fri Oct 15 02:16:02 2021 +0200
Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571 (#2816)
### Motivation
Upgrades to log4j2 to get rid of CVE-2019-17571.
### Changes
The migration of log4j has been done mainly taking the official guidelines:
https://logging.apache.org/log4j/2.x/manual/migration.html.
In this PR, the following changes are included:
- Replacement of `slf4j-log4j12` by `log4j-1.2-api`. Also included the
`log4j-slf4j-impl` binding as well as the `log4j-core` library.
- Changes in `pom`, `gradle` and license files to reflect the above library
upgrade.
- Test classes `TestOrderedExecutorDecorators`, `LoggerOutput`,
`MdcContextTest`, as well as the class `FIleSystemUpgrade` made use of log4j1.2
API. This PR attempts to keep the same functionality with the new APIs.
### Verification
- Existing tests are passing.
- log4j1.2 is removed from project:
https://github.com/apache/bookkeeper/pull/2816#issuecomment-937818625
- Using `localbookie`, we observe that logs are shown correctly:
```
2021-10-07T16:04:23,757 - INFO - [main:GarbageCollectorThread@245] - Minor
Compaction : enabled=true, threshold=0.20000000298023224, interval=3600000
2021-10-07T16:04:23,760 - INFO - [main:GarbageCollectorThread@247] - Major
Compaction : enabled=true, threshold=0.800000011920929, interval=86400000
2021-10-07T16:04:23,952 - INFO - [main:BookieImpl@920] - Finished
replaying journal in 2 ms.
2021-10-07T16:04:23,958 - INFO - [SyncThread-7-1:SyncThread@135] - Flush
ledger storage at checkpoint CheckpointList{checkpoints=[LogMark: logFileId - 0
, logFileOffset - 0]}.
2021-10-07T16:04:23,980 - INFO - [main:BookieImpl@1010] - Finished reading
journal, starting bookie
2021-10-07T16:04:24,011 - INFO - [BookieJournal-5000:Journal@919] -
Starting journal on /tmp/localbookkeeper06554024139823286046test/current
2021-10-07T16:04:24,031 - INFO -
[ForceWriteThread:Journal$ForceWriteThread@478] - ForceWrite Thread started
2021-10-07T16:04:24,048 - INFO - [BookieJournal-5000:JournalChannel@169] -
Opening journal
/tmp/localbookkeeper06554024139823286046test/current/17c5b11c65b.txn
```
In addition to that, if we change the `log4j.properties` file, the changes
are reflected in the console output, meaning that the legacy configuration
works and changes can be correctly applied:
```
Over Replicated Ledger Deletion : enabled=true, interval=86400000
Minor Compaction : enabled=true, threshold=0.20000000298023224,
interval=3600000
Major Compaction : enabled=true, threshold=0.800000011920929,
interval=86400000
Finished replaying journal in 5 ms.
Flush ledger storage at checkpoint CheckpointList{checkpoints=[LogMark:
logFileId - 0 , logFileOffset - 0]}.
Finished reading journal, starting bookie
Starting journal on /tmp/localbookkeeper015049859959001160726test/current
ForceWrite Thread started
Opening journal
/tmp/localbookkeeper015049859959001160726test/current/17c5b143063.txn
```
More verifications that logging works properly related to other Bookkeeper
sub-components impacted may be needed.
Master Issue: #2815
---
bookkeeper-common/build.gradle | 5 +--
bookkeeper-common/pom.xml | 20 +++++++++++
.../bookkeeper/common/util/OrderedExecutor.java | 2 +-
.../common/util/TestOrderedExecutorDecorators.java | 40 ++++++++++++----------
bookkeeper-dist/all/build.gradle | 2 +-
bookkeeper-dist/all/pom.xml | 12 +++++--
bookkeeper-dist/bkctl/pom.xml | 12 +++++--
bookkeeper-dist/server/build.gradle | 2 +-
bookkeeper-dist/server/pom.xml | 12 +++++--
.../src/main/resources/LICENSE-all.bin.txt | 8 +++--
.../src/main/resources/LICENSE-bkctl.bin.txt | 8 +++--
.../src/main/resources/LICENSE-server.bin.txt | 8 +++--
bookkeeper-server/build.gradle | 4 ++-
bookkeeper-server/pom.xml | 12 +++++--
.../bookkeeper/bookie/FileSystemUpgrade.java | 6 ++--
.../apache/bookkeeper/client/MdcContextTest.java | 36 ++++++++++---------
.../org/apache/bookkeeper/util/LoggerOutput.java | 38 +++++++++++---------
dependencies.gradle | 7 ++--
microbenchmarks/pom.xml | 12 +++++--
pom.xml | 27 ++++++++++-----
stats/utils/build.gradle | 2 +-
stream/distributedlog/io/dlfs/pom.xml | 8 +++++
stream/server/build.gradle | 2 +-
.../tests/integration/utils/MavenClassLoader.java | 3 +-
tests/integration/cluster/build.gradle | 2 +-
tools/ledger/pom.xml | 14 ++++++--
tools/perf/build.gradle | 2 +-
tools/stream/build.gradle | 2 +-
28 files changed, 210 insertions(+), 98 deletions(-)
diff --git a/bookkeeper-common/build.gradle b/bookkeeper-common/build.gradle
index 27a9a06..94b5b36 100644
--- a/bookkeeper-common/build.gradle
+++ b/bookkeeper-common/build.gradle
@@ -41,9 +41,10 @@ dependencies {
testImplementation depLibs.commonsLang3
testImplementation depLibs.hamcrest
testImplementation depLibs.junit
- testImplementation depLibs.log4j
+ testImplementation depLibs.log4jSlf4jImpl
+ testImplementation depLibs.log4j12api
+ testImplementation depLibs.log4jCore
testImplementation depLibs.mockito
- testImplementation depLibs.slf4jLog4j
annotationProcessor depLibs.lombok
testAnnotationProcessor depLibs.lombok
diff --git a/bookkeeper-common/pom.xml b/bookkeeper-common/pom.xml
index 82a6792..c1065c9 100644
--- a/bookkeeper-common/pom.xml
+++ b/bookkeeper-common/pom.xml
@@ -74,6 +74,26 @@
<artifactId>commons-lang3</artifactId>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-1.2-api</artifactId>
+ <version>${log4j.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ <version>${log4j.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <version>${log4j.version}</version>
+ </dependency>
</dependencies>
<build>
<plugins>
diff --git
a/bookkeeper-common/src/main/java/org/apache/bookkeeper/common/util/OrderedExecutor.java
b/bookkeeper-common/src/main/java/org/apache/bookkeeper/common/util/OrderedExecutor.java
index 4330340..2c89211 100644
---
a/bookkeeper-common/src/main/java/org/apache/bookkeeper/common/util/OrderedExecutor.java
+++
b/bookkeeper-common/src/main/java/org/apache/bookkeeper/common/util/OrderedExecutor.java
@@ -410,7 +410,7 @@ public class OrderedExecutor implements ExecutorService {
try {
CpuAffinity.acquireCore();
} catch (Throwable t) {
- log.warn("Failed to acquire CPU core for thread
{}", Thread.currentThread().getName(),
+ log.warn("Failed to acquire CPU core for thread
{}: {}", Thread.currentThread().getName(),
t.getMessage(), t);
}
}
diff --git
a/bookkeeper-common/src/test/java/org/apache/bookkeeper/common/util/TestOrderedExecutorDecorators.java
b/bookkeeper-common/src/test/java/org/apache/bookkeeper/common/util/TestOrderedExecutorDecorators.java
index f03b24d..f8d53b4 100644
---
a/bookkeeper-common/src/test/java/org/apache/bookkeeper/common/util/TestOrderedExecutorDecorators.java
+++
b/bookkeeper-common/src/test/java/org/apache/bookkeeper/common/util/TestOrderedExecutorDecorators.java
@@ -27,18 +27,18 @@ import static org.junit.Assert.assertThat;
import static org.mockito.AdditionalAnswers.answerVoid;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.doAnswer;
-import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.when;
+import static org.mockito.Mockito.spy;
import java.util.Queue;
+import java.util.UUID;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.TimeUnit;
-import org.apache.log4j.Appender;
-import org.apache.log4j.Level;
-import org.apache.log4j.LogManager;
import org.apache.log4j.MDC;
-import org.apache.log4j.spi.LoggingEvent;
+import org.apache.logging.log4j.Level;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.LoggerContext;
+import org.apache.logging.log4j.core.appender.NullAppender;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
@@ -53,7 +53,7 @@ public class TestOrderedExecutorDecorators {
private static final Logger log =
LoggerFactory.getLogger(TestOrderedExecutorDecorators.class);
private static final String MDC_KEY = "mdc-key";
- private Appender mockAppender;
+ private NullAppender mockAppender;
private final Queue<String> capturedEvents = new ConcurrentLinkedQueue<>();
public static String mdcFormat(Object mdc, String message) {
@@ -63,21 +63,25 @@ public class TestOrderedExecutorDecorators {
@Before
public void setUp() throws Exception {
MDC.clear();
- mockAppender = mock(Appender.class);
- when(mockAppender.getName()).thenReturn("MockAppender");
-
- LogManager.getRootLogger().addAppender(mockAppender);
- LogManager.getRootLogger().setLevel(Level.INFO);
-
- doAnswer(answerVoid((LoggingEvent event) -> {
- capturedEvents.add(mdcFormat(event.getMDC(MDC_KEY),
- event.getRenderedMessage()));
- })).when(mockAppender).doAppend(any());
+ LoggerContext lc = (LoggerContext)
org.apache.logging.log4j.LogManager.getContext(false);
+ mockAppender =
spy(NullAppender.createAppender(UUID.randomUUID().toString()));
+ mockAppender.start();
+ lc.getConfiguration().addAppender(mockAppender);
+
lc.getRootLogger().addAppender(lc.getConfiguration().getAppender(mockAppender.getName()));
+ lc.getConfiguration().getRootLogger().setLevel(Level.INFO);
+ lc.updateLoggers();
+
+ doAnswer(answerVoid((LogEvent event) -> {
+
capturedEvents.add(mdcFormat(event.getContextData().getValue(MDC_KEY),
+
event.getMessage().getFormattedMessage()));
+ })).when(mockAppender).append(any());
}
@After
public void tearDown() throws Exception {
- LogManager.getRootLogger().removeAppender(mockAppender);
+ LoggerContext lc = (LoggerContext)
org.apache.logging.log4j.LogManager.getContext(false);
+
lc.getRootLogger().removeAppender(lc.getConfiguration().getAppender(mockAppender.getName()));
+ lc.updateLoggers();
capturedEvents.clear();
MDC.clear();
}
diff --git a/bookkeeper-dist/all/build.gradle b/bookkeeper-dist/all/build.gradle
index 2ad3719..343f01a 100644
--- a/bookkeeper-dist/all/build.gradle
+++ b/bookkeeper-dist/all/build.gradle
@@ -40,7 +40,7 @@ dependencies {
compileOnly depLibs.lombok
compileOnly depLibs.spotbugsAnnotations
- implementation depLibs.slf4jLog4j
+ implementation depLibs.log4j12api
testCompileOnly depLibs.lombok
diff --git a/bookkeeper-dist/all/pom.xml b/bookkeeper-dist/all/pom.xml
index c337e99..146dafa 100644
--- a/bookkeeper-dist/all/pom.xml
+++ b/bookkeeper-dist/all/pom.xml
@@ -99,8 +99,16 @@
<!-- slf4j binding -->
<dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-1.2-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
</dependency>
<dependency>
diff --git a/bookkeeper-dist/bkctl/pom.xml b/bookkeeper-dist/bkctl/pom.xml
index a4ab31f..8ac6545 100644
--- a/bookkeeper-dist/bkctl/pom.xml
+++ b/bookkeeper-dist/bkctl/pom.xml
@@ -47,8 +47,16 @@
<!-- slf4j binding -->
<dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-1.2-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
</dependency>
</dependencies>
diff --git a/bookkeeper-dist/server/build.gradle
b/bookkeeper-dist/server/build.gradle
index a3264f7..6ce817d 100644
--- a/bookkeeper-dist/server/build.gradle
+++ b/bookkeeper-dist/server/build.gradle
@@ -42,7 +42,7 @@ dependencies {
compileOnly depLibs.lombok
compileOnly depLibs.spotbugsAnnotations
- implementation depLibs.slf4jLog4j
+ implementation depLibs.log4j12api
testCompileOnly depLibs.lombok
diff --git a/bookkeeper-dist/server/pom.xml b/bookkeeper-dist/server/pom.xml
index 41edb48..83777f5 100644
--- a/bookkeeper-dist/server/pom.xml
+++ b/bookkeeper-dist/server/pom.xml
@@ -85,8 +85,16 @@
<!-- slf4j binding -->
<dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-1.2-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
</dependency>
<dependency>
diff --git a/bookkeeper-dist/src/main/resources/LICENSE-all.bin.txt
b/bookkeeper-dist/src/main/resources/LICENSE-all.bin.txt
index 5eab1cf..d13f96a 100644
--- a/bookkeeper-dist/src/main/resources/LICENSE-all.bin.txt
+++ b/bookkeeper-dist/src/main/resources/LICENSE-all.bin.txt
@@ -241,7 +241,10 @@ Apache Software License, Version 2.
- lib/io.vertx-vertx-core-3.9.8.jar [15]
- lib/io.vertx-vertx-web-3.9.8.jar [16]
- lib/io.vertx-vertx-web-common-3.9.8.jar [16]
-- lib/log4j-log4j-1.2.17.jar [17]
+- lib/org.apache.logging.log4j-log4j-1.2-api-2.14.1.jar [17]
+- lib/org.apache.logging.log4j-log4j-api-2.14.1.jar [17]
+- lib/org.apache.logging.log4j-log4j-core-2.14.1.jar [17]
+- lib/org.apache.logging.log4j-log4j-slf4j-impl-2.14.1.jar [17]
- lib/net.java.dev.jna-jna-3.2.7.jar [18]
- lib/org.apache.commons-commons-collections4-4.1.jar [19]
- lib/org.apache.commons-commons-lang3-3.6.jar [20]
@@ -318,7 +321,7 @@ Apache Software License, Version 2.
[14] Source available at
https://github.com/vert-x3/vertx-bridge-common/tree/3.9.8
[15] Source available at https://github.com/eclipse/vert.x/tree/3.9.8
[16] Source available at https://github.com/vert-x3/vertx-web/tree/3.9.8
-[17] Source available at http://logging.apache.org/log4j/1.2/download.html
+[17] Source available at
https://github.com/apache/logging-log4j2/tree/rel/2.14.1
[18] Source available at https://github.com/java-native-access/jna/tree/3.2.7
[19] Source available at
https://git-wip-us.apache.org/repos/asf?p=commons-collections.git;a=tag;h=a3a5ad
[20] Source available at
https://git-wip-us.apache.org/repos/asf?p=commons-lang.git;a=shortlog;h=refs/tags/LANG_3_6
@@ -639,7 +642,6 @@ MIT license. For details, see deps/slf4j-1.7.32/LICENSE.txt.
Bundled as
- lib/org.slf4j-slf4j-api-1.7.32.jar
- - lib/org.slf4j-slf4j-log4j12-1.7.32.jar
Source available at https://github.com/qos-ch/slf4j/tree/v_1.7.32
------------------------------------------------------------------------------------
This product bundles the Google Auth Library, which is available under a
"3-clause BSD"
diff --git a/bookkeeper-dist/src/main/resources/LICENSE-bkctl.bin.txt
b/bookkeeper-dist/src/main/resources/LICENSE-bkctl.bin.txt
index b02283b..ceaa468 100644
--- a/bookkeeper-dist/src/main/resources/LICENSE-bkctl.bin.txt
+++ b/bookkeeper-dist/src/main/resources/LICENSE-bkctl.bin.txt
@@ -230,7 +230,10 @@ Apache Software License, Version 2.
- lib/io.netty-netty-transport-4.1.68.Final.jar [11]
- lib/io.netty-netty-transport-native-epoll-4.1.68.Final-linux-x86_64.jar [11]
- lib/io.netty-netty-transport-native-unix-common-4.1.68.Final.jar [11]
-- lib/log4j-log4j-1.2.17.jar [16]
+- lib/org.apache.logging.log4j-log4j-1.2-api-2.14.1.jar [16]
+- lib/org.apache.logging.log4j-log4j-api-2.14.1.jar [16]
+- lib/org.apache.logging.log4j-log4j-core-2.14.1.jar [16]
+- lib/org.apache.logging.log4j-log4j-slf4j-impl-2.14.1.jar [16]
- lib/net.java.dev.jna-jna-3.2.7.jar [17]
- lib/org.apache.commons-commons-collections4-4.1.jar [18]
- lib/org.apache.commons-commons-lang3-3.6.jar [19]
@@ -290,7 +293,7 @@ Apache Software License, Version 2.
[9] Source available at
https://git-wip-us.apache.org/repos/asf?p=commons-lang.git;a=tag;h=375459
[10] Source available at
http://svn.apache.org/viewvc/commons/proper/logging/tags/commons-logging-1.1.1/
[11] Source available at https://github.com/netty/netty/tree/netty-4.1.68.Final
-[16] Source available at http://logging.apache.org/log4j/1.2/download.html
+[16] Source available at
https://github.com/apache/logging-log4j2/tree/rel/2.14.1
[17] Source available at https://github.com/java-native-access/jna/tree/3.2.7
[18] Source available at
https://git-wip-us.apache.org/repos/asf?p=commons-collections.git;a=tag;h=a3a5ad
[19] Source available at
https://git-wip-us.apache.org/repos/asf?p=commons-lang.git;a=shortlog;h=refs/tags/LANG_3_6
@@ -565,7 +568,6 @@ MIT license. For details, see deps/slf4j-1.7.32/LICENSE.txt.
Bundled as
- lib/org.slf4j-slf4j-api-1.7.32.jar
- - lib/org.slf4j-slf4j-log4j12-1.7.32.jar
Source available at https://github.com/qos-ch/slf4j/tree/v_1.7.32
------------------------------------------------------------------------------------
This product bundles the Google Auth Library, which is available under a
"3-clause BSD"
diff --git a/bookkeeper-dist/src/main/resources/LICENSE-server.bin.txt
b/bookkeeper-dist/src/main/resources/LICENSE-server.bin.txt
index 4dfb97c..7c0bad9 100644
--- a/bookkeeper-dist/src/main/resources/LICENSE-server.bin.txt
+++ b/bookkeeper-dist/src/main/resources/LICENSE-server.bin.txt
@@ -241,7 +241,10 @@ Apache Software License, Version 2.
- lib/io.vertx-vertx-core-3.9.8.jar [15]
- lib/io.vertx-vertx-web-3.9.8.jar [16]
- lib/io.vertx-vertx-web-common-3.9.8.jar [16]
-- lib/log4j-log4j-1.2.17.jar [17]
+- lib/org.apache.logging.log4j-log4j-1.2-api-2.14.1.jar [17]
+- lib/org.apache.logging.log4j-log4j-api-2.14.1.jar [17]
+- lib/org.apache.logging.log4j-log4j-core-2.14.1.jar [17]
+- lib/org.apache.logging.log4j-log4j-slf4j-impl-2.14.1.jar [17]
- lib/net.java.dev.jna-jna-3.2.7.jar [18]
- lib/org.apache.commons-commons-collections4-4.1.jar [19]
- lib/org.apache.commons-commons-lang3-3.6.jar [20]
@@ -316,7 +319,7 @@ Apache Software License, Version 2.
[14] Source available at
https://github.com/vert-x3/vertx-bridge-common/tree/3.9.8
[15] Source available at https://github.com/eclipse/vert.x/tree/3.9.8
[16] Source available at https://github.com/vert-x3/vertx-web/tree/3.9.8
-[17] Source available at http://logging.apache.org/log4j/1.2/download.html
+[17] Source available at
https://github.com/apache/logging-log4j2/tree/rel/2.14.1
[18] Source available at https://github.com/java-native-access/jna/tree/3.2.7
[19] Source available at
https://git-wip-us.apache.org/repos/asf?p=commons-collections.git;a=tag;h=a3a5ad
[20] Source available at
https://git-wip-us.apache.org/repos/asf?p=commons-lang.git;a=shortlog;h=refs/tags/LANG_3_6
@@ -631,7 +634,6 @@ MIT license. For details, see deps/slf4j-1.7.32/LICENSE.txt.
Bundled as
- lib/org.slf4j-slf4j-api-1.7.32.jar
- - lib/org.slf4j-slf4j-log4j12-1.7.32.jar
Source available at https://github.com/qos-ch/slf4j/tree/v_1.7.32
------------------------------------------------------------------------------------
This product bundles the Google Auth Library, which is available under a
"3-clause BSD"
diff --git a/bookkeeper-server/build.gradle b/bookkeeper-server/build.gradle
index b413c34..b18ed01 100644
--- a/bookkeeper-server/build.gradle
+++ b/bookkeeper-server/build.gradle
@@ -72,7 +72,9 @@ dependencies {
testImplementation depLibs.zookeeperTest
annotationProcessor depLibs.lombok
testAnnotationProcessor depLibs.lombok
- testImplementation depLibs.slf4jLog4j
+ testImplementation depLibs.log4jSlf4jImpl
+ testImplementation depLibs.log4j12api
+ testImplementation depLibs.log4jCore
}
test {
diff --git a/bookkeeper-server/pom.xml b/bookkeeper-server/pom.xml
index 6c5e025..07004c8 100644
--- a/bookkeeper-server/pom.xml
+++ b/bookkeeper-server/pom.xml
@@ -50,8 +50,16 @@
<artifactId>rocksdbjni</artifactId>
</dependency>
<dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-1.2-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
</dependency>
<dependency>
<groupId>org.apache.zookeeper</groupId>
diff --git
a/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java
b/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java
index 3ba3675..b009eae 100644
---
a/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java
+++
b/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java
@@ -52,6 +52,7 @@ import org.apache.commons.cli.HelpFormatter;
import org.apache.commons.cli.Options;
import org.apache.commons.configuration.ConfigurationException;
import org.apache.commons.io.FileUtils;
+import org.apache.log4j.ConsoleAppender;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -335,8 +336,9 @@ public class FileSystemUpgrade {
public static void main(String[] args) throws Exception {
org.apache.log4j.Logger root = org.apache.log4j.Logger.getRootLogger();
- root.addAppender(new org.apache.log4j.ConsoleAppender(
- new org.apache.log4j.PatternLayout("%-5p
[%t]: %m%n")));
+ ConsoleAppender console = new org.apache.log4j.ConsoleAppender();
+ console.setLayout(new org.apache.log4j.PatternLayout("%-5p [%t]:
%m%n"));
+ root.addAppender(console);
root.setLevel(org.apache.log4j.Level.ERROR);
org.apache.log4j.Logger.getLogger(FileSystemUpgrade.class).setLevel(
org.apache.log4j.Level.INFO);
diff --git
a/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/MdcContextTest.java
b/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/MdcContextTest.java
index 54cfaa5..51d66d8 100644
---
a/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/MdcContextTest.java
+++
b/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/MdcContextTest.java
@@ -28,11 +28,11 @@ import static org.junit.Assert.fail;
import static org.mockito.AdditionalAnswers.answerVoid;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.doAnswer;
-import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.when;
+import static org.mockito.Mockito.spy;
import java.io.File;
import java.util.Queue;
+import java.util.UUID;
import java.util.concurrent.ConcurrentLinkedQueue;
import lombok.extern.slf4j.Slf4j;
@@ -43,12 +43,10 @@ import
org.apache.bookkeeper.bookie.InterleavedLedgerStorage;
import org.apache.bookkeeper.bookie.LedgerDirsManager;
import org.apache.bookkeeper.conf.ClientConfiguration;
import org.apache.bookkeeper.test.BookKeeperClusterTestCase;
-import org.apache.log4j.Appender;
-import org.apache.log4j.Level;
-import org.apache.log4j.LogManager;
-import org.apache.log4j.Logger;
import org.apache.log4j.MDC;
-import org.apache.log4j.spi.LoggingEvent;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.LoggerContext;
+import org.apache.logging.log4j.core.appender.NullAppender;
import org.hamcrest.CoreMatchers;
import org.junit.After;
import org.junit.Assert;
@@ -69,9 +67,8 @@ public class MdcContextTest extends BookKeeperClusterTestCase
{
BookKeeper bkc;
LedgerHandle lh;
- private Appender mockAppender;
+ private NullAppender mockAppender;
private Queue<String> capturedEvents;
- private Logger rootLogger = LogManager.getRootLogger();
public MdcContextTest() {
super(3);
@@ -118,23 +115,28 @@ public class MdcContextTest extends
BookKeeperClusterTestCase {
lh = bkc.createLedgerAdv(3, 3, 3, BookKeeper.DigestType.CRC32, new
byte[] {});
MDC.clear();
- mockAppender = mock(Appender.class);
- when(mockAppender.getName()).thenReturn("MockAppender");
+ LoggerContext lc = (LoggerContext)
org.apache.logging.log4j.LogManager.getContext(false);
+ mockAppender =
spy(NullAppender.createAppender(UUID.randomUUID().toString()));
+ mockAppender.start();
+ lc.getConfiguration().addAppender(mockAppender);
+
lc.getRootLogger().addAppender(lc.getConfiguration().getAppender(mockAppender.getName()));
+
lc.getConfiguration().getRootLogger().setLevel(org.apache.logging.log4j.Level.INFO);
+ lc.updateLoggers();
- rootLogger.addAppender(mockAppender);
- rootLogger.setLevel(Level.INFO);
capturedEvents = new ConcurrentLinkedQueue<>();
- doAnswer(answerVoid((LoggingEvent event) -> capturedEvents.add(
- mdcFormat(event.getMDC(MDC_REQUEST_ID),
event.getRenderedMessage())
- ))).when(mockAppender).doAppend(any());
+ doAnswer(answerVoid((LogEvent event) -> capturedEvents.add(
+ mdcFormat(event.getContextData().getValue(MDC_REQUEST_ID),
event.getMessage().getFormattedMessage())
+ ))).when(mockAppender).append(any());
}
@After
public void tearDown() throws Exception {
lh.close();
bkc.close();
- rootLogger.removeAppender(mockAppender);
+ LoggerContext lc = (LoggerContext)
org.apache.logging.log4j.LogManager.getContext(false);
+
lc.getRootLogger().removeAppender(lc.getConfiguration().getAppender(mockAppender.getName()));
+ lc.updateLoggers();
capturedEvents = null;
MDC.clear();
super.tearDown();
diff --git
a/bookkeeper-server/src/test/java/org/apache/bookkeeper/util/LoggerOutput.java
b/bookkeeper-server/src/test/java/org/apache/bookkeeper/util/LoggerOutput.java
index eeb161e..610997c 100644
---
a/bookkeeper-server/src/test/java/org/apache/bookkeeper/util/LoggerOutput.java
+++
b/bookkeeper-server/src/test/java/org/apache/bookkeeper/util/LoggerOutput.java
@@ -21,16 +21,18 @@
package org.apache.bookkeeper.util;
import static org.mockito.Mockito.atLeastOnce;
-import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.spy;
import static org.mockito.Mockito.verify;
import java.util.ArrayList;
import java.util.List;
+import java.util.UUID;
import java.util.function.Consumer;
import java.util.stream.Collectors;
-import org.apache.log4j.Appender;
-import org.apache.log4j.LogManager;
-import org.apache.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.LoggerContext;
+import org.apache.logging.log4j.core.appender.NullAppender;
import org.junit.rules.TestRule;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
@@ -44,13 +46,13 @@ import org.slf4j.event.LoggingEvent;
*/
public class LoggerOutput implements TestRule {
- private Appender logAppender;
- private ArgumentCaptor<org.apache.log4j.spi.LoggingEvent> logEventCaptor;
- private List<Consumer<List<LoggingEvent>>> logEventExpectations = new
ArrayList<>();
+ private NullAppender logAppender;
+ private ArgumentCaptor<LogEvent> logEventCaptor;
+ private final List<Consumer<List<LoggingEvent>>> logEventExpectations =
new ArrayList<>();
public void expect(Consumer<List<LoggingEvent>> expectation) {
if (logEventCaptor == null) {
- logEventCaptor =
ArgumentCaptor.forClass(org.apache.log4j.spi.LoggingEvent.class);
+ logEventCaptor = ArgumentCaptor.forClass(LogEvent.class);
}
logEventExpectations.add(expectation);
}
@@ -61,13 +63,16 @@ public class LoggerOutput implements TestRule {
@Override
public void evaluate() throws Throwable {
- logAppender = mock(Appender.class);
- Logger rootLogger = LogManager.getRootLogger();
- rootLogger.addAppender(logAppender);
+ LoggerContext lc = (LoggerContext)
LogManager.getContext(false);
+ logAppender =
spy(NullAppender.createAppender(UUID.randomUUID().toString()));
+ logAppender.start();
+ lc.getConfiguration().addAppender(logAppender);
+
lc.getRootLogger().addAppender(lc.getConfiguration().getAppender(logAppender.getName()));
+ lc.updateLoggers();
try {
base.evaluate();
if (!logEventExpectations.isEmpty()) {
- verify(logAppender,
atLeastOnce()).doAppend(logEventCaptor.capture());
+ verify(logAppender,
atLeastOnce()).append(logEventCaptor.capture());
List<LoggingEvent> logEvents =
logEventCaptor.getAllValues().stream()
.map(LoggerOutput::toSlf4j)
.collect(Collectors.toList());
@@ -76,7 +81,8 @@ public class LoggerOutput implements TestRule {
}
}
} finally {
- rootLogger.removeAppender(logAppender);
+
lc.getRootLogger().removeAppender(lc.getConfiguration().getAppender(logAppender.getName()));
+ lc.updateLoggers();
logEventExpectations.clear();
logEventCaptor = null;
}
@@ -84,7 +90,7 @@ public class LoggerOutput implements TestRule {
};
}
- private static LoggingEvent toSlf4j(org.apache.log4j.spi.LoggingEvent
log4jEvent) {
+ private static LoggingEvent toSlf4j(LogEvent log4jEvent) {
return new LoggingEvent() {
@Override
public Level getLevel() {
@@ -113,7 +119,7 @@ public class LoggerOutput implements TestRule {
@Override
public String getMessage() {
- return log4jEvent.getRenderedMessage();
+ return log4jEvent.getMessage().getFormattedMessage();
}
@Override
@@ -128,7 +134,7 @@ public class LoggerOutput implements TestRule {
@Override
public long getTimeStamp() {
- return log4jEvent.getTimeStamp();
+ return log4jEvent.getTimeMillis();
}
@Override
diff --git a/dependencies.gradle b/dependencies.gradle
index 6806a42..a7bc5bf 100644
--- a/dependencies.gradle
+++ b/dependencies.gradle
@@ -59,7 +59,7 @@ depVersions = [
junit: "4.12",
junitFoundation: "11.0.0",
kerby: "1.1.1",
- log4j: "1.2.27",
+ log4j: "2.14.1",
lombok: "1.18.20",
lz4: "1.3.0",
mockito: "3.0.0",
@@ -126,7 +126,9 @@ depLibs = [
junit: "junit:junit:${depVersions.junit}",
junitFoundation:
"com.nordstrom.tools:junit-foundation:${depVersions.junitFoundation}",
kerbySimpleKdc: "org.apache.kerby:kerb-simplekdc:${depVersions.kerby}",
- log4j: "log4j:log4j:${depVersions.log4j}",
+ log4jSlf4jImpl:
"org.apache.logging.log4j:log4j-slf4j-impl:${depVersions.log4j}",
+ log4j12api: "org.apache.logging.log4j:log4j-1.2-api:${depVersions.log4j}",
+ log4jCore: "org.apache.logging.log4j:log4j-core:${depVersions.log4j}",
lombok: "org.projectlombok:lombok:${depVersions.lombok}",
lz4: "net.jpountz.lz4:lz4:${depVersions.lz4}",
metricsCore:
"io.dropwizard.metrics:metrics-core:${depVersions.dropwizard}",
@@ -151,7 +153,6 @@ depLibs = [
rocksDb: "org.rocksdb:rocksdbjni:${depVersions.rocksDb}",
slf4j: "org.slf4j:slf4j-api:${depVersions.slf4j}",
slf4jSimple: "org.slf4j:slf4j-simple:${depVersions.slf4j}",
- slf4jLog4j: "org.slf4j:slf4j-log4j12:${depVersions.slf4j}",
shrinkwrapImpl:
"org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-impl-maven:${depVersions.shrinkwrap}",
shrinkwrapApi:
"org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-api:${depVersions.shrinkwrap}",
snappy: "org.xerial.snappy:snappy-java:${depVersions.snappy}",
diff --git a/microbenchmarks/pom.xml b/microbenchmarks/pom.xml
index c0cfedd..dc00939 100644
--- a/microbenchmarks/pom.xml
+++ b/microbenchmarks/pom.xml
@@ -43,8 +43,16 @@
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>log4j-over-slf4j</artifactId>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-1.2-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
</dependency>
<dependency>
<groupId>org.apache.bookkeeper</groupId>
diff --git a/pom.xml b/pom.xml
index cbe49e4..cbcd017 100644
--- a/pom.xml
+++ b/pom.xml
@@ -145,6 +145,7 @@
<junit.version>4.12</junit.version>
<libthrift.version>0.14.2</libthrift.version>
<lombok.version>1.18.20</lombok.version>
+ <log4j.version>2.14.1</log4j.version>
<lz4.version>1.3.0</lz4.version>
<mockito.version>3.0.0</mockito.version>
<netty.version>4.1.68.Final</netty.version>
@@ -242,14 +243,19 @@
<version>${slf4j.version}</version>
</dependency>
<dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
- <version>${slf4j.version}</version>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-1.2-api</artifactId>
+ <version>${log4j.version}</version>
</dependency>
<dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>log4j-over-slf4j</artifactId>
- <version>${slf4j.version}</version>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ <version>${log4j.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <version>${log4j.version}</version>
</dependency>
<!-- commons dependencies -->
@@ -790,8 +796,13 @@
<scope>test</scope>
</dependency>
<dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
<scope>test</scope>
</dependency>
<dependency>
diff --git a/stats/utils/build.gradle b/stats/utils/build.gradle
index b03ced3..39ff187 100644
--- a/stats/utils/build.gradle
+++ b/stats/utils/build.gradle
@@ -33,7 +33,7 @@ dependencies {
implementation depLibs.jacksonAnnotations
implementation depLibs.jcommander
implementation depLibs.reflections
- implementation depLibs.slf4jLog4j
+ implementation depLibs.log4j12api
implementation depLibs.snakeyaml
diff --git a/stream/distributedlog/io/dlfs/pom.xml
b/stream/distributedlog/io/dlfs/pom.xml
index 7818d45..31366ed 100644
--- a/stream/distributedlog/io/dlfs/pom.xml
+++ b/stream/distributedlog/io/dlfs/pom.xml
@@ -50,6 +50,14 @@
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>log4j</groupId>
+ <artifactId>log4j</artifactId>
+ </exclusion>
</exclusions>
</dependency>
<dependency>
diff --git a/stream/server/build.gradle b/stream/server/build.gradle
index 7eb3510..76f966a 100644
--- a/stream/server/build.gradle
+++ b/stream/server/build.gradle
@@ -50,7 +50,7 @@ dependencies {
implementation depLibs.zookeeper
runtimeOnly depLibs.metricsCore
runtimeOnly depLibs.snappy
- runtimeOnly depLibs.slf4jLog4j
+ runtimeOnly depLibs.log4j12api
runtimeOnly depLibs.commonsBeanutils
runtimeOnly depLibs.vertxCore
runtimeOnly depLibs.vertxWeb
diff --git
a/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/MavenClassLoader.java
b/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/MavenClassLoader.java
index 3c83f56..2404f21 100644
---
a/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/MavenClassLoader.java
+++
b/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/MavenClassLoader.java
@@ -71,14 +71,13 @@ public class MavenClassLoader implements AutoCloseable {
Optional<String> slf4jVersion =
Arrays.stream(resolver.resolve(mainArtifact)
.withTransitivity().asResolvedArtifact())
.filter((a) -> a.getCoordinate().getGroupId().equals("org.slf4j")
- &&
a.getCoordinate().getArtifactId().equals("slf4j-log4j12"))
+ &&
a.getCoordinate().getArtifactId().equals("slf4j-1.2-api"))
.map((a) -> a.getCoordinate().getVersion())
.findFirst();
List<MavenDependency> deps = Lists.newArrayList(
MavenDependencies.createDependency(
mainArtifact, ScopeType.COMPILE, false,
-
MavenDependencies.createExclusion("org.slf4j:slf4j-log4j12"),
MavenDependencies.createExclusion("log4j:log4j")));
if (slf4jVersion.isPresent()) {
deps.add(MavenDependencies.createDependency("org.slf4j:slf4j-simple:" +
slf4jVersion.get(),
diff --git a/tests/integration/cluster/build.gradle
b/tests/integration/cluster/build.gradle
index f90ff20..16e8fe6 100644
--- a/tests/integration/cluster/build.gradle
+++ b/tests/integration/cluster/build.gradle
@@ -37,7 +37,7 @@ dependencies {
testImplementation project(':stream:clients:java:kv')
testCompileOnly depLibs.lombok
- testImplementation depLibs.slf4jLog4j
+ testImplementation depLibs.log4j12api
testImplementation depLibs.testcontainers
testImplementation depLibs.commonsConfiguration
testAnnotationProcessor depLibs.lombok
diff --git a/tools/ledger/pom.xml b/tools/ledger/pom.xml
index 8a5f074f..8becea1 100644
--- a/tools/ledger/pom.xml
+++ b/tools/ledger/pom.xml
@@ -36,8 +36,18 @@
<version>${project.parent.version}</version>
</dependency>
<dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-1.2-api</artifactId>
+ <scope>runtime</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ <scope>runtime</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
diff --git a/tools/perf/build.gradle b/tools/perf/build.gradle
index b3a68c3..b7d659e 100644
--- a/tools/perf/build.gradle
+++ b/tools/perf/build.gradle
@@ -43,7 +43,7 @@ dependencies {
implementation depLibs.commonsConfiguration
implementation depLibs.guava
implementation depLibs.jcommander
- implementation depLibs.slf4jLog4j
+ implementation depLibs.log4j12api
implementation depLibs.zookeeper
implementation depLibs.curatorFramework
implementation depLibs.protobuf
diff --git a/tools/stream/build.gradle b/tools/stream/build.gradle
index 6f23289..223f27e 100644
--- a/tools/stream/build.gradle
+++ b/tools/stream/build.gradle
@@ -41,7 +41,7 @@ dependencies {
implementation depLibs.commonsConfiguration
implementation depLibs.guava
implementation depLibs.jcommander
- implementation depLibs.slf4jLog4j
+ implementation depLibs.log4j12api
implementation depLibs.zookeeper
implementation depLibs.curatorFramework
implementation depLibs.protobuf
