This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push:
new 1f28d20 Add logging module to load log configuration dynamically
(#7377)
1f28d20 is described below
commit 1f28d2036a0581d6d891933a9cc1933c3990eea4
Author: Gao Hongtao <[email protected]>
AuthorDate: Thu Jul 29 19:55:22 2021 +0800
Add logging module to load log configuration dynamically (#7377)
* Add logging module to load log configuration dynamically
Signed-off-by: Gao Hongtao <[email protected]>
---
CHANGES.md | 4 +-
docs/en/setup/backend/dynamic-config.md | 1 +
docs/en/setup/backend/dynamical-logging.md | 52 ++++++++++++
docs/menu.yml | 2 +
.../ConfigmapConfigurationWatcherRegister.java | 5 +-
.../ConfigmapConfigWatcherRegisterTest.java | 8 +-
.../oap/server/core/CoreModuleProvider.java | 4 +
.../server/core/logging/LoggingConfigWatcher.java | 99 ++++++++++++++++++++++
.../core/logging/log4j/OapConfiguration.java | 33 ++++++++
.../logging/log4j/OapConfigurationFactory.java | 59 +++++++++++++
10 files changed, 260 insertions(+), 7 deletions(-)
diff --git a/CHANGES.md b/CHANGES.md
index e80df80..c172f6f 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -122,7 +122,9 @@ Release Notes.
* Filtering NaN value samples when build SampleFamily
* Add Thread and ClassLoader Metrics for the self-observability and
otel-oc-rules
* Simple optimization of trace sql query statement. Avoid "select *" query
method
-* Breaking Change: rename `qps` to `rpm` in LAL
+* Introduce dynamical logging to update log configuration at runtime
+* Fix Kubernetes ConfigMap configuration center doesn't send delete event
+* Breaking Change: emove `qps` and add `rpm` in LAL
#### UI
diff --git a/docs/en/setup/backend/dynamic-config.md
b/docs/en/setup/backend/dynamic-config.md
index 44b7f86..eab8442 100755
--- a/docs/en/setup/backend/dynamic-config.md
+++ b/docs/en/setup/backend/dynamic-config.md
@@ -11,6 +11,7 @@ Right now, SkyWalking supports following dynamic
configurations.
|alarm.default.alarm-settings| The alarm settings, will override
`alarm-settings.yml`. | same as [`alarm-settings.yml`](backend-alarm.md) |
|core.default.apdexThreshold| The apdex threshold settings, will override
`service-apdex-threshold.yml`. | same as
[`service-apdex-threshold.yml`](apdex-threshold.md) |
|core.default.endpoint-name-grouping| The endpoint name grouping setting, will
override `endpoint-name-grouping.yml`. | same as
[`endpoint-name-grouping.yml`](endpoint-grouping-rules.md) |
+|core.default.log4j-xml| The log4j xml configuration, will override
`log4j2.xml`. | same as [`log4j2.xml`](dynamical-logging.md) |
|agent-analyzer.default.sampleRate| Trace sampling , override
`receiver-trace/default/sampleRate` of `application.yml`. | 10000 |
|agent-analyzer.default.slowTraceSegmentThreshold| Setting this threshold
about the latency would make the slow trace segments sampled if they cost more
time, even the sampling mechanism activated. The default value is `-1`, which
means would not sample slow traces. Unit, millisecond. override
`receiver-trace/default/slowTraceSegmentThreshold` of `application.yml`. | -1 |
|configuration-discovery.default.agentConfigurations| The
ConfigurationDiscovery settings | look at
[`configuration-discovery.md`](../service-agent/java-agent/configuration-discovery.md)
|
diff --git a/docs/en/setup/backend/dynamical-logging.md
b/docs/en/setup/backend/dynamical-logging.md
new file mode 100644
index 0000000..d143167
--- /dev/null
+++ b/docs/en/setup/backend/dynamical-logging.md
@@ -0,0 +1,52 @@
+# Dynamical Logging
+
+OAP server leverage the log4j2 to manage the logging system. The log4j2
supports changing the configuration
+at the runtime, but you have to update the XML configuration file manually,
which could waste you much time and
+easy to make mistakes.
+
+The dynamical logging, which depends on the dynamic configuration, provides us
an agile way to update all OAP log4j
+configurations through a single operation.
+
+The key of the configuration item is `core.default.log4j-xml`, and you can
select any one of the configuration implements
+to store the content of log4j.xml. In the booting phase, once the core module
gets started, the `core.default.log4j-xml`
+would come into the OAP log4j context.
+
+Supposing changing the configuration after OAP started, you have to wait a
while to get the changes applied.
+The default value is `60` seconds which you could change through
`configuration.<configuration implement>.peroid` in application.yaml.
+
+If you remove `core.default.log4j-xml` from the configuration center or
disable the configuration module, the `log4j.xml`
+laid in `config` directory would get the effect.
+
+> Caveat: OAP only supports XML configuration format.
+
+There is an example to show how to config dynamical logging through a
ConfigMap in a Kubernetes cluster. Other configuration
+clusters could follow the same convention to set up.
+
+```yaml
+apiVersion: v1
+data:
+ core.default.log4j-xml: |-
+ <Configuration status="WARN">
+ <Appenders>
+ <Console name="Console" target="SYSTEM_OUT">
+ <PatternLayout charset="UTF-8" pattern="%d - %c - %L [%t] %-5p %x -
%m%n"/>
+ </Console>
+ </Appenders>
+ <Loggers>
+ <logger name="io.grpc.netty" level="INFO"/>
+ <logger name="org.apache.skywalking.oap.server.configuration.api"
level="TRACE"/>
+ <logger
name="org.apache.skywalking.oap.server.configuration.configmap" level="DEBUG"/>
+ <Root level="WARN">
+ <AppenderRef ref="Console"/>
+ </Root>
+ </Loggers>
+ </Configuration>
+kind: ConfigMap
+metadata:
+ labels:
+ app: collector
+ release: skywalking
+ name: skywalking-oap
+ namespace: default
+```
+
diff --git a/docs/menu.yml b/docs/menu.yml
index bec98b6..a778217 100644
--- a/docs/menu.yml
+++ b/docs/menu.yml
@@ -149,6 +149,8 @@ catalog:
path: "/en/setup/backend/advanced-deployment"
- name: "Data Lifecycle. Time To Live (TTL)"
path: "/en/setup/backend/ttl"
+ - name: "Dynamical Logging"
+ path: "/en/setup/backend/dynamical-logging"
- name: "Deploy In Kubernetes"
path: "/en/setup/backend/backend-k8s"
- name: "Tracing"
diff --git
a/oap-server/server-configuration/configuration-k8s-configmap/src/main/java/org/apache/skywalking/oap/server/configuration/configmap/ConfigmapConfigurationWatcherRegister.java
b/oap-server/server-configuration/configuration-k8s-configmap/src/main/java/org/apache/skywalking/oap/server/configuration/configmap/ConfigmapConfigurationWatcherRegister.java
index c6bcdbc..f3ca25c 100644
---
a/oap-server/server-configuration/configuration-k8s-configmap/src/main/java/org/apache/skywalking/oap/server/configuration/configmap/ConfigmapConfigurationWatcherRegister.java
+++
b/oap-server/server-configuration/configuration-k8s-configmap/src/main/java/org/apache/skywalking/oap/server/configuration/configmap/ConfigmapConfigurationWatcherRegister.java
@@ -19,7 +19,6 @@
package org.apache.skywalking.oap.server.configuration.configmap;
import io.kubernetes.client.openapi.models.V1ConfigMap;
-import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import lombok.extern.slf4j.Slf4j;
@@ -46,9 +45,7 @@ public class ConfigmapConfigurationWatcherRegister extends
ConfigWatcherRegister
if (log.isDebugEnabled()) {
log.debug("read config: name:{} ,value:{}", name, value);
}
- if (Objects.nonNull(value)) {
- configTable.add(new ConfigTable.ConfigItem(name, value));
- }
+ configTable.add(new ConfigTable.ConfigItem(name, value));
}
return Optional.of(configTable);
}
diff --git
a/oap-server/server-configuration/configuration-k8s-configmap/src/test/java/org/apache/skywalking/oap/server/configuration/configmap/ConfigmapConfigWatcherRegisterTest.java
b/oap-server/server-configuration/configuration-k8s-configmap/src/test/java/org/apache/skywalking/oap/server/configuration/configmap/ConfigmapConfigWatcherRegisterTest.java
index 19e5177..4612547 100644
---
a/oap-server/server-configuration/configuration-k8s-configmap/src/test/java/org/apache/skywalking/oap/server/configuration/configmap/ConfigmapConfigWatcherRegisterTest.java
+++
b/oap-server/server-configuration/configuration-k8s-configmap/src/test/java/org/apache/skywalking/oap/server/configuration/configmap/ConfigmapConfigWatcherRegisterTest.java
@@ -67,7 +67,9 @@ public class ConfigmapConfigWatcherRegisterTest {
Assert.assertTrue(optionalConfigTable.isPresent());
ConfigTable configTable = optionalConfigTable.get();
- Assert.assertEquals(configTable.getItems().size(), 0);
+ Assert.assertEquals(configTable.getItems().size(), 1);
+ Assert.assertEquals(configTable.getItems().get(0).getName(), "key1");
+ Assert.assertNull(configTable.getItems().get(0).getValue());
}
@Test
@@ -79,7 +81,9 @@ public class ConfigmapConfigWatcherRegisterTest {
Assert.assertTrue(optionalConfigTable.isPresent());
ConfigTable configTable = optionalConfigTable.get();
- Assert.assertEquals(configTable.getItems().size(), 0);
+ Assert.assertEquals(configTable.getItems().size(), 1);
+ Assert.assertEquals(configTable.getItems().get(0).getName(), "key1");
+ Assert.assertNull(configTable.getItems().get(0).getValue());
}
@Test
diff --git
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModuleProvider.java
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModuleProvider.java
index 1be58a3..d6b1b97 100755
---
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModuleProvider.java
+++
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModuleProvider.java
@@ -49,6 +49,7 @@ import
org.apache.skywalking.oap.server.core.config.NamingControl;
import org.apache.skywalking.oap.server.core.config.group.EndpointNameGrouping;
import
org.apache.skywalking.oap.server.core.config.group.EndpointNameGroupingRuleWatcher;
import
org.apache.skywalking.oap.server.core.config.group.openapi.EndpointGroupingRuleReader4Openapi;
+import org.apache.skywalking.oap.server.core.logging.LoggingConfigWatcher;
import
org.apache.skywalking.oap.server.core.management.ui.template.UITemplateInitializer;
import
org.apache.skywalking.oap.server.core.management.ui.template.UITemplateManagementService;
import org.apache.skywalking.oap.server.core.oal.rt.DisableOALDefine;
@@ -121,6 +122,7 @@ public class CoreModuleProvider extends ModuleProvider {
private ApdexThresholdConfig apdexThresholdConfig;
private EndpointNameGroupingRuleWatcher endpointNameGroupingRuleWatcher;
private OALEngineLoaderService oalEngineLoaderService;
+ private LoggingConfigWatcher loggingConfigWatcher;
public CoreModuleProvider() {
super();
@@ -307,6 +309,7 @@ public class CoreModuleProvider extends ModuleProvider {
TopNStreamProcessor.getInstance().setTopNWorkerReportCycle(moduleConfig.getTopNReportPeriod());
apdexThresholdConfig = new ApdexThresholdConfig(this);
ApdexMetrics.setDICT(apdexThresholdConfig);
+ loggingConfigWatcher = new LoggingConfigWatcher(this);
}
@Override
@@ -349,6 +352,7 @@ public class CoreModuleProvider extends ModuleProvider {
DynamicConfigurationService.class);
dynamicConfigurationService.registerConfigChangeWatcher(apdexThresholdConfig);
dynamicConfigurationService.registerConfigChangeWatcher(endpointNameGroupingRuleWatcher);
+
dynamicConfigurationService.registerConfigChangeWatcher(loggingConfigWatcher);
}
@Override
diff --git
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/LoggingConfigWatcher.java
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/LoggingConfigWatcher.java
new file mode 100644
index 0000000..6ea59e9
--- /dev/null
+++
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/LoggingConfigWatcher.java
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.core.logging;
+
+import com.google.common.base.Strings;
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.core.LoggerContext;
+import org.apache.logging.log4j.core.config.ConfigurationSource;
+import org.apache.skywalking.oap.server.configuration.api.ConfigChangeWatcher;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.logging.log4j.OapConfiguration;
+import org.apache.skywalking.oap.server.library.module.ModuleProvider;
+
+/**
+ * LoggingConfigWatcher watches the change of logging configuration. Once got
the change content, it would apply them to
+ * the current logger context.
+ */
+@Slf4j
+public class LoggingConfigWatcher extends ConfigChangeWatcher {
+ private final LoggerContext ctx;
+ private final OapConfiguration originConfiguration;
+ private volatile String content;
+
+ public LoggingConfigWatcher(final ModuleProvider provider) {
+ super(CoreModule.NAME, provider, "log4j-xml");
+ this.ctx = (LoggerContext) LogManager.getContext(false);
+ this.originConfiguration = (OapConfiguration) ctx.getConfiguration();
+ }
+
+ @Override
+ public void notify(final ConfigChangeEvent value) {
+ String newValue;
+ if (EventType.DELETE.equals(value.getEventType())) {
+ this.content = "";
+ newValue = "";
+ } else {
+ newValue = value.getNewValue();
+ }
+ try {
+ if (!reconfigure(newValue)) {
+ return;
+ }
+ } catch (Throwable t) {
+ log.error("failed to apply configuration to log4j", t);
+ return;
+ }
+ StringBuilder builder = new StringBuilder();
+ ctx.getConfiguration().getLoggers().forEach((loggerName, config) -> {
+ builder.append(Strings.isNullOrEmpty(loggerName) ? "Root" :
loggerName)
+ .append(":")
+ .append(config.getLevel())
+ .append(",");
+ });
+ this.content = builder.toString();
+ }
+
+ @Override
+ public String value() {
+ return this.content;
+ }
+
+ private boolean reconfigure(final String newValue) {
+ if (Strings.isNullOrEmpty(newValue)) {
+ if (ctx.getConfiguration().equals(originConfiguration)) {
+ return false;
+ }
+ ctx.onChange(originConfiguration);
+ return true;
+ }
+ OapConfiguration oc;
+ try {
+ oc = new OapConfiguration(ctx, new ConfigurationSource(new
ByteArrayInputStream(newValue.getBytes())));
+ } catch (IOException e) {
+ throw new RuntimeException(String.format("failed to parse %s from
configuration center", newValue), e);
+ }
+ oc.initialize();
+ ctx.onChange(oc);
+ return true;
+ }
+}
diff --git
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/log4j/OapConfiguration.java
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/log4j/OapConfiguration.java
new file mode 100644
index 0000000..a9c07f7
--- /dev/null
+++
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/log4j/OapConfiguration.java
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.core.logging.log4j;
+
+import org.apache.logging.log4j.core.LoggerContext;
+import org.apache.logging.log4j.core.config.ConfigurationSource;
+import org.apache.logging.log4j.core.config.xml.XmlConfiguration;
+
+/**
+ * OapConfiguration is the object loaded into log4j context.
+ */
+public class OapConfiguration extends XmlConfiguration {
+ public OapConfiguration(final LoggerContext loggerContext,
+ final ConfigurationSource configSource) {
+ super(loggerContext, configSource);
+ }
+}
diff --git
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/log4j/OapConfigurationFactory.java
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/log4j/OapConfigurationFactory.java
new file mode 100644
index 0000000..54b829a
--- /dev/null
+++
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/log4j/OapConfigurationFactory.java
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.core.logging.log4j;
+
+import org.apache.logging.log4j.core.LoggerContext;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.ConfigurationFactory;
+import org.apache.logging.log4j.core.config.ConfigurationSource;
+import org.apache.logging.log4j.core.config.Order;
+import org.apache.logging.log4j.core.config.plugins.Plugin;
+
+/**
+ * OapConfigurationFactory guarantee the log4j configuration object is {@link
OapConfiguration}.
+ * It guarantees the downcast {@link Configuration} to {@link
OapConfiguration} safely.
+ */
+@Plugin(name = "OapConfigurationFactory", category = "ConfigurationFactory")
+@Order(10)
+public class OapConfigurationFactory extends ConfigurationFactory {
+ /**
+ * Valid file extensions for XML files.
+ */
+ public static final String[] SUFFIXES = new String[] {".xml", "*"};
+
+ /**
+ * Returns the file suffixes for XML files.
+ * @return An array of File extensions.
+ */
+ @Override
+ public String[] getSupportedTypes() {
+ return SUFFIXES;
+ }
+
+ /**
+ * Return the configuration.
+ * @param loggerContext The logger context
+ * @param source The input source
+ * @return the configuration
+ */
+ @Override
+ public Configuration getConfiguration(final LoggerContext loggerContext,
final ConfigurationSource source) {
+ return new OapConfiguration(loggerContext, source);
+ }
+}
