This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push:
new dd6b7a1fc0 Remove Prometheuse fetcher, fix MAL Histogram type bug, and
reorganize the doc (#9710)
dd6b7a1fc0 is described below
commit dd6b7a1fc0f8581e25c86cf6c904848f0c574e92
Author: kezhenxu94 <[email protected]>
AuthorDate: Fri Sep 30 13:13:03 2022 +0800
Remove Prometheuse fetcher, fix MAL Histogram type bug, and reorganize the
doc (#9710)
* [**Breaking Change**]: Remove prometheus-fetcher plugin, please use
OpenTelemetry to scrape Prometheus metrics and
set up SkyWalking OpenTelemetry receiver instead.
* BugFix: histogram metrics sent to MAL should be treated as OpenTelemetry
style, not Prometheus style:
```
(-infinity, explicit_bounds[i]] for i == 0
(explicit_bounds[i-1], explicit_bounds[i]] for 0 < i <
size(explicit_bounds)
(explicit_bounds[i-1], +infinity) for i == size(explicit_bounds)
```
---
docs/en/changes/changes.md | 8 +
docs/en/concepts-and-designs/mal.md | 41 ++++-
docs/en/setup/backend/backend-telemetry.md | 58 +------
docs/en/setup/backend/configuration-vocabulary.md | 3 -
docs/en/setup/backend/opentelemetry-receiver.md | 4 +-
docs/en/setup/backend/prometheus-metrics.md | 84 ----------
docs/menu.yml | 2 -
.../provider/meter/process/MeterProcessor.java | 3 +-
.../oap/meter/analyzer/dsl/HistogramType.java | 35 ----
.../oap/meter/analyzer/dsl/SampleFamily.java | 18 +-
.../meter/analyzer/dsl/SampleFamilyBuilder.java | 5 -
.../oap/meter/analyzer/prometheus/rule/Rule.java | 4 -
.../analyzer/prometheus/rule/StaticConfig.java | 34 ----
.../oap/meter/analyzer/prometheus/rule/Target.java | 32 ----
.../oap/meter/analyzer/dsl/AnalyzerTest.java | 26 +--
.../oap/meter/analyzer/dsl/FunctionTest.java | 24 ++-
oap-server/server-fetcher-plugin/pom.xml | 3 +-
.../prometheus-fetcher-plugin/pom.xml | 46 ------
.../server/fetcher/prometheus/http/HTTPClient.java | 88 ----------
.../fetcher/prometheus/http/HTTPClientHandler.java | 64 -------
.../prometheus/http/HTTPClientInitializer.java | 55 ------
.../prometheus/module/PrometheusFetcherModule.java | 34 ----
.../provider/PrometheusFetcherConfig.java | 48 ------
.../provider/PrometheusFetcherProvider.java | 184 ---------------------
...ywalking.oap.server.library.module.ModuleDefine | 19 ---
...alking.oap.server.library.module.ModuleProvider | 19 ---
.../prometheus/provider/rule/RulesTest.java | 40 -----
.../resources/fetcher-prom-rules/localhost.yaml | 27 ---
.../src/test/resources/log4j2.xml | 32 ----
oap-server/server-starter/pom.xml | 5 -
.../src/main/resources/application.yml | 6 -
.../main/resources/fetcher-prom-rules/self.yaml | 101 -----------
test/e2e-v2/cases/kafka/simple-so11y/e2e.yaml | 1 -
test/e2e-v2/cases/so11y/docker-compose.yml | 12 ++
test/e2e-v2/cases/so11y/otel-collector-config.yaml | 59 +++++++
test/e2e-v2/script/docker-compose/base-compose.yml | 4 +-
36 files changed, 150 insertions(+), 1078 deletions(-)
diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md
index 04d70430ba..0b576127fa 100644
--- a/docs/en/changes/changes.md
+++ b/docs/en/changes/changes.md
@@ -30,6 +30,14 @@
we should use `Get State and Start at Most Recent` semantic instead of
`Start at Exact`
because we don't need the changing history events, see
https://kubernetes.io/docs/reference/using-api/api-concepts/#semantics-for-watch.
* Unify query services and DAOs codes time range condition to `Duration`.
+* [**Breaking Change**]: Remove prometheus-fetcher plugin, please use
OpenTelemetry to scrape Prometheus metrics and
+ set up SkyWalking OpenTelemetry receiver instead.
+* BugFix: histogram metrics sent to MAL should be treated as OpenTelemetry
style, not Prometheus style:
+ ```
+ (-infinity, explicit_bounds[i]] for i == 0
+ (explicit_bounds[i-1], explicit_bounds[i]] for 0 < i < size(explicit_bounds)
+ (explicit_bounds[i-1], +infinity) for i == size(explicit_bounds)
+ ```
#### UI
diff --git a/docs/en/concepts-and-designs/mal.md
b/docs/en/concepts-and-designs/mal.md
index 2cced6af58..36a44d19dc 100644
--- a/docs/en/concepts-and-designs/mal.md
+++ b/docs/en/concepts-and-designs/mal.md
@@ -252,6 +252,45 @@ They extract level relevant labels from metric labels,
then informs the meter-sy
- `processRelation(detect_point_label, [service_label1...],
[instance_label1...], source_process_id_label, dest_process_id_label,
component_label)` extracts `DetectPoint` labels from first argument, the label
value should be `client` or `server`.
extracts `Service` labels from the first array argument, extracts
`Instance` labels from the second array argument, extracts `ProcessID` labels
from the fourth and fifth arguments of the source and destination.
+## Configuration file
+
+The OAP can load the configuration at bootstrap. If the new configuration is
not well-formed, the OAP fails to start up. The files
+are located at `$CLASSPATH/otel-rules`, `$CLASSPATH/meter-analyzer-config`,
`$CLASSPATH/envoy-metrics-rules` and `$CLASSPATH/zabbix-rules`.
+
+The file is written in YAML format, defined by the scheme described below.
Brackets indicate that a parameter is optional.
+
+A full example can be found
[here](../../../oap-server/server-starter/src/main/resources/otel-rules/oap.yaml)
+
+Generic placeholders are defined as follows:
+
+* `<string>`: A regular string.
+* `<closure>`: A closure with custom logic.
+
+```yaml
+# initExp is the expression that initializes the current configuration file
+initExp: <string>
+# filter the metrics, only those metrics that satisfy this condition will be
passed into the `metricsRules` below.
+filter: <closure> # example: '{ tags -> tags.job_name == "vm-monitoring" }'
+# expPrefix is executed before the metrics executes other functions.
+expPrefix: <string>
+# expSuffix is appended to all expression in this file.
+expSuffix: <string>
+# insert metricPrefix into metric name: <metricPrefix>_<raw_metric_name>
+metricPrefix: <string>
+# Metrics rule allow you to recompute queries.
+metricsRules:
+ [ - <metric_rules> ]
+```
+
+### <metric_rules>
+
+```yaml
+# The name of rule, which combinates with a prefix 'meter_' as the index/table
name in storage.
+name: <string>
+# MAL expression.
+exp: <string>
+```
+
## More Examples
-Please refer to [OAP
Self-Observability](../../../oap-server/server-starter/src/main/resources/fetcher-prom-rules/self.yaml)
+Please refer to [OAP
Self-Observability](../../../oap-server/server-starter/src/main/resources/otel-rules/oap.yaml).
diff --git a/docs/en/setup/backend/backend-telemetry.md
b/docs/en/setup/backend/backend-telemetry.md
index ad50439346..efc375ad74 100644
--- a/docs/en/setup/backend/backend-telemetry.md
+++ b/docs/en/setup/backend/backend-telemetry.md
@@ -33,63 +33,15 @@ telemetry:
port: 1543
```
-2. Set up Prometheus fetcher.
+2. Set up OpenTelemetry to scrape the metrics from OAP telemetry.
-```yaml
-prometheus-fetcher:
- selector: ${SW_PROMETHEUS_FETCHER:default}
- default:
- enabledRules: ${SW_PROMETHEUS_FETCHER_ENABLED_RULES:"self"}
-```
-
-3. Make sure `config/fetcher-prom-rules/self.yaml` exists.
-
-Once you deploy an OAP server cluster, the target host should be replaced with
a dedicated IP or hostname. For instance,
-if there are three OAP servers in your cluster, their hosts are `service1`,
`service2`, and `service3`, respectively. You should
-update each `self.yaml` to switch the target host.
-
-service1:
-```yaml
-fetcherInterval: PT15S
-fetcherTimeout: PT10S
-metricsPath: /metrics
-staticConfig:
- # targets will be labeled as "instance"
- targets:
- - service1:1234
- labels:
- service: oap-server
-...
-```
+Refer to [the E2E test
case](../../../../test/e2e-v2/cases/so11y/otel-collector-config.yaml) as an
example.
-service2:
-```yaml
-fetcherInterval: PT15S
-fetcherTimeout: PT10S
-metricsPath: /metrics
-staticConfig:
- # targets will be labeled as "instance"
- targets:
- - service2:1234
- labels:
- service: oap-server
-...
-```
+For Kubernetes deployments, read the following section, otherwise you should
be able to
+adjust the configurations below to fit your scenarios.
-service3:
-```yaml
-fetcherInterval: PT15S
-fetcherTimeout: PT10S
-metricsPath: /metrics
-staticConfig:
- # targets will be labeled as "instance"
- targets:
- - service3:1234
- labels:
- service: oap-server
-...
-```
### Service discovery on Kubernetes
+
If you deploy an OAP server cluster on Kubernetes, the oap-server instance
(pod) would not have a static IP or hostname. We can leverage [OpenTelemetry
Collector](https://opentelemetry.io/docs/collector/getting-started/#kubernetes)
to discover the oap-server instance, and scrape & transfer the metrics to OAP
[OpenTelemetry receiver](opentelemetry-receiver.md).
On how to install SkyWalking on k8s, you can refer to [Apache SkyWalking
Kubernetes](https://github.com/apache/skywalking-kubernetes).
diff --git a/docs/en/setup/backend/configuration-vocabulary.md
b/docs/en/setup/backend/configuration-vocabulary.md
index 397a09460d..72c7ea74ac 100644
--- a/docs/en/setup/backend/configuration-vocabulary.md
+++ b/docs/en/setup/backend/configuration-vocabulary.md
@@ -215,9 +215,6 @@ The Configuration Vocabulary lists all available
configurations provided by `app
| - | - | kafkaConsumers
| The number of
consumers to create.
[...]
| - | - | kafkaHandlerThreadPoolSize
| Pool size of Kafka
message handler executor.
[...]
| - | - | kafkaHandlerThreadPoolQueueSize
| Queue size of Kafka
message handler executor.
[...]
-| prometheus-fetcher | default | Prometheus fetcher reads metrics
from Prometheus endpoint, and transfer the metrics into SkyWalking native
format for the MAL engine. | -
[...]
-| - | - | enabledRules
| Enabled rules.
[...]
-| - | - | maxConvertWorker
| The maximize meter
convert worker.
[...]
| kafka-fetcher | default | Read SkyWalking's native
metrics/logs/traces through Kafka server.
| -
[...]
| - | - | bootstrapServers
| A list of host/port
pairs to use for establishing the initial connection to the Kafka cluster.
[...]
| - | - | namespace
| Namespace aims to
isolate multi OAP cluster when using the same Kafka cluster. If you set a
namespace for Kafka fetcher, OAP will add a prefix to topic name. You should
also set namespace in `agent.config`. The property is named
`plugin.kafka.namespace`. [...]
diff --git a/docs/en/setup/backend/opentelemetry-receiver.md
b/docs/en/setup/backend/opentelemetry-receiver.md
index dc852a5db1..d8f78856ef 100644
--- a/docs/en/setup/backend/opentelemetry-receiver.md
+++ b/docs/en/setup/backend/opentelemetry-receiver.md
@@ -6,11 +6,11 @@ If the new configuration is not well-formed, the OAP may fail
to start up. The f
Supported handlers:
* `oc`:
[OpenCensus](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/a08903f05d3a544f548535c222b1c205b9f5a154/exporter/opencensusexporter/README.md)
gRPC service handler.
-* `otlp`:
[OpenTelemetry](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlpexporter)
gRPC service handler.
+* `otlp`:
[OpenTelemetry](https://github.com/open-telemetry/opentelemetry-collector/tree/1c217b366fbdb209044d8f4c3fece079ae23bd3b/exporter/otlpexporter)
gRPC service handler.
**Notice:** Set `SW_OTEL_RECEIVER=default` through system environment or
change `receiver-otel/selector=${SW_OTEL_RECEIVER:default}` to activate the
OpenTelemetry receiver.
-The rule file should be in YAML format, defined by the scheme described in
[prometheus-fetcher](./prometheus-metrics.md).
+The rule file should be in YAML format, defined by the scheme described in
[MAL](../../concepts-and-designs/mal.md).
Note: `receiver-otel` only supports the `group`, `defaultMetricLevel`, and
`metricsRules` nodes of the scheme due to its push mode.
To activate the `oc` handler and relevant rules of `istio`:
diff --git a/docs/en/setup/backend/prometheus-metrics.md
b/docs/en/setup/backend/prometheus-metrics.md
deleted file mode 100644
index 8fa784e542..0000000000
--- a/docs/en/setup/backend/prometheus-metrics.md
+++ /dev/null
@@ -1,84 +0,0 @@
-# Prometheus Fetcher
-Prometheus fetcher reads metrics from the Prometheus endpoint and transfers
the metrics into SkyWalking native format for the MAL engine.
-
-## Configuration file
-Prometheus fetcher is configured via a configuration file. The configuration
file defines everything related to fetching
-services and their instances, as well as which rule files to load.
-
-The OAP can load the configuration at bootstrap. If the new configuration is
not well-formed, the OAP fails to start up. The files
-are located at `$CLASSPATH/fetcher-prom-rules`.
-
-The file is written in YAML format, defined by the scheme described below.
Brackets indicate that a parameter is optional.
-
-A full example can be found
[here](../../../../oap-server/server-starter/src/main/resources/fetcher-prom-rules/self.yaml)
-
-Generic placeholders are defined as follows:
-
-* `<duration>`: This is parsed into a textual representation of a duration.
The accepted formats are based on
- the ISO-8601 duration format `PnDTnHnMn.nS` with days of exactly 24 hours.
-* `<labelname>`: A string matching the regular expression
\[a-zA-Z_\]\[a-zA-Z0-9_\]*.
-* `<labelvalue>`: A string of Unicode characters.
-* `<host>`: A valid string consisting of a hostname or IP followed by an
optional port number.
-* `<path>`: A valid URL path.
-* `<string>`: A regular string.
-
-```yaml
-# How frequently to fetch targets.
-fetcherInterval: <duration>
-# Per-fetch timeout when fetching this target.
-fetcherTimeout: <duration>
-# The HTTP resource path on which to fetch metrics from targets.
-metricsPath: <path>
-#Statically configured targets.
-staticConfig:
- # The targets specified by the static config.
- targets:
- [ - <target> ]
- # Labels assigned to all metrics fetched from the targets.
- labels:
- [ <labelname>: <labelvalue> ... ]
-# initExp is the expression that initializes the current configuration file
-initExp: <string>
-# filter the metrics, only those metrics that satisfy this condition will be
passed into the `metricsRules` below.
-filter: <closure> # example: '{ tags -> tags.job_name == "vm-monitoring" }'
-# expPrefix is executed before the metrics executes other functions.
-expPrefix: <string>
-# expSuffix is appended to all expression in this file.
-expSuffix: <string>
-# insert metricPrefix into metric name: <metricPrefix>_<raw_metric_name>
-metricPrefix: <string>
-# Metrics rule allow you to recompute queries.
-metricsRules:
- [ - <metric_rules> ]
-```
-
-### <target>
-
-```yaml
-# The url of target exporter. the format should be complied with "java.net.URI"
-url: <string>
-# The path of root CA file.
-sslCaFilePath: <string>
-```
-
-### <metric_rules>
-
-```yaml
-# The name of rule, which combinates with a prefix 'meter_' as the index/table
name in storage.
-name: <string>
-# MAL expression.
-exp: <string>
-```
-
-To know more about MAL, please refer to
[mal.md](../../concepts-and-designs/mal.md)
-
-## Active Fetcher Rules
-Suppose you want to enable some `metric-custom.yaml` files stored at
`fetcher-prom-rules`, append its name to `enabledRules` of
-`prometheus-fetcher` as follows:
-
-```yaml
-prometheus-fetcher:
- selector: ${SW_PROMETHEUS_FETCHER:default}
- default:
- enabledRules: ${SW_PROMETHEUS_FETCHER_ENABLED_RULES:"self,metric-custom"}
-```
diff --git a/docs/menu.yml b/docs/menu.yml
index 64bd586524..847c5921d4 100644
--- a/docs/menu.yml
+++ b/docs/menu.yml
@@ -123,8 +123,6 @@ catalog:
path: "/en/setup/backend/apdex-threshold"
- name: "Spring Sleuth Metrics Analysis"
path: "/en/setup/backend/spring-sleuth-setup"
- - name: "Prometheus Metrics"
- path: "/en/setup/backend/prometheus-metrics"
- name: "Alerting"
path: "/en/setup/backend/backend-alarm"
- name: "Logging"
diff --git
a/oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/meter/process/MeterProcessor.java
b/oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/meter/process/MeterProcessor.java
index 698075da05..d94afe0cdf 100644
---
a/oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/meter/process/MeterProcessor.java
+++
b/oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/meter/process/MeterProcessor.java
@@ -28,7 +28,6 @@ import
org.apache.skywalking.apm.network.language.agent.v3.MeterHistogram;
import org.apache.skywalking.apm.network.language.agent.v3.MeterSingleValue;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
-import org.apache.skywalking.oap.meter.analyzer.dsl.HistogramType;
import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
import org.apache.skywalking.oap.server.library.util.CollectionUtils;
@@ -137,7 +136,7 @@ public class MeterProcessor {
Map.Entry::getKey,
v -> SampleFamilyBuilder.newBuilder(
v.getValue().stream().map(s -> s.build(service,
serviceInstance, timestamp)).toArray(Sample[]::new)
-
).histogramType(HistogramType.ORDINARY).defaultHistogramBucketUnit(TimeUnit.MILLISECONDS).build()
+ ).defaultHistogramBucketUnit(TimeUnit.MILLISECONDS).build()
))));
} catch (Exception e) {
log.warn("Process meters failure.", e);
diff --git
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/HistogramType.java
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/HistogramType.java
deleted file mode 100644
index faf5457f0b..0000000000
---
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/HistogramType.java
+++ /dev/null
@@ -1,35 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.meter.analyzer.dsl;
-
-/**
- * Histogram type, which calculation type of bucket value
- */
-public enum HistogramType {
-
- /**
- * Such as Prometheus, Open Telemetry.
- */
- CUMULATIVE,
-
- /**
- * Such as Agent Meter.
- */
- ORDINARY
-}
diff --git
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/SampleFamily.java
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/SampleFamily.java
index 383653f73f..b30578782c 100644
---
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/SampleFamily.java
+++
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/SampleFamily.java
@@ -47,7 +47,6 @@ import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicReference;
import java.util.function.DoubleBinaryOperator;
import java.util.function.Function;
import java.util.stream.Collectors;
@@ -56,7 +55,6 @@ import com.google.common.base.Preconditions;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Maps;
-import com.google.common.util.concurrent.AtomicDouble;
import groovy.lang.Closure;
import io.vavr.Function2;
import io.vavr.Function3;
@@ -385,8 +383,6 @@ public class SampleFamily {
if (this == EMPTY) {
return EMPTY;
}
- AtomicDouble pre = new AtomicDouble();
- AtomicReference<String> preLe = new AtomicReference<>("0");
return SampleFamily.build(
this.context,
Stream.concat(
@@ -395,21 +391,16 @@ public class SampleFamily {
.filter(s -> s.labels.containsKey(le))
.sorted(Comparator.comparingDouble(s ->
Double.parseDouble(s.labels.get(le))))
.map(s -> {
- double r = this.context.histogramType ==
HistogramType.ORDINARY ? s.value : s.value - pre.get();
- pre.set(s.value);
+ double r = s.value;
ImmutableMap<String, String> ll =
ImmutableMap.<String, String>builder()
.putAll(Maps.filterKeys(s.labels,
key -> !Objects.equals(
key, le)
))
.put(
-
"le", String.valueOf(
-
(long) ((Double.parseDouble(
-
this.context.histogramType == HistogramType.ORDINARY ? s.labels
-
.get(
-
le) : preLe.get())) * scale)))
+
"le",
+
String.valueOf((long) ((Double.parseDouble(s.labels.get(le))) * scale)))
.build();
- preLe.set(s.labels.get(le));
return InternalOps.newSample(s.name, ll,
s.timestamp, r);
})
).toArray(Sample[]::new)
@@ -619,15 +610,12 @@ public class SampleFamily {
static RunningContext instance() {
return RunningContext.builder()
- .histogramType(HistogramType.CUMULATIVE)
.defaultHistogramBucketUnit(TimeUnit.SECONDS)
.build();
}
private Map<MeterEntity, Sample[]> meterSamples = new HashMap<>();
- private HistogramType histogramType;
-
private TimeUnit defaultHistogramBucketUnit;
}
diff --git
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/SampleFamilyBuilder.java
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/SampleFamilyBuilder.java
index acef00be74..2893ad6a51 100644
---
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/SampleFamilyBuilder.java
+++
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/SampleFamilyBuilder.java
@@ -36,11 +36,6 @@ public class SampleFamilyBuilder {
return new SampleFamilyBuilder(samples,
SampleFamily.RunningContext.instance());
}
- public SampleFamilyBuilder histogramType(HistogramType type) {
- this.context.setHistogramType(type);
- return this;
- }
-
public SampleFamilyBuilder defaultHistogramBucketUnit(TimeUnit unit) {
this.context.setDefaultHistogramBucketUnit(unit);
return this;
diff --git
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/prometheus/rule/Rule.java
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/prometheus/rule/Rule.java
index 68d6b0a5b9..47cca3896a 100644
---
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/prometheus/rule/Rule.java
+++
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/prometheus/rule/Rule.java
@@ -31,10 +31,6 @@ import java.util.List;
@NoArgsConstructor
public class Rule implements MetricRuleConfig {
private String name;
- private String fetcherInterval;
- private String fetcherTimeout;
- private String metricsPath;
- private StaticConfig staticConfig;
private String metricPrefix;
private String expSuffix;
private String expPrefix;
diff --git
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/prometheus/rule/StaticConfig.java
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/prometheus/rule/StaticConfig.java
deleted file mode 100644
index 8ab9faed31..0000000000
---
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/prometheus/rule/StaticConfig.java
+++ /dev/null
@@ -1,34 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.meter.analyzer.prometheus.rule;
-
-import java.util.List;
-import java.util.Map;
-import lombok.Data;
-import lombok.NoArgsConstructor;
-
-/**
- * StaticConfig introduces the fetching targets.
- */
-@Data
-@NoArgsConstructor
-public class StaticConfig {
- private List<Target> targets;
- private Map<String, String> labels;
-}
diff --git
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/prometheus/rule/Target.java
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/prometheus/rule/Target.java
deleted file mode 100644
index efa244ff0a..0000000000
---
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/prometheus/rule/Target.java
+++ /dev/null
@@ -1,32 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.meter.analyzer.prometheus.rule;
-
-import lombok.Data;
-import lombok.NoArgsConstructor;
-
-/**
- * Target is the HTTP fetching endpoint.
- */
-@Data
-@NoArgsConstructor
-public class Target {
- private String url;
- private String sslCaFilePath;
-}
diff --git
a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/dsl/AnalyzerTest.java
b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/dsl/AnalyzerTest.java
index 36419803b7..b728210004 100644
---
a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/dsl/AnalyzerTest.java
+++
b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/dsl/AnalyzerTest.java
@@ -200,13 +200,6 @@ public class AnalyzerTest {
.labels(of("le", "0.75", "service", "service1",
"instance", "instance2"))
.value(122)
.name("instance_cpu_percentage")
- .build(),
- Sample.builder()
- .labels(of("le", String.valueOf(Integer.MAX_VALUE),
"service", "service1", "instance",
- "instance2"
- ))
- .value(410)
- .name("instance_cpu_percentage")
.build()
).build()
);
@@ -215,11 +208,10 @@ public class AnalyzerTest {
doAnswer(invocationOnMock -> {
AvgHistogramPercentileFunction actValue =
(AvgHistogramPercentileFunction) invocationOnMock.getArgument(
0, AcceptableValue.class);
- if (actValue.getSummation().hasKey("instance1:0")) {
+ if (actValue.getSummation().hasKey("instance1:25")) {
actValues.put("instance1", actValue);
} else {
actValues.put("instance2", actValue);
-
}
return null;
}).when(meterSystem).doStreamingCalculation(any());
@@ -240,14 +232,12 @@ public class AnalyzerTest {
});
AvgHistogramPercentileFunction instance1 = actValues.get("instance1");
AvgHistogramPercentileFunction instance2 = actValues.get("instance2");
- Assert.assertEquals(100L, instance1.getSummation().get("instance1:0"),
0.0);
- Assert.assertEquals(178L,
instance1.getSummation().get("instance1:750"), 0.0);
- Assert.assertEquals(1L, instance1.getCount().get("instance1:0"), 0.0);
- Assert.assertEquals(1L, instance1.getCount().get("instance1:750"),
0.0);
-
- Assert.assertEquals(22L, instance2.getSummation().get("instance2:25"),
0.0);
- Assert.assertEquals(110L,
instance2.getSummation().get("instance2:1250"), 0.0);
- Assert.assertEquals(1L, instance2.getCount().get("instance2:25"), 0.0);
- Assert.assertEquals(1L, instance2.getCount().get("instance2:1250"),
0.0);
+ Assert.assertEquals(100L,
instance1.getSummation().get("instance1:25"), 0.0);
+ Assert.assertEquals(300L,
instance1.getSummation().get("instance1:1250"), 0.0);
+ Assert.assertEquals(1L, instance1.getCount().get("instance1:25"), 0.0);
+ Assert.assertEquals(1L, instance1.getCount().get("instance1:1250"),
0.0);
+
+ Assert.assertEquals(122L,
instance2.getSummation().get("instance2:750"), 0.0);
+ Assert.assertEquals(1L, instance2.getCount().get("instance2:750"),
0.0);
}
}
diff --git
a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/dsl/FunctionTest.java
b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/dsl/FunctionTest.java
index e9f72cbc4a..568aa36f8a 100644
---
a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/dsl/FunctionTest.java
+++
b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/dsl/FunctionTest.java
@@ -93,16 +93,14 @@ public class FunctionTest {
"histogram",
of("http_success_request", SampleFamilyBuilder.newBuilder(
Sample.builder().labels(of("le",
"0.025")).value(100).name("http_success_request").build(),
- Sample.builder().labels(of("le",
"1.25")).value(300).name("http_success_request").build(),
- Sample.builder().labels(of("le",
"0.75")).value(122).name("http_success_request").build(),
- Sample.builder().labels(of("le",
String.valueOf(Integer.MAX_VALUE))).value(410).name("http_success_request").build()).build()
+ Sample.builder().labels(of("le",
"0.75")).value(12).name("http_success_request").build(),
+ Sample.builder().labels(of("le",
"1.25")).value(36).name("http_success_request").build()).build()
),
"http_success_request.histogram()",
Result.success(SampleFamilyBuilder.newBuilder(
- Sample.builder().labels(of("le",
"0")).value(100).name("http_success_request").build(),
- Sample.builder().labels(of("le",
"25")).value(22).name("http_success_request").build(),
- Sample.builder().labels(of("le",
"750")).value(178).name("http_success_request").build(),
- Sample.builder().labels(of("le",
"1250")).value(110).name("http_success_request").build()).build()
+ Sample.builder().labels(of("le",
"25")).value(100).name("http_success_request").build(),
+ Sample.builder().labels(of("le",
"750")).value(12).name("http_success_request").build(),
+ Sample.builder().labels(of("le",
"1250")).value(36).name("http_success_request").build()).build()
),
false,
},
@@ -110,16 +108,14 @@ public class FunctionTest {
"histogram_percentile",
of("http_success_request", SampleFamilyBuilder.newBuilder(
Sample.builder().labels(of("le",
"0.025")).value(100).name("http_success_request").build(),
- Sample.builder().labels(of("le",
"1.25")).value(300).name("http_success_request").build(),
- Sample.builder().labels(of("le",
"0.75")).value(122).name("http_success_request").build(),
- Sample.builder().labels(of("le",
String.valueOf(Integer.MAX_VALUE))).value(410).name("http_success_request").build()).build()
+ Sample.builder().labels(of("le",
"0.75")).value(22).name("http_success_request").build(),
+ Sample.builder().labels(of("le",
"1.25")).value(30).name("http_success_request").build()).build()
),
"http_success_request.histogram().histogram_percentile([75,99])",
Result.success(SampleFamilyBuilder.newBuilder(
- Sample.builder().labels(of("le",
"0")).value(100).name("http_success_request").build(),
- Sample.builder().labels(of("le",
"25")).value(22).name("http_success_request").build(),
- Sample.builder().labels(of("le",
"750")).value(178).name("http_success_request").build(),
- Sample.builder().labels(of("le",
"1250")).value(110).name("http_success_request").build()).build()
+ Sample.builder().labels(of("le",
"25")).value(100).name("http_success_request").build(),
+ Sample.builder().labels(of("le",
"750")).value(22).name("http_success_request").build(),
+ Sample.builder().labels(of("le",
"1250")).value(30).name("http_success_request").build()).build()
),
false,
},
diff --git a/oap-server/server-fetcher-plugin/pom.xml
b/oap-server/server-fetcher-plugin/pom.xml
index ab4cdcd4f5..691b63878d 100644
--- a/oap-server/server-fetcher-plugin/pom.xml
+++ b/oap-server/server-fetcher-plugin/pom.xml
@@ -29,7 +29,6 @@
<packaging>pom</packaging>
<modules>
- <module>prometheus-fetcher-plugin</module>
<module>kafka-fetcher-plugin</module>
</modules>
@@ -39,4 +38,4 @@
<artifactId>vavr</artifactId>
</dependency>
</dependencies>
-</project>
\ No newline at end of file
+</project>
diff --git a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/pom.xml
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/pom.xml
deleted file mode 100644
index 22008226ef..0000000000
--- a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/pom.xml
+++ /dev/null
@@ -1,46 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- ~
- -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
- <parent>
- <artifactId>server-fetcher-plugin</artifactId>
- <groupId>org.apache.skywalking</groupId>
- <version>9.3.0-SNAPSHOT</version>
- </parent>
- <modelVersion>4.0.0</modelVersion>
-
- <artifactId>prometheus-fetcher-plugin</artifactId>
-
- <dependencies>
- <dependency>
- <groupId>org.apache.skywalking</groupId>
- <artifactId>server-core</artifactId>
- <version>${project.version}</version>
- </dependency>
- <dependency>
- <groupId>org.apache.skywalking</groupId>
- <artifactId>meter-analyzer</artifactId>
- <version>${project.version}</version>
- </dependency>
- <dependency>
- <groupId>org.yaml</groupId>
- <artifactId>snakeyaml</artifactId>
- </dependency>
- </dependencies>
-</project>
\ No newline at end of file
diff --git
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/http/HTTPClient.java
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/http/HTTPClient.java
deleted file mode 100644
index bce1a45ced..0000000000
---
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/http/HTTPClient.java
+++ /dev/null
@@ -1,88 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.fetcher.prometheus.http;
-
-import io.netty.bootstrap.Bootstrap;
-import io.netty.buffer.Unpooled;
-import io.netty.channel.Channel;
-import io.netty.channel.EventLoopGroup;
-import io.netty.channel.nio.NioEventLoopGroup;
-import io.netty.channel.socket.nio.NioSocketChannel;
-import io.netty.handler.codec.http.DefaultFullHttpRequest;
-import io.netty.handler.codec.http.HttpHeaderNames;
-import io.netty.handler.codec.http.HttpHeaderValues;
-import io.netty.handler.codec.http.HttpMethod;
-import io.netty.handler.codec.http.HttpRequest;
-import io.netty.handler.codec.http.HttpVersion;
-import java.net.URI;
-import java.net.URISyntaxException;
-import java.util.Objects;
-import java.util.concurrent.BlockingQueue;
-import java.util.concurrent.SynchronousQueue;
-import java.util.concurrent.TimeUnit;
-import lombok.Builder;
-import
org.apache.skywalking.oap.server.library.server.ssl.HTTPDynamicSslContext;
-
-@Builder
-public class HTTPClient {
-
- private final String url;
-
- private final String caFilePath;
-
- public String request() throws URISyntaxException, InterruptedException {
- URI uri = new URI(url);
- String scheme = uri.getScheme() == null ? "http" : uri.getScheme();
- String host = uri.getHost() == null ? "127.0.0.1" : uri.getHost();
- int port = uri.getPort();
-
- // Configure SSL context if necessary.
- final boolean ssl = "https".equalsIgnoreCase(scheme);
- final HTTPDynamicSslContext sslCtx = ssl ?
HTTPDynamicSslContext.forClient(caFilePath) : null;
-
- // Configure the client.
- EventLoopGroup group = new NioEventLoopGroup();
- BlockingQueue<String> channel = new SynchronousQueue<>();
- try {
- Bootstrap b = new Bootstrap();
- b.group(group)
- .channel(NioSocketChannel.class)
- .handler(new HTTPClientInitializer(sslCtx, channel));
-
- // Make the connection attempt.
- Channel ch = b.connect(host, port).sync().channel();
-
- // Prepare the HTTP request.
- HttpRequest request = new DefaultFullHttpRequest(
- HttpVersion.HTTP_1_1, HttpMethod.GET, uri.getRawPath(),
Unpooled.EMPTY_BUFFER);
- request.headers().set(HttpHeaderNames.HOST, host);
- request.headers().set(HttpHeaderNames.CONNECTION,
HttpHeaderValues.CLOSE);
- request.headers().set(HttpHeaderNames.ACCEPT_ENCODING,
HttpHeaderValues.TEXT_PLAIN);
-
- // Send the HTTP request.
- ch.writeAndFlush(request);
-
- return Objects.requireNonNull(channel.poll(10, TimeUnit.SECONDS),
"Request timeout");
- } finally {
- // Shut down executor threads to exit.
- group.shutdownGracefully();
- }
- }
-
-}
diff --git
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/http/HTTPClientHandler.java
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/http/HTTPClientHandler.java
deleted file mode 100644
index c929711b4d..0000000000
---
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/http/HTTPClientHandler.java
+++ /dev/null
@@ -1,64 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.fetcher.prometheus.http;
-
-import io.netty.channel.ChannelHandlerContext;
-import io.netty.channel.SimpleChannelInboundHandler;
-import io.netty.handler.codec.http.HttpContent;
-import io.netty.handler.codec.http.HttpObject;
-import io.netty.handler.codec.http.HttpResponse;
-import io.netty.handler.codec.http.LastHttpContent;
-import io.netty.util.CharsetUtil;
-import java.util.concurrent.BlockingQueue;
-import lombok.RequiredArgsConstructor;
-import lombok.extern.slf4j.Slf4j;
-
-@RequiredArgsConstructor
-@Slf4j
-public class HTTPClientHandler extends SimpleChannelInboundHandler<HttpObject>
{
-
- private final BlockingQueue<String> channel;
-
- private final StringBuilder buf = new StringBuilder();
-
- @Override
- public void channelRead0(ChannelHandlerContext ctx, HttpObject msg) {
- if (msg instanceof HttpResponse) {
- buf.setLength(0);
- }
- if (msg instanceof HttpContent) {
- HttpContent content = (HttpContent) msg;
- buf.append(content.content().toString(CharsetUtil.UTF_8));
- if (content instanceof LastHttpContent) {
- try {
- channel.put(buf.toString());
- } catch (InterruptedException e) {
- ctx.fireExceptionCaught(e);
- }
- ctx.close();
- }
- }
- }
-
- @Override
- public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
- log.error("HTTP request error", cause);
- ctx.close();
- }
-}
diff --git
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/http/HTTPClientInitializer.java
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/http/HTTPClientInitializer.java
deleted file mode 100644
index 8e6773f443..0000000000
---
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/http/HTTPClientInitializer.java
+++ /dev/null
@@ -1,55 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.fetcher.prometheus.http;
-
-import io.netty.channel.ChannelInitializer;
-import io.netty.channel.ChannelPipeline;
-import io.netty.channel.socket.SocketChannel;
-import io.netty.handler.codec.http.HttpClientCodec;
-import io.netty.handler.codec.http.HttpContentDecompressor;
-import io.netty.handler.ssl.SslContext;
-import java.util.concurrent.BlockingQueue;
-
-public class HTTPClientInitializer extends ChannelInitializer<SocketChannel> {
-
- private final SslContext sslCtx;
- private final BlockingQueue<String> channel;
-
- public HTTPClientInitializer(SslContext sslCtx, BlockingQueue<String>
channel) {
- this.sslCtx = sslCtx;
- this.channel = channel;
- }
-
- @Override
- public void initChannel(SocketChannel ch) {
- ChannelPipeline p = ch.pipeline();
-
- // Enable HTTPS if necessary.
- if (sslCtx != null) {
- p.addLast(sslCtx.newHandler(ch.alloc()));
- }
-
- p.addLast(new HttpClientCodec());
-
- // Remove the following line if you don't want automatic content
decompression.
- p.addLast(new HttpContentDecompressor());
-
- p.addLast(new HTTPClientHandler(channel));
- }
-}
diff --git
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/module/PrometheusFetcherModule.java
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/module/PrometheusFetcherModule.java
deleted file mode 100644
index 356a784304..0000000000
---
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/module/PrometheusFetcherModule.java
+++ /dev/null
@@ -1,34 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.fetcher.prometheus.module;
-
-import org.apache.skywalking.oap.server.library.module.ModuleDefine;
-
-public class PrometheusFetcherModule extends ModuleDefine {
- public static final String NAME = "prometheus-fetcher";
-
- public PrometheusFetcherModule() {
- super(NAME);
- }
-
- @Override
- public Class[] services() {
- return new Class[0];
- }
-}
diff --git
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/provider/PrometheusFetcherConfig.java
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/provider/PrometheusFetcherConfig.java
deleted file mode 100644
index dc383c1f42..0000000000
---
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/provider/PrometheusFetcherConfig.java
+++ /dev/null
@@ -1,48 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.fetcher.prometheus.provider;
-
-import java.util.Arrays;
-import java.util.List;
-import java.util.Optional;
-import java.util.stream.Collectors;
-import lombok.Getter;
-import org.apache.skywalking.oap.server.library.util.StringUtil;
-import org.apache.skywalking.oap.server.library.module.ModuleConfig;
-
-@Getter
-public class PrometheusFetcherConfig extends ModuleConfig {
-
- private int maxConvertWorker;
-
- private String enabledRules;
-
- private final String rulePath = "fetcher-prom-rules";
-
- List<String> getEnabledRules() {
- return
Arrays.stream(Optional.ofNullable(enabledRules).orElse("").split(","))
- .map(String::trim)
- .filter(StringUtil::isNotEmpty)
- .collect(Collectors.toList());
- }
-
- public int getMaxConvertWorker() {
- return maxConvertWorker <= 0 ? Math.max(1,
Runtime.getRuntime().availableProcessors() / 2) : maxConvertWorker;
- }
-}
diff --git
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/provider/PrometheusFetcherProvider.java
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/provider/PrometheusFetcherProvider.java
deleted file mode 100644
index b4af4597d3..0000000000
---
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/prometheus/provider/PrometheusFetcherProvider.java
+++ /dev/null
@@ -1,184 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.fetcher.prometheus.provider;
-
-import com.google.common.collect.Maps;
-import io.vavr.CheckedFunction1;
-import java.io.ByteArrayInputStream;
-import java.io.InputStream;
-import java.net.URI;
-import java.time.Duration;
-import java.util.ArrayList;
-import java.util.Collection;
-import java.util.List;
-import java.util.Map;
-import java.util.Objects;
-import java.util.concurrent.Executors;
-import java.util.concurrent.ScheduledExecutorService;
-import java.util.concurrent.TimeUnit;
-import lombok.extern.slf4j.Slf4j;
-import org.apache.commons.codec.Charsets;
-import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
-import
org.apache.skywalking.oap.meter.analyzer.prometheus.PrometheusMetricConverter;
-import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule;
-import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rules;
-import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.StaticConfig;
-import org.apache.skywalking.oap.server.core.CoreModule;
-import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
-import org.apache.skywalking.oap.server.fetcher.prometheus.http.HTTPClient;
-import
org.apache.skywalking.oap.server.fetcher.prometheus.module.PrometheusFetcherModule;
-import org.apache.skywalking.oap.server.library.module.ModuleConfig;
-import org.apache.skywalking.oap.server.library.module.ModuleDefine;
-import org.apache.skywalking.oap.server.library.module.ModuleProvider;
-import org.apache.skywalking.oap.server.library.module.ModuleStartException;
-import
org.apache.skywalking.oap.server.library.module.ServiceNotProvidedException;
-import
org.apache.skywalking.oap.server.library.server.pool.CustomThreadFactory;
-import org.apache.skywalking.oap.server.library.util.prometheus.Parser;
-import org.apache.skywalking.oap.server.library.util.prometheus.Parsers;
-import org.apache.skywalking.oap.server.library.util.prometheus.metrics.Metric;
-import
org.apache.skywalking.oap.server.library.util.prometheus.metrics.MetricFamily;
-import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
-import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
-import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
-import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
-import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
-
-@Slf4j
-public class PrometheusFetcherProvider extends ModuleProvider {
-
- private final PrometheusFetcherConfig config;
-
- private List<Rule> rules;
-
- private ScheduledExecutorService ses;
-
- private HistogramMetrics histogram;
-
- private CounterMetrics errorCounter;
-
- public PrometheusFetcherProvider() {
- config = new PrometheusFetcherConfig();
- }
-
- @Override
- public String name() {
- return "default";
- }
-
- @Override
- public Class<? extends ModuleDefine> module() {
- return PrometheusFetcherModule.class;
- }
-
- @Override
- public ModuleConfig createConfigBeanIfAbsent() {
- return config;
- }
-
- @Override
- public void prepare() throws ServiceNotProvidedException,
ModuleStartException {
- rules = Rules.loadRules(config.getRulePath(),
config.getEnabledRules());
- ses = Executors.newScheduledThreadPool(
- Math.min(rules.size(), config.getMaxConvertWorker()),
- new CustomThreadFactory("meter-converter")
- );
- }
-
- @Override
- public void start() throws ServiceNotProvidedException,
ModuleStartException {
- MetricsCreator metricsCreator = getManager().find(TelemetryModule.NAME)
- .provider()
- .getService(MetricsCreator.class);
- histogram = metricsCreator.createHistogramMetric(
- "metrics_fetcher_latency", "The process latency of metrics
scraping",
- MetricsTag.EMPTY_KEY, MetricsTag.EMPTY_VALUE
- );
- errorCounter =
metricsCreator.createCounter("metrics_fetcher_error_count", "The error number
of metrics scraping",
- MetricsTag.EMPTY_KEY, MetricsTag.EMPTY_VALUE
- );
- }
-
- @Override
- public void notifyAfterCompleted() throws ServiceNotProvidedException,
ModuleStartException {
- if (rules.isEmpty()) {
- return;
- }
- final MeterSystem service =
getManager().find(CoreModule.NAME).provider().getService(MeterSystem.class);
- rules.forEach(r -> {
- ses.scheduleAtFixedRate(new Runnable() {
-
- private final PrometheusMetricConverter converter = new
PrometheusMetricConverter(r, service);
-
- @Override public void run() {
- try (HistogramMetrics.Timer ignored =
histogram.createTimer()) {
- if (Objects.isNull(r.getStaticConfig())) {
- return;
- }
- StaticConfig sc = r.getStaticConfig();
- long now = System.currentTimeMillis();
- converter.toMeter(sc.getTargets().stream()
- .map(CheckedFunction1.liftTry(target -> {
- URI url = new URI(target.getUrl());
- URI targetURL =
url.resolve(r.getMetricsPath());
- String content =
HTTPClient.builder().url(targetURL.toString()).caFilePath(target.getSslCaFilePath()).build().request();
- List<Metric> result = new ArrayList<>();
- try (InputStream targetStream = new
ByteArrayInputStream(content.getBytes(Charsets.UTF_8))) {
- Parser p = Parsers.text(targetStream);
- MetricFamily mf;
- while ((mf = p.parse(now)) != null) {
- mf.getMetrics().forEach(metric -> {
- if
(Objects.isNull(sc.getLabels())) {
- return;
- }
- Map<String, String>
extraLabels = Maps.newHashMap(sc.getLabels());
- extraLabels.put("instance",
target.getUrl());
- extraLabels.forEach((key,
value) -> {
- if
(metric.getLabels().containsKey(key)) {
-
metric.getLabels().put("exported_" + key, metric.getLabels().get(key));
- }
-
metric.getLabels().put(key, value);
- });
- });
- result.addAll(mf.getMetrics());
- }
- }
- if (log.isDebugEnabled()) {
- log.debug("Fetch metrics from
prometheus: {}", result);
- }
- return result;
- }))
- .flatMap(tryIt -> MetricConvert.log(tryIt,
"Load metric"))
- .flatMap(Collection::stream));
- } catch (Exception e) {
- errorCounter.inc();
- log.error(e.getMessage(), e);
- }
- }
- }, 0L, Duration.parse(r.getFetcherInterval()).getSeconds(),
TimeUnit.SECONDS);
- });
- }
-
- @Override
- public String[] requiredModules() {
- return new String[] {
- TelemetryModule.NAME,
- CoreModule.NAME
- };
- }
-}
diff --git
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleDefine
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleDefine
deleted file mode 100644
index 9d67848753..0000000000
---
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleDefine
+++ /dev/null
@@ -1,19 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-#
-
-org.apache.skywalking.oap.server.fetcher.prometheus.module.PrometheusFetcherModule
\ No newline at end of file
diff --git
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider
deleted file mode 100644
index a14eab44df..0000000000
---
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider
+++ /dev/null
@@ -1,19 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-#
-
-org.apache.skywalking.oap.server.fetcher.prometheus.provider.PrometheusFetcherProvider
\ No newline at end of file
diff --git
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/test/java/org/apache/skywalking/oap/server/fetcher/prometheus/provider/rule/RulesTest.java
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/test/java/org/apache/skywalking/oap/server/fetcher/prometheus/provider/rule/RulesTest.java
deleted file mode 100644
index f11c04c51f..0000000000
---
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/test/java/org/apache/skywalking/oap/server/fetcher/prometheus/provider/rule/RulesTest.java
+++ /dev/null
@@ -1,40 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.fetcher.prometheus.provider.rule;
-
-import java.util.Collections;
-import java.util.List;
-import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule;
-import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rules;
-import org.apache.skywalking.oap.server.library.module.ModuleStartException;
-import org.junit.Test;
-
-import static org.hamcrest.MatcherAssert.assertThat;
-import static org.hamcrest.core.Is.is;
-
-public class RulesTest {
-
- @Test
- public void testFetcherPrometheusRulesLoader() throws ModuleStartException
{
- List<Rule> rr = Rules.loadRules("fetcher-prom-rules",
Collections.singletonList("localhost"));
-
- assertThat(rr.size(), is(1));
- }
-
-}
\ No newline at end of file
diff --git
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/test/resources/fetcher-prom-rules/localhost.yaml
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/test/resources/fetcher-prom-rules/localhost.yaml
deleted file mode 100644
index 7ed00f0e17..0000000000
---
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/test/resources/fetcher-prom-rules/localhost.yaml
+++ /dev/null
@@ -1,27 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-fetcherInterval: PT2S
-fetcherTimeout: PT10S
-metricsPath: /metrics
-staticConfig:
- # targets will be labeled as "instance"
- targets:
- - url: http://localhost:1234
- labels:
- app: test-oap
-metricsRules:
- - name: instance_cpu_percentage
- exp: (instance_cpu_percentage * 100).rate("PT1M").instance(["app"],
["instance"])
diff --git
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/test/resources/log4j2.xml
b/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/test/resources/log4j2.xml
deleted file mode 100644
index 8ba63b3c44..0000000000
---
a/oap-server/server-fetcher-plugin/prometheus-fetcher-plugin/src/test/resources/log4j2.xml
+++ /dev/null
@@ -1,32 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- ~
- -->
-
-<Configuration status="WARN">
- <Appenders>
- <Console name="Console" target="SYSTEM_OUT">
- <PatternLayout charset="UTF-8" pattern="%d - %c -%-4r [%t] %-5p %x
- %m%n"/>
- </Console>
- </Appenders>
- <Loggers>
- <logger name="org.apache.skywalking.oap.server.fetcher" level="DEBUG"/>
- <Root level="INFO">
- <AppenderRef ref="Console"/>
- </Root>
- </Loggers>
-</Configuration>
diff --git a/oap-server/server-starter/pom.xml
b/oap-server/server-starter/pom.xml
index 4bc5b871da..219b1dadeb 100644
--- a/oap-server/server-starter/pom.xml
+++ b/oap-server/server-starter/pom.xml
@@ -169,11 +169,6 @@
<!-- receiver module -->
<!-- fetcher module -->
- <dependency>
- <groupId>org.apache.skywalking</groupId>
- <artifactId>prometheus-fetcher-plugin</artifactId>
- <version>${project.version}</version>
- </dependency>
<dependency>
<groupId>org.apache.skywalking</groupId>
<artifactId>kafka-fetcher-plugin</artifactId>
diff --git a/oap-server/server-starter/src/main/resources/application.yml
b/oap-server/server-starter/src/main/resources/application.yml
index 42a62d37a2..a182ab6c0b 100644
--- a/oap-server/server-starter/src/main/resources/application.yml
+++ b/oap-server/server-starter/src/main/resources/application.yml
@@ -325,12 +325,6 @@ envoy-metric:
# Be careful, when using environment variables to pass this configuration,
use single quotes(`''`) to avoid it being evaluated by the shell.
k8sServiceNameRule:
${K8S_SERVICE_NAME_RULE:"${pod.metadata.labels.(service.istio.io/canonical-name)}"}
-prometheus-fetcher:
- selector: ${SW_PROMETHEUS_FETCHER:-}
- default:
- enabledRules: ${SW_PROMETHEUS_FETCHER_ENABLED_RULES:"self"}
- maxConvertWorker: ${SW_PROMETHEUS_FETCHER_NUM_CONVERT_WORKER:-1}
-
kafka-fetcher:
selector: ${SW_KAFKA_FETCHER:-}
default:
diff --git
a/oap-server/server-starter/src/main/resources/fetcher-prom-rules/self.yaml
b/oap-server/server-starter/src/main/resources/fetcher-prom-rules/self.yaml
deleted file mode 100644
index ea85506a56..0000000000
--- a/oap-server/server-starter/src/main/resources/fetcher-prom-rules/self.yaml
+++ /dev/null
@@ -1,101 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# This will parse a textual representation of a duration. The formats
-# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
-# with days considered to be exactly 24 hours.
-# <p>
-# Examples:
-# <pre>
-# "PT20.345S" -- parses as "20.345 seconds"
-# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
-# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
-# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400
seconds)
-# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
-# "P-6H3M" -- parses as "-6 hours and +3 minutes"
-# "-P6H3M" -- parses as "-6 hours and -3 minutes"
-# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
-# </pre>
-fetcherInterval: PT15S
-fetcherTimeout: PT10S
-metricsPath: /metrics
-staticConfig:
- # targets will be labeled as "instance"
- targets:
- - url: http://localhost:1234
- sslCaFilePath:
- labels:
- service: oap-server
-expSuffix: instance(['service'], ['instance'], Layer.SO11Y_OAP)
-metricPrefix: meter_oap
-metricsRules:
- - name: instance_cpu_percentage
- exp: (process_cpu_seconds_total * 100).sum(['service',
'instance']).rate('PT1M')
- - name: instance_jvm_memory_bytes_used
- exp: jvm_memory_bytes_used.sum(['service', 'instance'])
- - name: instance_jvm_gc_count
- exp: "jvm_gc_collection_seconds_count.tagMatch('gc', 'PS
Scavenge|Copy|ParNew|G1 Young Generation|PS
MarkSweep|MarkSweepCompact|ConcurrentMarkSweep|G1 Old Generation')
- .sum(['service', 'instance', 'gc']).increase('PT1M')
- .tag({tags -> if (tags['gc'] == 'PS Scavenge' || tags['gc'] == 'Copy' ||
tags['gc'] == 'ParNew' || tags['gc'] == 'G1 Young Generation') {tags.gc =
'young_gc_count'} })
- .tag({tags -> if (tags['gc'] == 'PS MarkSweep' || tags['gc'] ==
'MarkSweepCompact' || tags['gc'] == 'ConcurrentMarkSweep' || tags['gc'] == 'G1
Old Generation') {tags.gc = 'old_gc_count'} })"
- - name: instance_jvm_young_gc_time
- exp: jvm_gc_collection_seconds_sum.tagMatch('gc', 'PS
Scavenge|Copy|ParNew|G1 Young Generation').sum(['service',
'instance']).increase('PT1M') * 1000
- - name: instance_jvm_old_gc_time
- exp: jvm_gc_collection_seconds_sum.tagMatch('gc', 'PS
MarkSweep|MarkSweepCompact|ConcurrentMarkSweep|G1 Old
Generation').sum(['service', 'instance']).increase('PT1M') * 1000
- - name: instance_trace_count
- exp: trace_in_latency_count.sum(['service', 'instance']).increase('PT1M')
- - name: instance_trace_latency_percentile
- exp: trace_in_latency.sum(['le', 'service',
'instance']).increase('PT1M').histogram().histogram_percentile([50,70,90,99])
- - name: instance_trace_analysis_error_count
- exp: trace_analysis_error_count.sum(['service',
'instance']).increase('PT1M')
- - name: instance_mesh_count
- exp: mesh_analysis_latency_count.sum(['service',
'instance']).increase('PT1M')
- - name: instance_mesh_latency_percentile
- exp: mesh_analysis_latency.sum(['le', 'service',
'instance']).increase('PT1M').histogram().histogram_percentile([50,70,90,99])
- - name: instance_mesh_analysis_error_count
- exp: mesh_analysis_error_count.sum(['service',
'instance']).increase('PT1M')
- - name: instance_metrics_aggregation
- exp: "metrics_aggregation.tagEqual('dimensionality',
'minute').sum(['service', 'instance', 'level']).increase('PT1M')
- .tag({tags -> if (tags['level'] == '1') {tags.level = 'L1 aggregation'}
}).tag({tags -> if (tags['level'] == '2') {tags.level = 'L2 aggregation'} })"
- - name: instance_persistence_execute_percentile
- exp: persistence_timer_bulk_execute_latency.sum(['le', 'service',
'instance']).increase('PT5M').histogram().histogram_percentile([50,70,90,99])
- - name: instance_persistence_prepare_percentile
- exp: persistence_timer_bulk_prepare_latency.sum(['le', 'service',
'instance']).increase('PT5M').histogram().histogram_percentile([50,70,90,99])
- - name: instance_persistence_error_count
- exp: persistence_timer_bulk_error_count.sum(['service',
'instance']).increase('PT5M')
- - name: instance_persistence_execute_count
- exp: persistence_timer_bulk_execute_latency_count.sum(['service',
'instance']).increase('PT5M')
- - name: instance_persistence_prepare_count
- exp: persistence_timer_bulk_prepare_latency_count.sum(['service',
'instance']).increase('PT5M')
- - name: jvm_thread_live_count
- exp: jvm_threads_current.sum(['service', 'instance'])
- - name: jvm_thread_daemon_count
- exp: jvm_threads_daemon.sum(['service', 'instance'])
- - name: jvm_thread_peak_count
- exp: jvm_threads_peak.sum(['service', 'instance'])
- - name: jvm_thread_runnable_count
- exp: jvm_threads_state.tagMatch('state', 'RUNNABLE').sum(['service',
'instance'])
- - name: jvm_thread_blocked_count
- exp: jvm_threads_state.tagMatch('state', 'BLOCKED').sum(['service',
'instance'])
- - name: jvm_thread_waiting_count
- exp: jvm_threads_state.tagMatch('state', 'WAITING').sum(['service',
'instance'])
- - name: jvm_thread_timed_waiting_count
- exp: jvm_threads_state.tagMatch('state', 'TIMED_WAITING').sum(['service',
'instance'])
- - name: jvm_class_loaded_count
- exp: jvm_classes_loaded.sum(['service', 'instance'])
- - name: jvm_class_total_unloaded_count
- exp: jvm_classes_unloaded_total.sum(['service', 'instance'])
- - name: jvm_class_total_loaded_count
- exp: jvm_classes_loaded_total.sum(['service', 'instance'])
diff --git a/test/e2e-v2/cases/kafka/simple-so11y/e2e.yaml
b/test/e2e-v2/cases/kafka/simple-so11y/e2e.yaml
index fcfce60776..a91249e4e4 100644
--- a/test/e2e-v2/cases/kafka/simple-so11y/e2e.yaml
+++ b/test/e2e-v2/cases/kafka/simple-so11y/e2e.yaml
@@ -45,4 +45,3 @@ verify:
cases:
- includes:
- ../../simple/simple-cases.yaml
- - ../../so11y/so11y-cases.yaml
diff --git a/test/e2e-v2/cases/so11y/docker-compose.yml
b/test/e2e-v2/cases/so11y/docker-compose.yml
index ba86f2c179..480b440cf8 100644
--- a/test/e2e-v2/cases/so11y/docker-compose.yml
+++ b/test/e2e-v2/cases/so11y/docker-compose.yml
@@ -26,6 +26,7 @@ services:
SW_METER_ANALYZER_ACTIVE_FILES: satellite
ports:
- 12800
+ - 1234
provider:
extends:
@@ -53,6 +54,17 @@ services:
provider:
condition: service_healthy
+ otel-collector:
+ image: otel/opentelemetry-collector:0.50.0
+ networks:
+ - e2e
+ command: [ "--config=/etc/otel-collector-config.yaml" ]
+ volumes:
+ - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
+ depends_on:
+ oap:
+ condition: service_healthy
+
satellite:
build:
context: ../satellite
diff --git a/test/e2e-v2/cases/so11y/otel-collector-config.yaml
b/test/e2e-v2/cases/so11y/otel-collector-config.yaml
new file mode 100644
index 0000000000..c3c91b0b5a
--- /dev/null
+++ b/test/e2e-v2/cases/so11y/otel-collector-config.yaml
@@ -0,0 +1,59 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+receivers:
+ prometheus:
+ config:
+ scrape_configs:
+ - job_name: 'skywalking-so11y'
+ scrape_interval: 5s
+ static_configs:
+ - targets: ['oap:1234']
+ labels:
+ host_name: http://localhost:1234
+ service: oap-server
+processors:
+ batch:
+
+exporters:
+
####################################################################################
+ # If you want to use otlp exporter please ensure that your OAP version is >=
9.2.0 #
+ # or you can use oc exporter, The config format should be:
#
+ # opencensus:
#
+ # endpoint: "oap:11800"
#
+ # tls:
#
+ # insecure: true
#
+
####################################################################################
+ otlp:
+ endpoint: oap:11800
+ tls:
+ insecure: true
+ logging:
+ logLevel: debug
+
+service:
+ pipelines:
+ metrics:
+ receivers:
+ - prometheus
+ processors:
+ - batch
+ exporters:
+ #########################################
+ # oc exporter config format: #
+ # - opencensus #
+ #########################################
+ - otlp
+ - logging
diff --git a/test/e2e-v2/script/docker-compose/base-compose.yml
b/test/e2e-v2/script/docker-compose/base-compose.yml
index 20cc63228b..d8224075cc 100644
--- a/test/e2e-v2/script/docker-compose/base-compose.yml
+++ b/test/e2e-v2/script/docker-compose/base-compose.yml
@@ -34,8 +34,6 @@ services:
SW_JDBC_URL: jdbc:mysql://mysql:3306/swtest
SW_STORAGE_INFLUXDB_URL: http://influxdb:8086
SW_STORAGE_BANYANDB_HOST: "banyandb"
- SW_STORAGE_IOTDB_HOST: iotdb
- SW_STORAGE_IOTDB_SESSIONPOOL_SIZE: 2
SW_CONFIG_ETCD_PERIOD: 1
SW_CONFIG_ETCD_ENDPOINTS: http://etcd:2379
SW_CLUSTER_ETCD_ENDPOINTS: http://etcd:2379
@@ -53,7 +51,7 @@ services:
- e2e
environment:
- SW_OAP_ADDRESS=http://oap:12800
-
+
banyandb:
image: "ghcr.io/apache/skywalking-banyandb:${SW_BANYANDB_COMMIT}"
networks:
