wu-sheng commented on a change in pull request #4972: URL: https://github.com/apache/skywalking/pull/4972#discussion_r446476018
########## File path: docs/en/setup/backend/backend-meter.md ########## @@ -0,0 +1,77 @@ +# Meter Receiver +Meter receiver is accept meter from [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) into the [Meter System](./../../concepts-and-designs/meter.md).o Review comment: ```suggestion Meter receiver is accepting the metrics of [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) format into the [Meter System](./../../concepts-and-designs/meter.md). ``` ########## File path: docs/en/setup/backend/backend-meter.md ########## @@ -0,0 +1,77 @@ +# Meter Receiver +Meter receiver is accept meter from [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) into the [Meter System](./../../concepts-and-designs/meter.md).o + +## Module define +receiver-meter: + selector: ${SW_RECEIVER_METER:default} + default: + +## Configuration file +Meter receiver is configured via a configuration file. The configuration file defines everything related to receiving + from agents, as well as which rule files to load. + +OAP can load the configuration at bootstrap. If the new configuration is not well-formed, OAP fails to start up. The files +are located at `$CLASSPATH/meter-receive-config`. + +The file is written in YAML format, defined by the scheme described below. Brackets indicate that a parameter is optional. + +A example can be found [here](../../../../oap-server/server-bootstrap/src/main/resources/meter-receive-config/meter-receive-config.yaml) + +### Meters configure + +```yaml +# Meter config allow your to recompute +meters: + # Meter name which combines with a prefix 'meter_' as the index/table name in storage. + - name: <string> + # The meter scope + scope: + # Scope type should be one of SERVICE, SERVICE_INSTANCE, ENDPOINT + type: <string> + # Appoint the endpoint name if using ENDPOINT scope + endpoint: <string> + # The agent source of the transformation operation. + meter: + # The transformation operation from prometheus metrics to Skywalking ones. + operation: <string> + # Meter value parse groovy script. + value: <string> + # Appoint percentiles if using avgHistogramPercentile operation. Review comment: ```suggestion # <Optional> Appoint percentiles if using avgHistogramPercentile operation. ``` ########## File path: docs/en/setup/backend/backend-meter.md ########## @@ -0,0 +1,77 @@ +# Meter Receiver +Meter receiver is accept meter from [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) into the [Meter System](./../../concepts-and-designs/meter.md).o + +## Module define +receiver-meter: + selector: ${SW_RECEIVER_METER:default} + default: + +## Configuration file +Meter receiver is configured via a configuration file. The configuration file defines everything related to receiving + from agents, as well as which rule files to load. + +OAP can load the configuration at bootstrap. If the new configuration is not well-formed, OAP fails to start up. The files +are located at `$CLASSPATH/meter-receive-config`. + +The file is written in YAML format, defined by the scheme described below. Brackets indicate that a parameter is optional. + +A example can be found [here](../../../../oap-server/server-bootstrap/src/main/resources/meter-receive-config/meter-receive-config.yaml) + +### Meters configure + +```yaml +# Meter config allow your to recompute +meters: + # Meter name which combines with a prefix 'meter_' as the index/table name in storage. + - name: <string> + # The meter scope + scope: + # Scope type should be one of SERVICE, SERVICE_INSTANCE, ENDPOINT + type: <string> + # Appoint the endpoint name if using ENDPOINT scope + endpoint: <string> + # The agent source of the transformation operation. + meter: + # The transformation operation from prometheus metrics to Skywalking ones. + operation: <string> + # Meter value parse groovy script. + value: <string> + # Appoint percentiles if using avgHistogramPercentile operation. + percentile: + - <rank> +``` + +#### Meter transform operation + +The available operations are `avg`, `avgHistogram` and `avgHistogramPercentile`. The `avg` and `avgXXX` mean to average +the raw received metrics. + +When you specify `avgHistogram` and `avgHistogramPercentile`, the source should be the type of `histogram`. + +#### Meter value script + +The script is provide a easy way to custom build a complex value, and it also support combine multiple meter into one. + +Using `meter[""]` to get one meter from all of the meter witch received at ***per agent***, also there have multiple build-in method to help filter or operate the value. Review comment: ```suggestion Use `meter[""]` to refer the metrics raw data, and multiple build-in methods to help filter or operate the value. ``` ########## File path: docs/en/setup/backend/backend-meter.md ########## @@ -0,0 +1,77 @@ +# Meter Receiver +Meter receiver is accept meter from [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) into the [Meter System](./../../concepts-and-designs/meter.md).o + +## Module define +receiver-meter: + selector: ${SW_RECEIVER_METER:default} + default: + +## Configuration file +Meter receiver is configured via a configuration file. The configuration file defines everything related to receiving + from agents, as well as which rule files to load. + +OAP can load the configuration at bootstrap. If the new configuration is not well-formed, OAP fails to start up. The files +are located at `$CLASSPATH/meter-receive-config`. + +The file is written in YAML format, defined by the scheme described below. Brackets indicate that a parameter is optional. + +A example can be found [here](../../../../oap-server/server-bootstrap/src/main/resources/meter-receive-config/meter-receive-config.yaml) + +### Meters configure + +```yaml +# Meter config allow your to recompute +meters: + # Meter name which combines with a prefix 'meter_' as the index/table name in storage. + - name: <string> + # The meter scope + scope: + # Scope type should be one of SERVICE, SERVICE_INSTANCE, ENDPOINT + type: <string> + # Appoint the endpoint name if using ENDPOINT scope + endpoint: <string> + # The agent source of the transformation operation. + meter: + # The transformation operation from prometheus metrics to Skywalking ones. + operation: <string> + # Meter value parse groovy script. + value: <string> + # Appoint percentiles if using avgHistogramPercentile operation. + percentile: + - <rank> +``` + +#### Meter transform operation + +The available operations are `avg`, `avgHistogram` and `avgHistogramPercentile`. The `avg` and `avgXXX` mean to average +the raw received metrics. + +When you specify `avgHistogram` and `avgHistogramPercentile`, the source should be the type of `histogram`. + +#### Meter value script + +The script is provide a easy way to custom build a complex value, and it also support combine multiple meter into one. + +Using `meter[""]` to get one meter from all of the meter witch received at ***per agent***, also there have multiple build-in method to help filter or operate the value. +It will combine the values and collect into the Meter System. + +|Function|Support meter type|Description| +|------|-------|------| +|tagFilter(String, String)|Counter,Gauge,Histogram|Filter tag key/value from meter| +|add(double)|Counter,Gauge|Add value into meter| +|add(Meter)|Counter,Gauge|Add value from a meter value| +|reduce(double)|Counter,Gauge|Reduce value into meter| +|reduce(Meter)|Counter,Gauge|Reduce value from a meter value| +|multiply(double)|Counter,Gauge|Multiply value into meter| +|multiply(Meter)|Counter,Gauge|Multiply value form a meter value| +|mean(double)|Counter,Gauge|Mean value into meter| +|mean(Meter)|Counter,Gauge|Mean value from a meter value| Review comment: What do you mean `mean`? A little confused about this term ########## File path: docs/en/setup/backend/backend-meter.md ########## @@ -0,0 +1,77 @@ +# Meter Receiver +Meter receiver is accept meter from [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) into the [Meter System](./../../concepts-and-designs/meter.md).o + +## Module define +receiver-meter: + selector: ${SW_RECEIVER_METER:default} + default: + +## Configuration file +Meter receiver is configured via a configuration file. The configuration file defines everything related to receiving + from agents, as well as which rule files to load. + +OAP can load the configuration at bootstrap. If the new configuration is not well-formed, OAP fails to start up. The files +are located at `$CLASSPATH/meter-receive-config`. + +The file is written in YAML format, defined by the scheme described below. Brackets indicate that a parameter is optional. + +A example can be found [here](../../../../oap-server/server-bootstrap/src/main/resources/meter-receive-config/meter-receive-config.yaml) + +### Meters configure + +```yaml +# Meter config allow your to recompute +meters: + # Meter name which combines with a prefix 'meter_' as the index/table name in storage. + - name: <string> + # The meter scope + scope: + # Scope type should be one of SERVICE, SERVICE_INSTANCE, ENDPOINT + type: <string> + # Appoint the endpoint name if using ENDPOINT scope + endpoint: <string> + # The agent source of the transformation operation. + meter: + # The transformation operation from prometheus metrics to Skywalking ones. + operation: <string> + # Meter value parse groovy script. + value: <string> + # Appoint percentiles if using avgHistogramPercentile operation. + percentile: + - <rank> +``` + +#### Meter transform operation + +The available operations are `avg`, `avgHistogram` and `avgHistogramPercentile`. The `avg` and `avgXXX` mean to average +the raw received metrics. + +When you specify `avgHistogram` and `avgHistogramPercentile`, the source should be the type of `histogram`. + +#### Meter value script + +The script is provide a easy way to custom build a complex value, and it also support combine multiple meter into one. + +Using `meter[""]` to get one meter from all of the meter witch received at ***per agent***, also there have multiple build-in method to help filter or operate the value. +It will combine the values and collect into the Meter System. Review comment: Don't use `future tense` if possible. ########## File path: oap-server/server-receiver-plugin/skywalking-meter-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/meter/provider/config/MeterConfigs.java ########## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.skywalking.oap.server.receiver.meter.provider.config; + +import lombok.Data; +import org.apache.skywalking.oap.server.library.module.ModuleStartException; +import org.apache.skywalking.oap.server.library.util.ResourceUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.yaml.snakeyaml.Yaml; + +import java.io.File; +import java.io.FileNotFoundException; +import java.io.FileReader; +import java.io.IOException; +import java.io.Reader; +import java.util.Arrays; +import java.util.List; +import java.util.Objects; +import java.util.stream.Collectors; + +public class MeterConfigs { + private static final Logger LOGGER = LoggerFactory.getLogger(MeterConfigs.class); Review comment: Code style is not right. ########## File path: oap-server/server-bootstrap/src/main/resources/meter-receive-config/config.yaml ########## @@ -0,0 +1,33 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +meters: Review comment: This file is being packeged into the boostrap jar, please recheck the pom.xml ########## File path: oap-server/server-bootstrap/src/main/resources/meter-receive-config/config.yaml ########## @@ -0,0 +1,33 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +meters: + - name: jvm_heap_usage_avg Review comment: What are these metrics for? Who are sending this? Please check my suggestion about your default meter YAML. ########## File path: oap-server/server-bootstrap/src/main/resources/application.yml ########## @@ -196,6 +196,10 @@ prometheus-fetcher: default: active: ${SW_PROMETHEUS_FETCHER_ACTIVE:false} +receiver-meter: + selector: ${SW_RECEIVER_METER:default} Review comment: I think this should be `-` by default. ########## File path: docs/en/setup/backend/backend-meter.md ########## @@ -0,0 +1,77 @@ +# Meter Receiver +Meter receiver is accept meter from [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) into the [Meter System](./../../concepts-and-designs/meter.md).o + +## Module define +receiver-meter: + selector: ${SW_RECEIVER_METER:default} + default: + +## Configuration file +Meter receiver is configured via a configuration file. The configuration file defines everything related to receiving + from agents, as well as which rule files to load. + +OAP can load the configuration at bootstrap. If the new configuration is not well-formed, OAP fails to start up. The files +are located at `$CLASSPATH/meter-receive-config`. + +The file is written in YAML format, defined by the scheme described below. Brackets indicate that a parameter is optional. + +A example can be found [here](../../../../oap-server/server-bootstrap/src/main/resources/meter-receive-config/meter-receive-config.yaml) + +### Meters configure + +```yaml +# Meter config allow your to recompute +meters: + # Meter name which combines with a prefix 'meter_' as the index/table name in storage. + - name: <string> + # The meter scope + scope: + # Scope type should be one of SERVICE, SERVICE_INSTANCE, ENDPOINT + type: <string> + # Appoint the endpoint name if using ENDPOINT scope + endpoint: <string> + # The agent source of the transformation operation. + meter: + # The transformation operation from prometheus metrics to Skywalking ones. + operation: <string> + # Meter value parse groovy script. + value: <string> + # Appoint percentiles if using avgHistogramPercentile operation. + percentile: + - <rank> +``` + +#### Meter transform operation + +The available operations are `avg`, `avgHistogram` and `avgHistogramPercentile`. The `avg` and `avgXXX` mean to average +the raw received metrics. + +When you specify `avgHistogram` and `avgHistogramPercentile`, the source should be the type of `histogram`. + +#### Meter value script + +The script is provide a easy way to custom build a complex value, and it also support combine multiple meter into one. + +Using `meter[""]` to get one meter from all of the meter witch received at ***per agent***, also there have multiple build-in method to help filter or operate the value. +It will combine the values and collect into the Meter System. + +|Function|Support meter type|Description| +|------|-------|------| +|tagFilter(String, String)|Counter,Gauge,Histogram|Filter tag key/value from meter| +|add(double)|Counter,Gauge|Add value into meter| +|add(Meter)|Counter,Gauge|Add value from a meter value| +|reduce(double)|Counter,Gauge|Reduce value into meter| +|reduce(Meter)|Counter,Gauge|Reduce value from a meter value| +|multiply(double)|Counter,Gauge|Multiply value into meter| +|multiply(Meter)|Counter,Gauge|Multiply value form a meter value| +|mean(double)|Counter,Gauge|Mean value into meter| +|mean(Meter)|Counter,Gauge|Mean value from a meter value| +|scale(int)|Counter,Gauge|Scale the meter value| +|rate(string)|Counter,Gauge,Histogram|Rate value from the time range| +|irate(string)|Counter,Gauge,Histogram|IRate value from the time range| +|increase(string)|Counter,Gauge,Histogram|Increase value from the time range| + +The `add`, `reduce`, `multiply`, `mean` with Meter only support operate from single Meter value, you could using `tagFilter` to filter, Because we could not operation between two multiple values. Review comment: ```suggestion The `add`, `reduce`, `multiply`, and `mean` support single Meter value. Use `tagFilter` to filter. ``` What do you mean `Because we could not operation between two multiple values.`? ########## File path: docs/en/setup/backend/backend-meter.md ########## @@ -0,0 +1,77 @@ +# Meter Receiver +Meter receiver is accept meter from [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) into the [Meter System](./../../concepts-and-designs/meter.md).o + +## Module define +receiver-meter: + selector: ${SW_RECEIVER_METER:default} + default: + +## Configuration file +Meter receiver is configured via a configuration file. The configuration file defines everything related to receiving + from agents, as well as which rule files to load. + +OAP can load the configuration at bootstrap. If the new configuration is not well-formed, OAP fails to start up. The files +are located at `$CLASSPATH/meter-receive-config`. + +The file is written in YAML format, defined by the scheme described below. Brackets indicate that a parameter is optional. + +A example can be found [here](../../../../oap-server/server-bootstrap/src/main/resources/meter-receive-config/meter-receive-config.yaml) + +### Meters configure + +```yaml +# Meter config allow your to recompute +meters: + # Meter name which combines with a prefix 'meter_' as the index/table name in storage. + - name: <string> + # The meter scope + scope: + # Scope type should be one of SERVICE, SERVICE_INSTANCE, ENDPOINT + type: <string> + # Appoint the endpoint name if using ENDPOINT scope + endpoint: <string> + # The agent source of the transformation operation. + meter: + # The transformation operation from prometheus metrics to Skywalking ones. + operation: <string> + # Meter value parse groovy script. + value: <string> + # Appoint percentiles if using avgHistogramPercentile operation. + percentile: + - <rank> +``` + +#### Meter transform operation + +The available operations are `avg`, `avgHistogram` and `avgHistogramPercentile`. The `avg` and `avgXXX` mean to average +the raw received metrics. + +When you specify `avgHistogram` and `avgHistogramPercentile`, the source should be the type of `histogram`. + +#### Meter value script + +The script is provide a easy way to custom build a complex value, and it also support combine multiple meter into one. + +Using `meter[""]` to get one meter from all of the meter witch received at ***per agent***, also there have multiple build-in method to help filter or operate the value. +It will combine the values and collect into the Meter System. + +|Function|Support meter type|Description| +|------|-------|------| +|tagFilter(String, String)|Counter,Gauge,Histogram|Filter tag key/value from meter| +|add(double)|Counter,Gauge|Add value into meter| +|add(Meter)|Counter,Gauge|Add value from a meter value| +|reduce(double)|Counter,Gauge|Reduce value into meter| +|reduce(Meter)|Counter,Gauge|Reduce value from a meter value| +|multiply(double)|Counter,Gauge|Multiply value into meter| +|multiply(Meter)|Counter,Gauge|Multiply value form a meter value| +|mean(double)|Counter,Gauge|Mean value into meter| +|mean(Meter)|Counter,Gauge|Mean value from a meter value| +|scale(int)|Counter,Gauge|Scale the meter value| +|rate(string)|Counter,Gauge,Histogram|Rate value from the time range| +|irate(string)|Counter,Gauge,Histogram|IRate value from the time range| +|increase(string)|Counter,Gauge,Histogram|Increase value from the time range| + +The `add`, `reduce`, `multiply`, `mean` with Meter only support operate from single Meter value, you could using `tagFilter` to filter, Because we could not operation between two multiple values. + +If you want to using `rate`, `irate`, `increase` function, I suggest to let it rate counter at the agent side. Review comment: ```suggestion If you want to use `rate`, `irate`, `increase` function, use client-side API. ``` ########## File path: docs/en/setup/backend/backend-meter.md ########## @@ -0,0 +1,77 @@ +# Meter Receiver +Meter receiver is accept meter from [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) into the [Meter System](./../../concepts-and-designs/meter.md).o + +## Module define +receiver-meter: + selector: ${SW_RECEIVER_METER:default} + default: + +## Configuration file +Meter receiver is configured via a configuration file. The configuration file defines everything related to receiving + from agents, as well as which rule files to load. + +OAP can load the configuration at bootstrap. If the new configuration is not well-formed, OAP fails to start up. The files +are located at `$CLASSPATH/meter-receive-config`. + +The file is written in YAML format, defined by the scheme described below. Brackets indicate that a parameter is optional. + +A example can be found [here](../../../../oap-server/server-bootstrap/src/main/resources/meter-receive-config/meter-receive-config.yaml) + +### Meters configure + +```yaml +# Meter config allow your to recompute +meters: + # Meter name which combines with a prefix 'meter_' as the index/table name in storage. + - name: <string> + # The meter scope + scope: + # Scope type should be one of SERVICE, SERVICE_INSTANCE, ENDPOINT + type: <string> + # Appoint the endpoint name if using ENDPOINT scope + endpoint: <string> + # The agent source of the transformation operation. + meter: + # The transformation operation from prometheus metrics to Skywalking ones. + operation: <string> + # Meter value parse groovy script. + value: <string> + # Appoint percentiles if using avgHistogramPercentile operation. + percentile: + - <rank> +``` + +#### Meter transform operation + +The available operations are `avg`, `avgHistogram` and `avgHistogramPercentile`. The `avg` and `avgXXX` mean to average +the raw received metrics. + +When you specify `avgHistogram` and `avgHistogramPercentile`, the source should be the type of `histogram`. + +#### Meter value script + +The script is provide a easy way to custom build a complex value, and it also support combine multiple meter into one. + +Using `meter[""]` to get one meter from all of the meter witch received at ***per agent***, also there have multiple build-in method to help filter or operate the value. +It will combine the values and collect into the Meter System. + +|Function|Support meter type|Description| +|------|-------|------| +|tagFilter(String, String)|Counter,Gauge,Histogram|Filter tag key/value from meter| +|add(double)|Counter,Gauge|Add value into meter| Review comment: Do we have counter and guage type? In the protocol, we define https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto#L49. There is no such thing. You must keep the nouns consistent, otherwise, people are going to be very confused. Even I am hard to follow. ########## File path: docs/en/setup/backend/backend-meter.md ########## @@ -0,0 +1,77 @@ +# Meter Receiver +Meter receiver is accept meter from [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) into the [Meter System](./../../concepts-and-designs/meter.md).o + +## Module define +receiver-meter: + selector: ${SW_RECEIVER_METER:default} + default: + +## Configuration file +Meter receiver is configured via a configuration file. The configuration file defines everything related to receiving + from agents, as well as which rule files to load. + +OAP can load the configuration at bootstrap. If the new configuration is not well-formed, OAP fails to start up. The files +are located at `$CLASSPATH/meter-receive-config`. + +The file is written in YAML format, defined by the scheme described below. Brackets indicate that a parameter is optional. + +A example can be found [here](../../../../oap-server/server-bootstrap/src/main/resources/meter-receive-config/meter-receive-config.yaml) + +### Meters configure + +```yaml +# Meter config allow your to recompute +meters: + # Meter name which combines with a prefix 'meter_' as the index/table name in storage. + - name: <string> + # The meter scope + scope: + # Scope type should be one of SERVICE, SERVICE_INSTANCE, ENDPOINT + type: <string> + # Appoint the endpoint name if using ENDPOINT scope + endpoint: <string> + # The agent source of the transformation operation. + meter: + # The transformation operation from prometheus metrics to Skywalking ones. + operation: <string> + # Meter value parse groovy script. + value: <string> + # Appoint percentiles if using avgHistogramPercentile operation. + percentile: + - <rank> +``` + +#### Meter transform operation + +The available operations are `avg`, `avgHistogram` and `avgHistogramPercentile`. The `avg` and `avgXXX` mean to average +the raw received metrics. + +When you specify `avgHistogram` and `avgHistogramPercentile`, the source should be the type of `histogram`. + +#### Meter value script + +The script is provide a easy way to custom build a complex value, and it also support combine multiple meter into one. + +Using `meter[""]` to get one meter from all of the meter witch received at ***per agent***, also there have multiple build-in method to help filter or operate the value. +It will combine the values and collect into the Meter System. + +|Function|Support meter type|Description| +|------|-------|------| +|tagFilter(String, String)|Counter,Gauge,Histogram|Filter tag key/value from meter| +|add(double)|Counter,Gauge|Add value into meter| +|add(Meter)|Counter,Gauge|Add value from a meter value| +|reduce(double)|Counter,Gauge|Reduce value into meter| +|reduce(Meter)|Counter,Gauge|Reduce value from a meter value| +|multiply(double)|Counter,Gauge|Multiply value into meter| +|multiply(Meter)|Counter,Gauge|Multiply value form a meter value| +|mean(double)|Counter,Gauge|Mean value into meter| +|mean(Meter)|Counter,Gauge|Mean value from a meter value| +|scale(int)|Counter,Gauge|Scale the meter value| +|rate(string)|Counter,Gauge,Histogram|Rate value from the time range| +|irate(string)|Counter,Gauge,Histogram|IRate value from the time range| +|increase(string)|Counter,Gauge,Histogram|Increase value from the time range| + +The `add`, `reduce`, `multiply`, `mean` with Meter only support operate from single Meter value, you could using `tagFilter` to filter, Because we could not operation between two multiple values. + +If you want to using `rate`, `irate`, `increase` function, I suggest to let it rate counter at the agent side. +Because if the connection between agent and backend has reconnected, the time windows will be break, so we cannot find previous value from the window. Review comment: ```suggestion - FAQ, why no `rate`, `irate`, `increase` at the backend. Once the agent reconnected to another OAP instance, the time windows of rate calculation will break. Then, the result would not be accurate. ``` ########## File path: docs/en/setup/backend/backend-meter.md ########## @@ -0,0 +1,77 @@ +# Meter Receiver +Meter receiver is accept meter from [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) into the [Meter System](./../../concepts-and-designs/meter.md).o + +## Module define +receiver-meter: + selector: ${SW_RECEIVER_METER:default} + default: + +## Configuration file +Meter receiver is configured via a configuration file. The configuration file defines everything related to receiving + from agents, as well as which rule files to load. + +OAP can load the configuration at bootstrap. If the new configuration is not well-formed, OAP fails to start up. The files +are located at `$CLASSPATH/meter-receive-config`. Review comment: I think you should provide Sleuth config in the folder, the example of agent setup, UI template, and full documentation to explain the document. There is nothing than an example to show how to use this feature. ########## File path: oap-server/pom.xml ########## @@ -89,6 +89,7 @@ <freemarker.version>2.3.28</freemarker.version> <javaassist.version>3.25.0-GA</javaassist.version> <vavr.version>0.10.3</vavr.version> + <groovy.version>3.0.3</groovy.version> Review comment: Check `check-LICENSE.sh` and `known-oap-backend-dependencies.txt`/`known-oap-backend-dependencies-es7.txt`. You need to update LICENSE/NOTICE and license check process. You are bringing a new like, maybe including many jars into OAP. ########## File path: docs/en/setup/backend/backend-meter.md ########## @@ -0,0 +1,77 @@ +# Meter Receiver +Meter receiver is accept meter from [meter protocol](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto) into the [Meter System](./../../concepts-and-designs/meter.md).o + +## Module define +receiver-meter: + selector: ${SW_RECEIVER_METER:default} + default: + +## Configuration file +Meter receiver is configured via a configuration file. The configuration file defines everything related to receiving + from agents, as well as which rule files to load. + +OAP can load the configuration at bootstrap. If the new configuration is not well-formed, OAP fails to start up. The files +are located at `$CLASSPATH/meter-receive-config`. + +The file is written in YAML format, defined by the scheme described below. Brackets indicate that a parameter is optional. + +A example can be found [here](../../../../oap-server/server-bootstrap/src/main/resources/meter-receive-config/meter-receive-config.yaml) + +### Meters configure + +```yaml +# Meter config allow your to recompute +meters: + # Meter name which combines with a prefix 'meter_' as the index/table name in storage. + - name: <string> + # The meter scope + scope: + # Scope type should be one of SERVICE, SERVICE_INSTANCE, ENDPOINT + type: <string> + # Appoint the endpoint name if using ENDPOINT scope + endpoint: <string> + # The agent source of the transformation operation. + meter: + # The transformation operation from prometheus metrics to Skywalking ones. + operation: <string> + # Meter value parse groovy script. + value: <string> + # Appoint percentiles if using avgHistogramPercentile operation. + percentile: + - <rank> +``` + +#### Meter transform operation + +The available operations are `avg`, `avgHistogram` and `avgHistogramPercentile`. The `avg` and `avgXXX` mean to average +the raw received metrics. + +When you specify `avgHistogram` and `avgHistogramPercentile`, the source should be the type of `histogram`. + +#### Meter value script + +The script is provide a easy way to custom build a complex value, and it also support combine multiple meter into one. + +Using `meter[""]` to get one meter from all of the meter witch received at ***per agent***, also there have multiple build-in method to help filter or operate the value. +It will combine the values and collect into the Meter System. Review comment: ```suggestion ``` ########## File path: oap-server/server-receiver-plugin/skywalking-meter-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/meter/provider/config/MeterConfigs.java ########## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.skywalking.oap.server.receiver.meter.provider.config; + +import lombok.Data; +import org.apache.skywalking.oap.server.library.module.ModuleStartException; +import org.apache.skywalking.oap.server.library.util.ResourceUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.yaml.snakeyaml.Yaml; + +import java.io.File; +import java.io.FileNotFoundException; +import java.io.FileReader; +import java.io.IOException; +import java.io.Reader; +import java.util.Arrays; +import java.util.List; +import java.util.Objects; +import java.util.stream.Collectors; + +public class MeterConfigs { Review comment: Comments? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
