This is an automated email from the ASF dual-hosted git repository.
dnskr pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 44fcf2872 [KYUUBI #6580] [K8S][HELM] Improve metrics configuration
44fcf2872 is described below
commit 44fcf28720469767904c9cd7c2c3396d90e13c31
Author: dnskr <[email protected]>
AuthorDate: Tue Aug 6 14:54:09 2024 +0200
[KYUUBI #6580] [K8S][HELM] Improve metrics configuration
# :mag: Description
## Issue References ๐
This pull request fixes https://github.com/apache/kyuubi/issues/6565
## Describe Your Solution ๐ง
- Fixed misleading `kyuubi.metrics` properties in
`charts/kyuubi/templates/kyuubi-configmap.yaml`.
- Fixed condition to create `PodMonitor`, `ServiceMonitor` and
`PrometheusRule`.
- Move metrics related properties to `metrics` property tree.
## Types of changes :bookmark:
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
## Test Plan ๐งช
#### Disabled metrics - prometheus port is not renderred
```shell
helm template kyuubi charts/kyuubi --set metrics.enabled=false \
-s templates/kyuubi-statefulset.yaml \
-s templates/kyuubi-headless-service.yaml
```
#### JMX reporter only - prometheus port is not renderred
```shell
helm template kyuubi charts/kyuubi --set metrics.reporters="JMX" \
-s templates/kyuubi-statefulset.yaml \
-s templates/kyuubi-headless-service.yaml
```
#### Default properties - prometheus port is renderred in StatefulSet and
Headless Service
```shell
helm template kyuubi charts/kyuubi -s templates/kyuubi-statefulset.yaml \
-s templates/kyuubi-headless-service.yaml
```
#### Default properties - PodMonitor is not renderred
```shell
helm template kyuubi charts/kyuubi -s templates/kyuubi-podmonitor.yaml
```
#### Default properties - ServiceMonitor is not renderred
```shell
helm template kyuubi charts/kyuubi -s templates/kyuubi-servicemonitor.yaml
```
#### Default properties - PrometheusRule is not renderred
```shell
helm template kyuubi charts/kyuubi -s templates/kyuubi-alert.yaml
```
#### Enabled metrics, PodMonitor, ServiceMonitor, PrometheusRule,
PROMETHEUS reporter and port set to 9999
```shell
helm template kyuubi charts/kyuubi --set metrics.enabled=true \
--set metrics.reporters="PROMETHEUS\,
JMX" \
--set metrics.prometheusPort=9999 \
--set metrics.podMonitor.enabled=true \
--set
metrics.serviceMonitor.enabled=true \
--set
metrics.prometheusRule.enabled=true \
-s templates/kyuubi-statefulset.yaml \
-s
templates/kyuubi-headless-service.yaml \
-s templates/kyuubi-configmap.yaml \
-s templates/kyuubi-podmonitor.yaml \
-s templates/kyuubi-servicemonitor.yaml \
-s templates/kyuubi-alert.yaml
```
#### Install the chart and test Prometheus endpoint
```shell
helm install kyuubi charts/kyuubi --set metrics.enabled=true \
--set metrics.reporters="PROMETHEUS\,
JMX" \
--set metrics.prometheusPort=9999
...
kyuubikyuubi-0:/opt/kyuubi$ curl 127.0.0.1:9999/metrics
# HELP kyuubi_buffer_pool_mapped_count Generated from Dropwizard metric
import (metric=kyuubi.buffer_pool.mapped.count,
type=com.codahale.metrics.jvm.JmxAttributeGauge)
# TYPE kyuubi_buffer_pool_mapped_count gauge
kyuubi_buffer_pool_mapped_count 0.0
# HELP kyuubi_gc_MarkSweepCompact_time Generated from Dropwizard metric
import (metric=kyuubi.gc.MarkSweepCompact.time,
type=com.codahale.metrics.jvm.GarbageCollectorMetricSet$$Lambda$227/1493158871)
# TYPE kyuubi_gc_MarkSweepCompact_time gauge
kyuubi_gc_MarkSweepCompact_time 91.0
...
```
---
# Checklist ๐
- [ ] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes #6580 from dnskr/helm-improve-monitoring-configuration.
Closes #6580
1f20cabef [dnskr] [K8S][HELM] Improve metrics configuration
Authored-by: dnskr <[email protected]>
Signed-off-by: dnskr <[email protected]>
---
charts/kyuubi/templates/kyuubi-alert.yaml | 4 +-
charts/kyuubi/templates/kyuubi-configmap.yaml | 5 +-
.../kyuubi/templates/kyuubi-headless-service.yaml | 7 +--
charts/kyuubi/templates/kyuubi-podmonitor.yaml | 4 +-
charts/kyuubi/templates/kyuubi-servicemonitor.yaml | 8 +--
charts/kyuubi/templates/kyuubi-statefulset.yaml | 4 +-
charts/kyuubi/values.yaml | 65 +++++++++++-----------
7 files changed, 47 insertions(+), 50 deletions(-)
diff --git a/charts/kyuubi/templates/kyuubi-alert.yaml
b/charts/kyuubi/templates/kyuubi-alert.yaml
index 89fd11dc7..a1a7537b3 100644
--- a/charts/kyuubi/templates/kyuubi-alert.yaml
+++ b/charts/kyuubi/templates/kyuubi-alert.yaml
@@ -15,7 +15,7 @@
limitations under the License.
*/}}
-{{- if and .Values.monitoring.prometheus.enabled (eq .Values.metricsReporters
"PROMETHEUS") .Values.prometheusRule.enabled }}
+{{- if and .Values.metrics.enabled (.Values.metrics.reporters | nospace |
splitList "," | has "PROMETHEUS") .Values.metrics.prometheusRule.enabled }}
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
@@ -24,5 +24,5 @@ metadata:
{{- include "kyuubi.labels" . | nindent 4 }}
spec:
groups:
- {{- toYaml .Values.prometheusRule.groups | nindent 4 }}
+ {{- toYaml .Values.metrics.prometheusRule.groups | nindent 4 }}
{{- end }}
diff --git a/charts/kyuubi/templates/kyuubi-configmap.yaml
b/charts/kyuubi/templates/kyuubi-configmap.yaml
index 0f838857e..a171d5855 100644
--- a/charts/kyuubi/templates/kyuubi-configmap.yaml
+++ b/charts/kyuubi/templates/kyuubi-configmap.yaml
@@ -37,8 +37,9 @@ data:
kyuubi.frontend.protocols={{ include "kyuubi.frontend.protocols" . }}
# Kyuubi Metrics
- kyuubi.metrics.enabled={{ .Values.monitoring.prometheus.enabled }}
- kyuubi.metrics.reporters={{ .Values.metricsReporters }}
+ kyuubi.metrics.enabled={{ .Values.metrics.enabled }}
+ kyuubi.metrics.reporters={{ .Values.metrics.reporters }}
+ kyuubi.metrics.prometheus.port={{ .Values.metrics.prometheusPort }}
## User provided Kyuubi configurations
{{- with .Values.kyuubiConf.kyuubiDefaults }}
diff --git a/charts/kyuubi/templates/kyuubi-headless-service.yaml
b/charts/kyuubi/templates/kyuubi-headless-service.yaml
index fa04ffeef..4b75c0b2e 100644
--- a/charts/kyuubi/templates/kyuubi-headless-service.yaml
+++ b/charts/kyuubi/templates/kyuubi-headless-service.yaml
@@ -30,11 +30,10 @@ spec:
port: {{ tpl $frontend.service.port $ }}
targetPort: {{ $frontend.port }}
{{- end }}
- {{- if .Values.monitoring.prometheus.enabled }}
+ {{- if and .Values.metrics.enabled (.Values.metrics.reporters | nospace |
splitList "," | has "PROMETHEUS") }}
- name: prometheus
- port: {{ .Values.monitoring.prometheus.port }}
- targetPort: {{ .Values.monitoring.prometheus.port }}
+ port: {{ .Values.metrics.prometheusPort }}
+ targetPort: prometheus
{{- end }}
selector:
{{- include "kyuubi.selectorLabels" $ | nindent 4 }}
-
diff --git a/charts/kyuubi/templates/kyuubi-podmonitor.yaml
b/charts/kyuubi/templates/kyuubi-podmonitor.yaml
index 458ff66ed..b10d80c3f 100644
--- a/charts/kyuubi/templates/kyuubi-podmonitor.yaml
+++ b/charts/kyuubi/templates/kyuubi-podmonitor.yaml
@@ -15,7 +15,7 @@
limitations under the License.
*/}}
-{{- if and .Values.monitoring.prometheus.enabled (eq .Values.metricsReporters
"PROMETHEUS") .Values.podMonitor.enabled }}
+{{- if and .Values.metrics.enabled (.Values.metrics.reporters | nospace |
splitList "," | has "PROMETHEUS") .Values.metrics.podMonitor.enabled }}
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
@@ -27,5 +27,5 @@ spec:
matchLabels:
app: {{ .Release.Name }}
podMetricsEndpoints:
- {{- toYaml .Values.podMonitor.podMetricsEndpoint | nindent 4 }}
+ {{- toYaml .Values.metrics.podMonitor.podMetricsEndpoints | nindent 4 }}
{{- end }}
diff --git a/charts/kyuubi/templates/kyuubi-servicemonitor.yaml
b/charts/kyuubi/templates/kyuubi-servicemonitor.yaml
index 3f0771b63..a75da50e3 100644
--- a/charts/kyuubi/templates/kyuubi-servicemonitor.yaml
+++ b/charts/kyuubi/templates/kyuubi-servicemonitor.yaml
@@ -15,20 +15,20 @@
limitations under the License.
*/}}
-{{- if and .Values.monitoring.prometheus.enabled (eq .Values.metricsReporters
"PROMETHEUS") .Values.serviceMonitor.enabled }}
+{{- if and .Values.metrics.enabled (.Values.metrics.reporters | nospace |
splitList "," | has "PROMETHEUS") .Values.metrics.serviceMonitor.enabled }}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ .Release.Name }}
labels:
{{- include "kyuubi.labels" . | nindent 4 }}
- {{- if .Values.serviceMonitor.labels }}
- {{- toYaml .Values.serviceMonitor.labels | nindent 4 }}
+ {{- if .Values.metrics.serviceMonitor.labels }}
+ {{- toYaml .Values.metrics.serviceMonitor.labels | nindent 4 }}
{{- end }}
spec:
selector:
matchLabels:
{{- include "kyuubi.selectorLabels" . | nindent 6 }}
endpoints:
- {{- toYaml .Values.serviceMonitor.endpoints | nindent 4 }}
+ {{- toYaml .Values.metrics.serviceMonitor.endpoints | nindent 4 }}
{{- end }}
diff --git a/charts/kyuubi/templates/kyuubi-statefulset.yaml
b/charts/kyuubi/templates/kyuubi-statefulset.yaml
index 303103fdd..caea7d251 100644
--- a/charts/kyuubi/templates/kyuubi-statefulset.yaml
+++ b/charts/kyuubi/templates/kyuubi-statefulset.yaml
@@ -80,9 +80,9 @@ spec:
containerPort: {{ $frontend.port }}
{{- end }}
{{- end }}
- {{- if .Values.monitoring.prometheus.enabled }}
+ {{- if and .Values.metrics.enabled (.Values.metrics.reporters |
nospace | splitList "," | has "PROMETHEUS") }}
- name: prometheus
- containerPort: {{ .Values.monitoring.prometheus.port }}
+ containerPort: {{ .Values.metrics.prometheusPort }}
{{- end }}
{{- if .Values.livenessProbe.enabled }}
livenessProbe:
diff --git a/charts/kyuubi/values.yaml b/charts/kyuubi/values.yaml
index e67be9eb8..23e5e7fdc 100644
--- a/charts/kyuubi/values.yaml
+++ b/charts/kyuubi/values.yaml
@@ -145,12 +145,6 @@ server:
# clientIP:
# timeoutSeconds: 10800
-monitoring:
- # Exposes metrics in Prometheus format
- prometheus:
- enabled: true
- port: 10019
-
# $KYUUBI_CONF_DIR directory
kyuubiConfDir: /opt/kyuubi/conf
# Kyuubi configuration files
@@ -279,31 +273,34 @@ affinity: {}
# Kyuubi pods security context
securityContext: {}
-# Monitoring Kyuubi - Server Metrics
-# PROMETHEUS - PrometheusReporter which exposes metrics in Prometheus format
-metricsReporters: ~
-
-# Prometheus pod monitor
-podMonitor:
- # If enabled, podMonitor for operator's pod will be created
- enabled: false
- # The podMetricsEndpoint contains metrics information such as port,
interval, scheme, and possibly other relevant details.
- # This information is used to configure the endpoint from which Prometheus
can scrape and collect metrics for a specific Pod in Kubernetes.
- podMetricsEndpoint: []
-
-# Prometheus service monitor
-serviceMonitor:
- # If enabled, ServiceMonitor resources for Prometheus Operator are created
- enabled: false
- # The endpoints section in a ServiceMonitor specifies the metrics
information for each target endpoint.
- # This allows you to collect metrics from multiple Services across your
Kubernetes cluster in a standardized and automated way.
- endpoints: []
- # Additional labels that can be used so ServiceMonitor will be discovered by
Prometheus
- labels: {}
-
-# Rules for the Prometheus Operator
-prometheusRule:
- # If enabled, a PrometheusRule resource for Prometheus Operator is created
- enabled: false
- # Contents of Prometheus rules file
- groups: []
+# Metrics configuration
+metrics:
+ # Enable metrics system, used for 'kyuubi.metrics.enabled' property
+ enabled: true
+ # A comma-separated list of metrics reporters, used for
'kyuubi.metrics.reporters' property
+ reporters: PROMETHEUS
+ # Prometheus port, used for 'kyuubi.metrics.prometheus.port' property
+ prometheusPort: 10019
+
+ # PodMonitor by Prometheus Operator
+ podMonitor:
+ # Enable PodMonitor creation
+ enabled: false
+ # List of pod endpoints serving metrics to be scraped by Prometheus, see
Prometheus Operator docs for more details
+ podMetricsEndpoints: []
+
+ # ServiceMonitor by Prometheus Operator
+ serviceMonitor:
+ # Enable ServiceMonitor creation
+ enabled: false
+ # List of service endpoints serving metrics to be scraped by Prometheus,
see Prometheus Operator docs for more details
+ endpoints: []
+ # Additional labels to be used to make ServiceMonitor discovered by
Prometheus
+ labels: {}
+
+ # PrometheusRule by Prometheus Operator
+ prometheusRule:
+ # Enable PrometheusRule creation
+ enabled: false
+ # Content of Prometheus rule file
+ groups: []
