ptlrs commented on code in PR #20:
URL: https://github.com/apache/ozone-helm-charts/pull/20#discussion_r2497334380
##########
charts/ozone/templates/helm/om-leader-transfer-job.yaml:
##########
@@ -0,0 +1,82 @@
+{{- if .Values.om.persistence.enabled }}
+{{- $dnodes := ternary (splitList "," (include "ozone.om.decommissioned.nodes"
.)) (list) (ne "" (include "ozone.om.decommissioned.nodes" .)) }}
+{{- $env := concat .Values.env .Values.helm.env }}
+{{- $envFrom := concat .Values.envFrom .Values.helm.envFrom }}
+{{- $nodeSelector := or .Values.helm.nodeSelector .Values.nodeSelector }}
+{{- $affinity := or .Values.helm.affinity .Values.affinity }}
+{{- $tolerations := or .Values.helm.tolerations .Values.tolerations }}
+{{- $securityContext := or .Values.helm.securityContext
.Values.securityContext }}
+{{- if (gt (len $dnodes) 0) }}
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: {{ printf "%s-helm-manager-leader-transfer" $.Release.Name }}
+ labels:
+ {{- include "ozone.labels" $ | nindent 4 }}
+ app.kubernetes.io/component: helm-manager
+ annotations:
+ "helm.sh/hook": pre-upgrade
+ "helm.sh/hook-weight": "0"
+ "helm.sh/hook-delete-policy": hook-succeeded,hook-failed
+spec:
+ backoffLimit: {{ $.Values.helm.backoffLimit }}
+ template:
+ metadata:
+ labels:
+ {{- include "ozone.selectorLabels" $ | nindent 8 }}
+ app.kubernetes.io/component: helm-manager
+ spec:
+ containers:
+ - name: om-leader-transfer
+ image: "{{ $.Values.image.repository }}:{{ $.Values.image.tag |
default $.Chart.AppVersion }}"
+ imagePullPolicy: {{ $.Values.image.pullPolicy }}
+ {{- with $.Values.om.command }}
+ command: {{- tpl (toYaml .) $ | nindent 12 }}
+ {{- end }}
+ args:
+ - sh
+ - -c
+ - |
+ set -e
+ exec ozone admin om transfer -id={{ $.Values.clusterId }} -n={{
$.Release.Name }}-om-0
+ env:
Review Comment:
Some thoughts about this process. We should see if there is an alternate
approach available here.
1. Transferring to om-0 assumes that om-0 is not a lagging replica. If it is
a significantly lagging replica then we will have to go through the bootstrap
process on om-0 which may take an indeterministic amount of time.
2. Here we don't wait or validate if the transfer to om-0 even completed. It
is on a best effort basis.
3. When we always transfer to om-0 we are introducing a dependency to our
pods in the Ozone cluster. We may never be able to decommission the om-0 pod
for any reason even if we wanted to.
##########
charts/ozone/templates/helm/om-decommission-job.yaml:
##########
@@ -0,0 +1,239 @@
+{{- if or .Values.om.persistence.enabled }}
+{{- $dnodes := ternary (splitList "," (include "ozone.om.decommissioned.nodes"
.)) (list) (ne "" (include "ozone.om.decommissioned.nodes" .)) }}
+{{- $env := concat .Values.env .Values.helm.env }}
+{{- $envFrom := concat .Values.envFrom .Values.helm.envFrom }}
+{{- $nodeSelector := or .Values.helm.nodeSelector .Values.nodeSelector }}
+{{- $affinity := or .Values.helm.affinity .Values.affinity }}
+{{- $tolerations := or .Values.helm.tolerations .Values.tolerations }}
+{{- $securityContext := or .Values.helm.securityContext
.Values.securityContext }}
+{{- if and (gt (len $dnodes) 0) ( .Values.om.persistence.enabled) }}
+
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: {{ printf "%s-helm-manager-leader-transfer" $.Release.Name }}
+ labels:
+ {{- include "ozone.labels" $ | nindent 4 }}
+ app.kubernetes.io/component: helm-manager
+ annotations:
+ "helm.sh/hook": pre-upgrade
+ "helm.sh/hook-weight": "0"
+ "helm.sh/hook-delete-policy": hook-succeeded, hook-failed
+spec:
+ backoffLimit: {{ $.Values.helm.backoffLimit }}
+ template:
+ metadata:
+ labels:
+ {{- include "ozone.selectorLabels" $ | nindent 8 }}
+ app.kubernetes.io/component: helm-manager
+ spec:
+ containers:
+ - name: om-leader-transfer
+ image: "{{ $.Values.image.repository }}:{{ $.Values.image.tag |
default $.Chart.AppVersion }}"
+ imagePullPolicy: {{ $.Values.image.pullPolicy }}
+ {{- with $.Values.om.command }}
+ command: {{- tpl (toYaml .) $ | nindent 12 }}
+ {{- end }}
+ args:
+ - sh
+ - -c
+ - |
+ set -e
+ exec ozone admin om transfer -id=cluster1 -n={{ $.Release.Name
}}-om-0
+ env:
+ {{- include "ozone.configuration.env.prehook" $ | nindent 12 }}
+ {{- with $env }}
+ {{- tpl (toYaml .) $ | nindent 12 }}
+ {{- end }}
+ {{- with $envFrom }}
+ envFrom: {{- tpl (toYaml .) $ | nindent 12 }}
+ {{- end }}
+ ports:
+ - name: data-ratis-ipc
+ containerPort: 9858
+ - name: data-ipc
+ containerPort: 9859
+ - name: scm-rpc-client
+ containerPort: 9860
+ - name: scm-block-cl
+ containerPort: 9863
+ - name: scm-rpc-data
+ containerPort: 9861
+ - name: scm-ratis
+ containerPort: 9894
+ - name: scm-grpc
+ containerPort: 9895
+ - name: om-rpc
+ containerPort: 9862
+ - name: om-ratis
+ containerPort: 9872
+ volumeMounts:
+ - name: config
+ mountPath: {{ $.Values.configuration.dir }}
+ - name: om-data
+ mountPath: {{ $.Values.om.persistence.path }}
+ {{- with $nodeSelector }}
+ nodeSelector: {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with $affinity }}
+ affinity: {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with $tolerations }}
+ tolerations: {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with $securityContext }}
+ securityContext: {{- toYaml . | nindent 8 }}
+ {{- end }}
+ volumes:
+ - name: om-data
+ emptyDir: { }
+ - name: config
+ projected:
+ sources:
+ - configMap:
+ name: {{ $.Release.Name }}-ozone
+ {{- with $.Values.configuration.filesFrom }}
+ {{- tpl (toYaml .) $ | nindent 14 }}
+ {{- end }}
+ restartPolicy: Never
+
+{{- range $dnode := $dnodes }}
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: {{ printf "%s-helm-manager-decommission-%s" $.Release.Name $dnode }}
+ labels:
+ {{- include "ozone.labels" $ | nindent 4 }}
+ app.kubernetes.io/component: helm-manager
+ annotations:
+ "helm.sh/hook": post-upgrade
+ "helm.sh/hook-weight": "0"
+ "helm.sh/hook-delete-policy": hook-succeeded, hook-failed
+spec:
+ backoffLimit: {{ $.Values.helm.backoffLimit }}
+ template:
+ metadata:
+ labels:
+ {{- include "ozone.selectorLabels" $ | nindent 8 }}
+ app.kubernetes.io/component: helm-manager
+ spec:
+ containers:
+ - name: om-decommission
+ image: "{{ $.Values.image.repository }}:{{ $.Values.image.tag |
default $.Chart.AppVersion }}"
+ imagePullPolicy: {{ $.Values.image.pullPolicy }}
+ {{- with $.Values.om.command }}
+ command: {{- tpl (toYaml .) $ | nindent 12 }}
+ {{- end }}
+ args:
+ - sh
+ - -c
+ - |
+ set -e
+ decommission_finalizer() {
+ echo "Init decommission finalizer process..."
+ while true; do
+ IFS= read -r line;
+ echo "$line"
+ if echo "$line" | grep -q "Successfully decommissioned OM
{{ $dnode }}"; then
+ echo "{{ $dnode }} was successfully decommissioned!"
+ if [ -d /old{{ $.Values.om.persistence.path }} ]; then
+ echo "Delete old data on pvc to enable rescheduling
without manual PVC deletion!"
+ rm -rf /old{{ $.Values.om.persistence.path }}/*
+ echo "Data deleted!"
+ fi
Review Comment:
If the only reason to clean up the volume this way is to speed up the
scaling up of pods, we should reconsider this.
We should let kubernetes handle the volume lifecycle for us than taking the
responsibility of maintaining the correct state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
