MonkeyCanCode commented on code in PR #4755:
URL: https://github.com/apache/polaris/pull/4755#discussion_r3422570575
##########
helm/polaris/templates/cronjob-maintenance.yaml:
##########
@@ -0,0 +1,149 @@
+{{/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+*/}}
+
+{{- range $jobName, $jobSpec := .Values.maintenance.jobs }}
+{{- if $jobSpec.enabled }}
+{{- $cronName := printf "%s-maint-%s" (include "polaris.fullname" $) $jobName
}}
Review Comment:
Good idea. I will work on this refactor. Yes, the scheduled cj will added
timestamp and suffix (thus I capped at 52).
##########
helm/polaris/templates/cronjob-maintenance.yaml:
##########
@@ -0,0 +1,149 @@
+{{/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+*/}}
+
+{{- range $jobName, $jobSpec := .Values.maintenance.jobs }}
+{{- if $jobSpec.enabled }}
+{{- $cronName := printf "%s-maint-%s" (include "polaris.fullname" $) $jobName
}}
+{{- if gt (len $cronName) 52 }}
+ {{- fail (printf "Maintenance CronJob name %q is %d chars; must be <= 52
(K8s appends a timestamp+random suffix to generated Pod names, which are capped
at 63). Shorten release name, nameOverride/fullnameOverride, or the job key
%q." $cronName (len $cronName) $jobName) }}
+{{- end }}
+---
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+ name: {{ $cronName }}
+ namespace: {{ $.Release.Namespace }}
+ labels:
+ {{- include "polaris.labels" $ | nindent 4 }}
+ {{- with $.Values.maintenance.annotations }}
+ annotations:
+ {{- tpl (toYaml .) $ | nindent 4 }}
+ {{- end }}
+spec:
+ schedule: {{ $jobSpec.schedule | quote }}
+ concurrencyPolicy: {{ default "Forbid" $jobSpec.concurrencyPolicy }}
+ {{- if hasKey $jobSpec "successfulJobsHistoryLimit" }}
+ successfulJobsHistoryLimit: {{ $jobSpec.successfulJobsHistoryLimit }}
+ {{- end }}
+ {{- if hasKey $jobSpec "failedJobsHistoryLimit" }}
+ failedJobsHistoryLimit: {{ $jobSpec.failedJobsHistoryLimit }}
+ {{- end }}
+ jobTemplate:
+ spec:
+ {{- if hasKey $jobSpec "backoffLimit" }}
+ backoffLimit: {{ $jobSpec.backoffLimit }}
+ {{- end }}
+ {{- with $jobSpec.activeDeadlineSeconds }}
+ activeDeadlineSeconds: {{ . }}
+ {{- end }}
+ template:
+ metadata:
+ labels:
+ helm.sh/chart: {{ include "polaris.chart" $ }}
Review Comment:
This is done so they don't get pick up by pod selector:
```
{{/*
Common labels
*/}}
{{- define "polaris.labels" -}}
helm.sh/chart: {{ include "polaris.chart" . }}
{{ include "polaris.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "polaris.selectorLabels" -}}
app.kubernetes.io/name: {{ include "polaris.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
```
##########
helm/polaris/values.yaml:
##########
@@ -1167,3 +1167,104 @@ tasks:
# -- The maximum number of tasks that can be queued up for execution. If
unspecified or zero, defaults to Integer.MAX_VALUE.
# @section -- Tasks
maxQueuedTasks: 0 # 1000
+
+# @schema type: object
+# -- Configuration for maintenance tasks using as Kubernetes CronJob
+# @section -- Maintenance
+maintenance:
+ # @schema additionalProperties: {type: string}
+ # -- Annotations to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ annotations: {}
+
+ # @schema item: object
+ # -- Extra environment variables to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ extraEnv: []
+ # - name: AWS_STORAGE_BUCKET
+ # value: s3://xxxxx/
+ # - name: AWS_ACCESS_KEY_ID
+ # valueFrom:
+ # secretKeyRef:
+ # name: aws-secret
+ # key: access_key_id
+ # - name: AWS_SECRET_ACCESS_KEY
+ # valueFrom:
+ # secretKeyRef:
+ # name: aws-secret
+ # key: secret_access_key
+
+ # @schema item: object
+ # -- Bulk import environment variables from Secrets or ConfigMaps to every
maintenance Cronjob object.
+ # @section -- Maintenance
+ envFrom: []
+ # - secretRef:
+ # name: polaris-env-secret
+ # - configMapRef:
+ # name: polaris-env-configmap
+
+ # @schema item: object
+ # -- Extra volumes to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ extraVolumes: []
+ # - name: extra-volume
+ # emptyDir: {}
+
+ # @schema item: object
+ # -- Extra volume mounts to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ extraVolumeMounts: []
+ # - name: extra-volume
+ # mountPath: /usr/share/extra-volume
+
+ # -- The container image used by every maintenance Cronjob object.
+ # @section -- Maintenance
+ image:
Review Comment:
This is assuming all jobs are based off polaris admin tool. Same commented
on the other one, do we want to support true "generic" job which users can
define any arbitrary jobs and run whatever images they want?
##########
helm/polaris/values.yaml:
##########
@@ -1167,3 +1167,104 @@ tasks:
# -- The maximum number of tasks that can be queued up for execution. If
unspecified or zero, defaults to Integer.MAX_VALUE.
# @section -- Tasks
maxQueuedTasks: 0 # 1000
+
+# @schema type: object
+# -- Configuration for maintenance tasks using as Kubernetes CronJob
+# @section -- Maintenance
+maintenance:
+ # @schema additionalProperties: {type: string}
+ # -- Annotations to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ annotations: {}
+
+ # @schema item: object
+ # -- Extra environment variables to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ extraEnv: []
+ # - name: AWS_STORAGE_BUCKET
+ # value: s3://xxxxx/
+ # - name: AWS_ACCESS_KEY_ID
+ # valueFrom:
+ # secretKeyRef:
+ # name: aws-secret
+ # key: access_key_id
+ # - name: AWS_SECRET_ACCESS_KEY
+ # valueFrom:
+ # secretKeyRef:
+ # name: aws-secret
+ # key: secret_access_key
+
+ # @schema item: object
+ # -- Bulk import environment variables from Secrets or ConfigMaps to every
maintenance Cronjob object.
+ # @section -- Maintenance
+ envFrom: []
+ # - secretRef:
+ # name: polaris-env-secret
+ # - configMapRef:
+ # name: polaris-env-configmap
+
+ # @schema item: object
+ # -- Extra volumes to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ extraVolumes: []
+ # - name: extra-volume
+ # emptyDir: {}
+
+ # @schema item: object
+ # -- Extra volume mounts to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ extraVolumeMounts: []
+ # - name: extra-volume
+ # mountPath: /usr/share/extra-volume
+
+ # -- The container image used by every maintenance Cronjob object.
+ # @section -- Maintenance
+ image:
+ # -- The image repository to pull from for the Polaris admin tool.
+ # @section -- Maintenance
+ repository: apache/polaris-admin-tool
+ # @schema enum: [Always, IfNotPresent, Never]
+ # -- The image pull policy.
+ # @section -- Maintenance
+ pullPolicy: IfNotPresent
+ # -- The image tag.
+ # @section -- Maintenance
+ tag: "latest" # This tag will be replaced with the chart version at
release time.
+
+ # @schema additionalProperties: {type: object, required: [schedule]}
+ # -- Define maintenance CronJobs. The key is the name of the job.
+ # @section -- Maintenance
+ jobs:
+ nosql-maintenance:
+ # -- Enable this maintenance job.
+ # @section -- Maintenance
+ enabled: false
+ # -- The schedule in Cron format.
+ # @section -- Maintenance
+ schedule: "0 2 * * *"
+ # -- The arguments to pass to the admin tool.
+ # @section -- Maintenance
+ args: ["nosql", "maintenance-run"]
Review Comment:
We can if we want to support true "generic" job. Same commented above. This
is assuming we will only support polaris admin tool images here thus only args
should be support.
##########
helm/polaris/templates/cronjob-maintenance.yaml:
##########
@@ -0,0 +1,149 @@
+{{/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+*/}}
+
+{{- range $jobName, $jobSpec := .Values.maintenance.jobs }}
+{{- if $jobSpec.enabled }}
+{{- $cronName := printf "%s-maint-%s" (include "polaris.fullname" $) $jobName
}}
+{{- if gt (len $cronName) 52 }}
+ {{- fail (printf "Maintenance CronJob name %q is %d chars; must be <= 52
(K8s appends a timestamp+random suffix to generated Pod names, which are capped
at 63). Shorten release name, nameOverride/fullnameOverride, or the job key
%q." $cronName (len $cronName) $jobName) }}
+{{- end }}
+---
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+ name: {{ $cronName }}
+ namespace: {{ $.Release.Namespace }}
+ labels:
+ {{- include "polaris.labels" $ | nindent 4 }}
+ {{- with $.Values.maintenance.annotations }}
+ annotations:
+ {{- tpl (toYaml .) $ | nindent 4 }}
+ {{- end }}
+spec:
+ schedule: {{ $jobSpec.schedule | quote }}
+ concurrencyPolicy: {{ default "Forbid" $jobSpec.concurrencyPolicy }}
+ {{- if hasKey $jobSpec "successfulJobsHistoryLimit" }}
+ successfulJobsHistoryLimit: {{ $jobSpec.successfulJobsHistoryLimit }}
+ {{- end }}
+ {{- if hasKey $jobSpec "failedJobsHistoryLimit" }}
+ failedJobsHistoryLimit: {{ $jobSpec.failedJobsHistoryLimit }}
+ {{- end }}
+ jobTemplate:
+ spec:
+ {{- if hasKey $jobSpec "backoffLimit" }}
+ backoffLimit: {{ $jobSpec.backoffLimit }}
+ {{- end }}
+ {{- with $jobSpec.activeDeadlineSeconds }}
+ activeDeadlineSeconds: {{ . }}
+ {{- end }}
+ template:
+ metadata:
+ labels:
+ helm.sh/chart: {{ include "polaris.chart" $ }}
+ app.kubernetes.io/name: {{ include "polaris.name" $ }}-maintenance
+ app.kubernetes.io/instance: {{ $.Release.Name }}
+ app.kubernetes.io/component: maintenance
+ app.kubernetes.io/managed-by: {{ $.Release.Service }}
+ {{- if $.Chart.AppVersion }}
+ app.kubernetes.io/version: {{ $.Chart.AppVersion | quote }}
+ {{- end }}
+ {{- with $jobSpec.podAnnotations }}
+ annotations:
+ {{- tpl (toYaml .) $ | nindent 12 }}
+ {{- end }}
+ spec:
+ {{- if $.Values.imagePullSecrets }}
+ imagePullSecrets:
+ {{- range $.Values.imagePullSecrets }}
+ - name: {{ . | quote }}
+ {{- end }}
+ {{- end }}
+ restartPolicy: {{ default "Never" $jobSpec.restartPolicy }}
+ {{- with $.Values.priorityClassName }}
+ priorityClassName: {{ tpl . $ | quote }}
+ {{- end }}
+ serviceAccountName: {{ include "polaris.serviceAccountName" $ }}
Review Comment:
Yes. I also worried the global one may has too much access than it really
needed. I will proceed with a diff SA and default back to global one if not
defined.
##########
helm/polaris/values.yaml:
##########
@@ -1167,3 +1167,104 @@ tasks:
# -- The maximum number of tasks that can be queued up for execution. If
unspecified or zero, defaults to Integer.MAX_VALUE.
# @section -- Tasks
maxQueuedTasks: 0 # 1000
+
+# @schema type: object
+# -- Configuration for maintenance tasks using as Kubernetes CronJob
+# @section -- Maintenance
+maintenance:
+ # @schema additionalProperties: {type: string}
+ # -- Annotations to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ annotations: {}
+
+ # @schema item: object
+ # -- Extra environment variables to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ extraEnv: []
+ # - name: AWS_STORAGE_BUCKET
+ # value: s3://xxxxx/
+ # - name: AWS_ACCESS_KEY_ID
+ # valueFrom:
+ # secretKeyRef:
+ # name: aws-secret
+ # key: access_key_id
+ # - name: AWS_SECRET_ACCESS_KEY
+ # valueFrom:
+ # secretKeyRef:
+ # name: aws-secret
+ # key: secret_access_key
+
+ # @schema item: object
+ # -- Bulk import environment variables from Secrets or ConfigMaps to every
maintenance Cronjob object.
+ # @section -- Maintenance
+ envFrom: []
+ # - secretRef:
+ # name: polaris-env-secret
+ # - configMapRef:
+ # name: polaris-env-configmap
+
+ # @schema item: object
+ # -- Extra volumes to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ extraVolumes: []
+ # - name: extra-volume
+ # emptyDir: {}
+
+ # @schema item: object
+ # -- Extra volume mounts to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ extraVolumeMounts: []
+ # - name: extra-volume
+ # mountPath: /usr/share/extra-volume
+
+ # -- The container image used by every maintenance Cronjob object.
+ # @section -- Maintenance
+ image:
+ # -- The image repository to pull from for the Polaris admin tool.
+ # @section -- Maintenance
+ repository: apache/polaris-admin-tool
+ # @schema enum: [Always, IfNotPresent, Never]
+ # -- The image pull policy.
+ # @section -- Maintenance
+ pullPolicy: IfNotPresent
+ # -- The image tag.
+ # @section -- Maintenance
+ tag: "latest" # This tag will be replaced with the chart version at
release time.
+
+ # @schema additionalProperties: {type: object, required: [schedule]}
+ # -- Define maintenance CronJobs. The key is the name of the job.
+ # @section -- Maintenance
+ jobs:
+ nosql-maintenance:
+ # -- Enable this maintenance job.
+ # @section -- Maintenance
+ enabled: false
+ # -- The schedule in Cron format.
+ # @section -- Maintenance
+ schedule: "0 2 * * *"
+ # -- The arguments to pass to the admin tool.
+ # @section -- Maintenance
+ args: ["nosql", "maintenance-run"]
+ # @schema enum: [Allow, Forbid, Replace]
+ # -- The concurrency policy, Valid values are: Allow, Forbid, Replace.
+ # @section -- Maintenance
+ concurrencyPolicy: Forbid
+ # @schema type: object
+ # -- Configures the resources requests and limits for this job's
container.
+ # @section -- Maintenance
+ resources:
+ {}
+ # limits:
+ # cpu: 100m
+ # memory: 128Mi
+ # requests:
+ # cpu: 100m
+ # memory: 128Mi
+ # Optional per-job knobs (uncomment to use):
+ # restartPolicy: Never
+ # backoffLimit: 6
+ # activeDeadlineSeconds: 1800
+ # successfulJobsHistoryLimit: 3
+ # failedjobsHistoryLimit: 1
+ # podAnnotations:
Review Comment:
1. annotations can be really powerful and potentially grant more access via
it. This is done to ensure the job pods don't get extra access from global when
it is not required.
2. let me add this. I pre-populated a set of labels to avoid pods get pick
up by service via pod selector. I was avoiding this in case if people add those
labels back. Maybe that will be more toward to operator issues?
##########
helm/polaris/templates/cronjob-maintenance.yaml:
##########
@@ -0,0 +1,149 @@
+{{/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+*/}}
+
+{{- range $jobName, $jobSpec := .Values.maintenance.jobs }}
+{{- if $jobSpec.enabled }}
+{{- $cronName := printf "%s-maint-%s" (include "polaris.fullname" $) $jobName
}}
+{{- if gt (len $cronName) 52 }}
+ {{- fail (printf "Maintenance CronJob name %q is %d chars; must be <= 52
(K8s appends a timestamp+random suffix to generated Pod names, which are capped
at 63). Shorten release name, nameOverride/fullnameOverride, or the job key
%q." $cronName (len $cronName) $jobName) }}
+{{- end }}
+---
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+ name: {{ $cronName }}
+ namespace: {{ $.Release.Namespace }}
+ labels:
+ {{- include "polaris.labels" $ | nindent 4 }}
+ {{- with $.Values.maintenance.annotations }}
+ annotations:
+ {{- tpl (toYaml .) $ | nindent 4 }}
+ {{- end }}
+spec:
+ schedule: {{ $jobSpec.schedule | quote }}
+ concurrencyPolicy: {{ default "Forbid" $jobSpec.concurrencyPolicy }}
+ {{- if hasKey $jobSpec "successfulJobsHistoryLimit" }}
+ successfulJobsHistoryLimit: {{ $jobSpec.successfulJobsHistoryLimit }}
+ {{- end }}
+ {{- if hasKey $jobSpec "failedJobsHistoryLimit" }}
+ failedJobsHistoryLimit: {{ $jobSpec.failedJobsHistoryLimit }}
+ {{- end }}
+ jobTemplate:
+ spec:
+ {{- if hasKey $jobSpec "backoffLimit" }}
+ backoffLimit: {{ $jobSpec.backoffLimit }}
+ {{- end }}
+ {{- with $jobSpec.activeDeadlineSeconds }}
+ activeDeadlineSeconds: {{ . }}
+ {{- end }}
+ template:
+ metadata:
+ labels:
+ helm.sh/chart: {{ include "polaris.chart" $ }}
+ app.kubernetes.io/name: {{ include "polaris.name" $ }}-maintenance
+ app.kubernetes.io/instance: {{ $.Release.Name }}
+ app.kubernetes.io/component: maintenance
+ app.kubernetes.io/managed-by: {{ $.Release.Service }}
+ {{- if $.Chart.AppVersion }}
+ app.kubernetes.io/version: {{ $.Chart.AppVersion | quote }}
+ {{- end }}
+ {{- with $jobSpec.podAnnotations }}
+ annotations:
+ {{- tpl (toYaml .) $ | nindent 12 }}
+ {{- end }}
+ spec:
+ {{- if $.Values.imagePullSecrets }}
+ imagePullSecrets:
+ {{- range $.Values.imagePullSecrets }}
+ - name: {{ . | quote }}
+ {{- end }}
+ {{- end }}
+ restartPolicy: {{ default "Never" $jobSpec.restartPolicy }}
+ {{- with $.Values.priorityClassName }}
+ priorityClassName: {{ tpl . $ | quote }}
+ {{- end }}
+ serviceAccountName: {{ include "polaris.serviceAccountName" $ }}
+ automountServiceAccountToken: false
+ {{- if $.Values.podSecurityContext }}
+ securityContext:
+ {{- tpl (toYaml $.Values.podSecurityContext) $ | nindent 12 }}
+ {{- end }}
+ containers:
+ - name: polaris-admin-tool
Review Comment:
That is an interesting aspect. So you think people may be running
non-polaris admin jobs here? I thought we only want to support people to define
what type of maintenance jobs they want to run (with assumptions everything
will be from polaris-admin-tool).
##########
helm/polaris/values.yaml:
##########
@@ -1167,3 +1167,104 @@ tasks:
# -- The maximum number of tasks that can be queued up for execution. If
unspecified or zero, defaults to Integer.MAX_VALUE.
# @section -- Tasks
maxQueuedTasks: 0 # 1000
+
+# @schema type: object
+# -- Configuration for maintenance tasks using as Kubernetes CronJob
+# @section -- Maintenance
+maintenance:
+ # @schema additionalProperties: {type: string}
+ # -- Annotations to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ annotations: {}
+
+ # @schema item: object
+ # -- Extra environment variables to add to every maintenance Cronjob object.
+ # @section -- Maintenance
+ extraEnv: []
Review Comment:
I am not sure if it is a good idea to merge those. For example, assuming
people is using `extraEnv` to populate AWS related credential info (such as the
commented out values), we won't need them on those maintenance jobs as they are
interacting with backend database directly. Thus, for anything that may expose
extra privileges, I think it is better to keep them seperate. For others such
as pod security context, those just tied down the access, which I think we
should get from global. Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
