h00jraq opened a new issue, #31458:
URL: https://github.com/apache/airflow/issues/31458
### Official Helm Chart version
1.9.0 (latest released)
### Apache Airflow version
2.5.0
### Kubernetes Version
1.26
### Helm Chart configuration
Here is the part of postgresql in my values.yaml file:
airflow:
....
postgresql:
primary:
persistence:
enabled: true
size: 50Gi
existingClaim: airflow-postgresql-db
resources:
limits:
cpu: 1000m
memory: 2048Mi
requests:
cpu: 200m
memory: 256Mi
....
I've also tried with below but it has not changed anything.
```
airflow:
....
postgresql:
persistence:
enabled: true
size: 50Gi
existingClaim: airflow-postgresql-db
resources:
limits:
cpu: 1000m
memory: 2048Mi
requests:
cpu: 200m
memory: 256Mi
....
```
tried also with different indentation:
```
airflow:
....
postgresql:
persistence:
enabled: true
size: 50Gi
existingClaim: airflow-postgresql-db
resources:
limits:
cpu: 1000m
memory: 2048Mi
requests:
cpu: 200m
memory: 256Mi
....
```
PVC and PV are created and in bound state:
```
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
annotations:
volume.beta.kubernetes.io/storage-provisioner: disk.csi.azure.com
volume.kubernetes.io/storage-provisioner: disk.csi.azure.com
finalizers:
- kubernetes.io/pvc-protection
labels:
app.kubernetes.io/component: primary
app.kubernetes.io/instance: airflow
app.kubernetes.io/name: postgresql
name: airflow-postgresql-db
namespace: airflow-sandbox
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Gi
storageClassName: default
volumeMode: Filesystem
volumeName: postgres-db-pv
```
PV:
```
apiVersion: v1
kind: PersistentVolume
metadata:
annotations:
pv.kubernetes.io/provisioned-by: disk.csi.azure.com
finalizers:
- kubernetes.io/pv-protection
- external-attacher/disk-csi-azure-com
name: postgres-db-pv
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 50Gi
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
name: airflow-postgresql-db
namespace: airflow-sandbox
csi:
driver: disk.csi.azure.com
volumeAttributes:
csi.storage.k8s.io/pvc/namespace: airflow-sandbox
requestedsizegib: "50"
skuname: StandardSSD_LRS
volumeHandle:
/mysub/myrg/providers/Microsoft.Compute/disks/postgres-db-pv
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: topology.disk.csi.azure.com/zone
operator: In
values:
- uksouth-3
persistentVolumeReclaimPolicy: Delete
storageClassName: default
volumeMode: Filesystem
```
### Docker Image customizations
I use standard docker image from Airflow
### What happened
For some reason, postgres is always creating pvc named
`data-airflow-postgresql-0`. Is it possible that this comes from
volumeClaimTemplates or something? This is what I see when running `kubectl get
statefulsets.apps -n airflow-sandbox airflow-postgresql -o yaml`
```
apiVersion: apps/v1
kind: StatefulSet
metadata:
annotations:
[meta.helm.sh/release-name](http://meta.helm.sh/release-name): airflow
[meta.helm.sh/release-namespace](http://meta.helm.sh/release-namespace):
airflow-sandbox
creationTimestamp: "2023-05-22T07:08:01Z"
generation: 1
labels:
[app.kubernetes.io/component](http://app.kubernetes.io/component):
primary
[app.kubernetes.io/instance](http://app.kubernetes.io/instance): airflow
[app.kubernetes.io/managed-by](http://app.kubernetes.io/managed-by): Helm
[app.kubernetes.io/name](http://app.kubernetes.io/name): postgresql
[helm.sh/chart](http://helm.sh/chart): postgresql-12.1.9
name: airflow-postgresql
namespace: airflow-sandbox
resourceVersion: "88975048"
uid: 58c0d7d5-5dd3-428a-bba0-fd3a300b4a6f
spec:
podManagementPolicy: OrderedReady
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
[app.kubernetes.io/component](http://app.kubernetes.io/component):
primary
[app.kubernetes.io/instance](http://app.kubernetes.io/instance):
airflow
[app.kubernetes.io/name](http://app.kubernetes.io/name): postgresql
serviceName: airflow-postgresql-hl
template:
metadata:
creationTimestamp: null
labels:
[app.kubernetes.io/component](http://app.kubernetes.io/component):
primary
[app.kubernetes.io/instance](http://app.kubernetes.io/instance):
airflow
[app.kubernetes.io/managed-by](http://app.kubernetes.io/managed-by):
Helm
[app.kubernetes.io/name](http://app.kubernetes.io/name): postgresql
[helm.sh/chart](http://helm.sh/chart): postgresql-12.1.9
name: airflow-postgresql
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
[app.kubernetes.io/component](http://app.kubernetes.io/component): primary
[app.kubernetes.io/instance](http://app.kubernetes.io/instance): airflow
[app.kubernetes.io/name](http://app.kubernetes.io/name):
postgresql
topologyKey:
[kubernetes.io/hostname](http://kubernetes.io/hostname)
weight: 1
containers:
- env:
- name: BITNAMI_DEBUG
value: "false"
- name: POSTGRESQL_PORT_NUMBER
value: "5432"
- name: POSTGRESQL_VOLUME_DIR
value: /bitnami/postgresql
- name: PGDATA
value: /bitnami/postgresql/data
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
key: postgres-password
name: airflow-postgresql
- name: POSTGRESQL_ENABLE_LDAP
value: "no"
- name: POSTGRESQL_ENABLE_TLS
value: "no"
- name: POSTGRESQL_LOG_HOSTNAME
value: "false"
- name: POSTGRESQL_LOG_CONNECTIONS
value: "false"
- name: POSTGRESQL_LOG_DISCONNECTIONS
value: "false"
- name: POSTGRESQL_PGAUDIT_LOG_CATALOG
value: "off"
- name: POSTGRESQL_CLIENT_MIN_MESSAGES
value: error
- name: POSTGRESQL_SHARED_PRELOAD_LIBRARIES
value: pgaudit
image:
[docker.io/bitnami/postgresql:11](http://docker.io/bitnami/postgresql:11)
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- /bin/sh
- -c
- exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432
failureThreshold: 6
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: postgresql
ports:
- containerPort: 5432
name: tcp-postgresql
protocol: TCP
readinessProbe:
exec:
command:
- /bin/sh
- -c
- -e
- |
exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432
[ -f /opt/bitnami/postgresql/tmp/.initialized ] || [ -f
/bitnami/postgresql/.initialized ]
failureThreshold: 6
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
requests:
cpu: 250m
memory: 256Mi
securityContext:
runAsUser: 1001
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /dev/shm
name: dshm
- mountPath: /bitnami/postgresql
name: data
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 1001
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
volumes:
- emptyDir:
medium: Memory
name: dshm
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: null
name: data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 8Gi
volumeMode: Filesystem
status:
phase: Pending
status:
availableReplicas: 1
collisionCount: 0
currentReplicas: 1
currentRevision: airflow-postgresql-78588bb7f9
observedGeneration: 1
readyReplicas: 1
replicas: 1
updateRevision: airflow-postgresql-78588bb7f9
updatedReplicas: 1
```
### What you think should happen instead
I would like postgresql to use the PVC which I have declared in
existingClaim in my values.yaml
### How to reproduce
1. Create PVC based on the one I have provided
2. Add your PVC name to `existingClaim:` in postgresql part of `values.yaml`
3. Uninstall your Airflow release and delete pvc previously used by
postgresql pod (you will loose your data!!)
4. Install new Airflow release - Airflow will ignore the PVC which you had
used in `existingClaim:` and will create and use `data-airflow-postgresql-0`
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
