Hello,

We are currently running airflow v1.10.6 on Kubernetes. We are currently
using a custom helm chart to sync dags and airflow variables from GCS.
While airflow provides support to just copy dags to a folder and then dags
are automatically loaded, airflow variables are not imported from a
file, one needs to execute a command to import them from a file.
We require to store variables in the repository for ease of rollbacks.

Our setup to sync airflow variables is as follows:

Repository -> GCS -> Kubernetes sidecar container -> Import variables from
file.

In order to import variables from a file, we execute the command: airflow
variables -i <file-name>
This command first deletes the variables and then imports them, in case
there is a dag running which uses the variable, it fails since the variable
doesn't exist. Thus we added a check to only execute the command if the
variables file has changed.
This setup has been working fine for us, still, there are chances that the
dags might fail because of syncing airflow variables.

Can someone please share their setup of how to sync variables for airflow
running on Kubernetes.

Thanks and regards,
Maulik

Reply via email to