Using Too Many Aiflow Variables in Dag is Good thing ?

[Pramiti Goel]

Hi,

We want to make owner and email Id general, so we don't want to put in
airflow dag. Using variables will help us in changing the email/owner
later, if there are lot of dags of same owner.

For example:


default_args = {
    'owner': Variable.get('test_owner_de'),
    'depends_on_past': False,
    'start_date': datetime(2018, 10, 17),
    'email': Variable.get('de_infra_email'),
    'email_on_failure': True,
    'email_on_retry': True,
    'retries': 2,
    'retry_delay': timedelta(minutes=1)}


Looking into the code of Airflow, it is making connection session everytime
the variable is created, and then close it. (Let me know if I understand
wrong). If there are many dags with variables in default args running
parallel, querying variable table in MySQL, will it have any sort of
limitation on number of sessions of SQLAlchemy ? Will that make dag slow as
there will be many queries to mysql for each dag? is the above approach
good ?

 >using Airlfow 1.9

Thanks,
Pramiti.