opeida commented on issue #51816: URL: https://github.com/apache/airflow/issues/51816#issuecomment-2980811694
> [@opeida](https://github.com/opeida) The ideal way would be to not access Variables or Connections at the top of your file (i.e outside Task Context) @kaxil are there workarounds to eliminate the use of top-level variables for dynamic DAG generation? We pull data from numerous accounts of a third-party provider. The accounts variable can change dynamically and stores sensitive information including API keys. In all other cases, we utilize Jinja templates following Airflow's best practices. ``` provider_name = "3rd-party provider" ALL_ACCOUNTS = Variable.get(f"{provider_name}_accounts") for account in ALL_ACCOUNTS: with DAG( f"{provider_name}.{account.get('name')}", default_args=default_args, schedule="10 * * * *", catchup=False, max_active_runs=1 ) as dag: # DAG operators ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
