hussein-awala commented on PR #29772: URL: https://github.com/apache/airflow/pull/29772#issuecomment-1445365103
I just realized that there is not conf to pass kwargs to TaskHandler class, and that the `delete_local_copy` of wasb handler is [fixed to False](https://github.com/apache/airflow/blob/main/airflow/config_templates/airflow_local_settings.py#L255-L255), so the config is necessary to activate the option. I'm wondering if we add a new conf `remote_task_handler_kwargs` to let old Airflow version users use the future version without the need to upgrade it. > Yes, I thought about this, and I had two different options to add the config: > - Add the config and use it as default value for delete_local_copy -> the users of Airflow < 2.6.0 should add the config manually because to avoid [AirflowConfigException](https://github.com/apache/airflow/blob/main/airflow/configuration.py#L661-L661) > - Add the config and use it in a try except with a second default value False defined in each provider -> All the users can configure the RemoteTaskHandler using Airflow config or keyword config and it is fully bc My initial idea was based on this conf, but now the only solution to override the conf `delete_local_copy` for Airflow < 2.6.0 users is extending the TaskHandler class and override the value of this conf. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
