At least for the k8s executor it would be easier to just restart the scheduler pod. I can see some potential issues if some configs are stored in local variables. On Wed, Dec 6, 2017 at 11:45 AM Feng Lu <[email protected]> wrote:
> Hi, > > It's probably well-known that Airflow only loads config file (i.e., > airflow.cfg) at instance creation time, if one needs to change the config > file, all Airflow instances have to be restarted (understand that Airflow > worker does restart itself for each task execution and therefore picks up > the latest config updates). > > Are people from this mailing group interested in adding dynamic config > loading support inside AirflowConfigParser ( > > https://github.com/apache/incubator-airflow/blob/master/airflow/configuration.py#L110 > )? > > Initial implementation ideas: > - introduce threadling.Rlock to AirflowConfigParser and guard all method > access > - add a periodical timer task that reads in the config file (of course > needs to acquire the lock beforehand). > > Since config data is always accessed via the AirflowConfigParser object, > this essentially gives us dynamic config update without restarting Airflow > scheduler/webserver. > > Thoughts? > Thank you. > > Feng >
