A while ago I worked on a branch for allowing the initdb static connections 
(from utils/db.py)


At present when you run 'airflow initdb' it creates a set of default 
connections hardcoded in utils/db.py. However, for anyone deploying with 
puppet/chef/ansible/etc who is looking to be able to idempotently assert the 
state of connections this leaves a gap - how should I initialise my airflow to 
a known state and assert it's in that state over time?


After discussing it a little with Jerminah a while ago I put together a branch 
which separated this out into a separate config file whilst also adding the 
ability to assert variables and pools. I've implemented it as a YAML file since 
this easily converts into a dict whilst also allowing for comments to be easily 
embedded.


Logically, when initdb was executed it did the following:

  *   Checked for the presence of an airflow-initdb.yaml file and if not 
present created one with the YAML that represents the default set previously 
hardcoded.
  *   Read in the YAML sections associated with connections, pools and variables
  *   Iterated over each item in turn. If the item already exists in the 
database it skips it, unless the newly defined arg '--replace' is specified 
with initdb.

I didn't want to make a PR out of it without discussing it on here. The branch 
is a little old right now and I haven't rebased it but you can see the working 
version (sans tests) at 
https://github.com/withnale/airflow/tree/initdb_from_config - I'm happy to wrap 
it in more testing...

Paul

Reply via email to