Hey Paul, While I recognize the use case for this, I view it as more of a deployment-related thing. It adds some complexity to Airflow, and I think it's better suited to be run as part of a deployment system. A Python script can be written that uses Airflow's models and SQLAlchemy to initiate and validate DBs are exactly as expected. This includes connections, variables, pools, etc.
My vote would be to keep this outside of Airflow-proper, but have some docs on the wiki describing how to init DBs properly (including connections, vars, pools, etc). Cheers, Chris On Thu, Jun 2, 2016 at 3:07 PM, Paul Rhodes <[email protected]> wrote: > A while ago I worked on a branch for allowing the initdb static > connections (from utils/db.py) > > > At present when you run 'airflow initdb' it creates a set of default > connections hardcoded in utils/db.py. However, for anyone deploying with > puppet/chef/ansible/etc who is looking to be able to idempotently assert > the state of connections this leaves a gap - how should I initialise my > airflow to a known state and assert it's in that state over time? > > > After discussing it a little with Jerminah a while ago I put together a > branch which separated this out into a separate config file whilst also > adding the ability to assert variables and pools. I've implemented it as a > YAML file since this easily converts into a dict whilst also allowing for > comments to be easily embedded. > > > Logically, when initdb was executed it did the following: > > * Checked for the presence of an airflow-initdb.yaml file and if not > present created one with the YAML that represents the default set > previously hardcoded. > * Read in the YAML sections associated with connections, pools and > variables > * Iterated over each item in turn. If the item already exists in the > database it skips it, unless the newly defined arg '--replace' is specified > with initdb. > > I didn't want to make a PR out of it without discussing it on here. The > branch is a little old right now and I haven't rebased it but you can see > the working version (sans tests) at > https://github.com/withnale/airflow/tree/initdb_from_config - I'm happy > to wrap it in more testing... > > Paul > >
