All, After a few weeks of work I have finalized the implementation of a Rest API Framework. Out of the box it supports Kerberos authentication, which is now fully end to end tested on Travis’ with a working KDC. You can also switch the CLI to use the API endpoints when available. Currently, only the “trigger_dag” functionality is available this way, but I hope others to pick up and create new endpoints that the CLI can then use.
For Contributors: In case you are implementing new functionality in the CLI please make sure to implement the actual functionality in api/common/…/<function_name.py> and expose it through api_client (abstract), json_client (JSON), local_client (direct). Endpoints are defined in www/api/experimental. Direct exposure in cli.py I would consider deprecated and I would prefer to deny it from now on. Hopefully, this gives us a gradual path to improved integration and improved security while maintaining backwards compatibility. Also note that the APIs are still marked experimental and are subject to change. Next steps: - Swagger definitions (http://swagger.io) - Research possible integration between different authentication backends - Use “airflow api” instead of “airflow webserver” to separate concerns - Remove all direct DB access from cli.py - Improve documentation - Design API graduation roadmap (when is something not experimental anymore) Feedback obviously appreciated. Bolke
