Thanks Max. We are currently storing the metadata in our DB and reading off that.
Since this setup will be exclusively used via our UI, we don't have a VCS configured, but I agree with the idea of generating the DSL so that the user can learn how Airflow works. I was hoping there would be an option other than generating Python code, but looks like there isn't any. I saw a section in the documentation titled experimental API. Is there a possibility that an API will be exposed to author workflows? Sent from my iPhone > On Jul 9, 2017, at 10:31 AM, Maxime Beauchemin <[email protected]> > wrote: > > One option is to save the metadata in a relational database from your UI > and read from there as you create the DAG. The challenge might be around > change management where you may want to recreate things that you get for > free in source control. > > To get the best of both worlds (UI and DSL in source control) I've seen it > done where a UI generates Python code and submits a PR on your behalf. That > allows for the user of the UI to learn the underlying DSL and follows the > same process (code review) as other workflow authors. > > Max > >> On Fri, Jul 7, 2017 at 9:05 AM, Arpan Dhandhania <[email protected]> wrote: >> >> Hi, >> >> We are building a web-based UI to author flows. A user will drag-n-drop >> tasks on a canvas and link them to define the dependency. In Airflow, these >> tasks are PythonOperators that invoke functions containing the business >> logic (written by us). >> >> I was wondering what is the best way to submit the UI-generated DAG to >> Airflow. As a test, we generated Python code and saved it to a file in the >> DAGs directory. While that works, I don’t think it is the best solution. >> >> Is there an API to define the DAG? Could Jinja templating help? >> >> Any recommendations of how to do this elegantly will be appreciated. >> >> Arpan Dhandhania >> e: [email protected] >> m: +91 782 939 5618 >> >> — >> Perfection is achieved, not when there is nothing more to add, but when >> there is nothing left to take away. >> >>
