Also you might take a look at Airflow Summit videos https://www.youtube.com/playlist?list=PLGudixcDaxY2NIjMYT8t5zA9KJ47wTCkM -> and look back to 2023. There were at least several talks about using LLMs to generate Airflow Dags, and our users are doing it already - and I guess it's quite natural for people to generate the Dags with the help of LLMs already.
On Mon, Jul 7, 2025 at 8:52 AM Jarek Potiuk <ja...@potiuk.com> wrote: > FYI I added your email directly - because apparently you are not > subscribed to devlist - please do subscribe following the "community" tab > on our website. > > I don't want to cut down your wings and excitement, but this is a > deliberate choice that Airflow UI does not allow to author DAGs. This is a > security feature. And our security model > https://airflow.apache.org/docs/apache-airflow/stable/security/security_model.html > is very clear that "UI users" do not have (and should not have) > capabilities of authoring DAGs (not as Python code - that allows > arbitrary code execution). Maybe (and that is something we might consider > in the future) if there is a declarative way of creating DAGs which does > not allow to provide arbitrary code, we could allow that, but we have not > even settled on the idea of having a single declarative way of creating > Dags. > > Also Airflow DAGS are just Python Code placed in a folder. And there is > absolutely nothing stopping you to open your IDE with Claude , Cursor, > Copilot, use the prompt of your choice and ... generate DAGs with LLM. > There is absolutely no need to have a UI for that.- all the IDEs out there > already have a fantastic LLM integration, with capability of adding prompt, > using MCP servers (there are even several MCP servers for Airflow created > by the community and we are discussing about creating our own MCP server > https://lists.apache.org/thread/xgd66v6s7zf0xkvy3c7ysqvn4csgmw0 - those > IDEs have code completion, syntax check, allow you to interact with the > Agents and approve/reject proposals when you are using agents to create > your DAGs. They even allow you to use your own models that can be RAG-ified > based - for example - on the private DAGs your company might have. This all > works **today**. > > I don't think personally there is any benefit of creating a similar > feature in Airflow UI. I can't see any to be honest. Maybe others have a > different opinion or maybe you can explain what benefits you see by adding > such a "UI feature" to Airflow itself (and also the problem about security > is extremely important and a huge blocker for the whole idea - until this > is somewhat addressed the whole idea is basically impossible to be accepted > by the community. > > J. > > > On Mon, Jul 7, 2025 at 8:37 AM Harikrishnan Girikumar < > harikrishnangiriku...@gmail.com> wrote: > >> Hello Team, >> >> My name is Harikrishnan(Hari), I have an idea/improvement proposal for >> Airflow. >> LLM-powered feature within Apache Airflow to significantly enhance the >> DAG >> authoring experience. Users would be able to provide natural language >> descriptions or queries and leverage Large Language Models (LLMs) to >> automatically generate and modify Airflow DAGs. This aims to democratize >> DAG creation, reduce the learning curve for new users, and accelerate the >> development of complex workflows. For example: We can have a UI tab in >> Airflow where users can add their respective authentication credentials >> for >> the LLMs they want to use (OpenAI, Claude or their personal model serving >> link etc.) they can select their AI from drop down and a chat window to >> input queries like: Create a DAG to copy my data from S3 to Postgres and >> the code generated would be copied to DAG folder. We can restrict the >> Prompts to be strictly for DAG generation for initial trial, further down >> the line a RAG feature could be added where a Vectorized version of >> Airflow >> documentation is used to improve the accuracy of DAG creation. >> >> I am really excited about this feature, this would reduce the learning >> curve and improve the interaction for new users. Let me know your >> thoughts, >> looking forward to hearing from the team. >> >> Regards, >> Hari >> >
