Hi all, I have been thinking about proposing a public testing package for Airflow users. Before I spend more time on it, I would like to hear your opinions.
The problem I see is that there is no good way to test a Dag with pytest today. The docs [1] suggest calling the operator directly with a context you build by yourself, or running the whole Dag in a way that was made for debugging. TBH I don't think either one is good enough. Why I think we need this: - If users build the context and fake the inputs by hand, they are testing their own setup, not Airflow. The test still passes even when Airflow's behavior changes. - Users who want a real test have to use our private interfaces. They only find out it is broken after they upgrade. - A real test needs a prepared metadata database today, which is slow and hard to set up in CI. A test for a single task should not need one. (This might be related to the recent discussion on a faster `import airflow` [2].) - Many Dags are written with AI agents now. I think a fast and real test is what users will need most. It gives agents a quick way to verify their work. I found an earlier discussion started by Jarek [3], where releasing our “whole internal test utilities” to users was rejected because of the compatibility burden. What I want to discuss here is different: a public API designed for users, not our internal tools. And we already maintain the hard part, because our own tests run tasks this way. I would like to hear your thoughts. If it sounds reasonable, I would be happy to write an AIP for it! Thanks, Andrew [1] https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#unit-tests [2] https://lists.apache.org/thread/1nhdc8y1x7dclcpcck98q5chzh1tvodk [3] https://lists.apache.org/thread/jonodvjnycv01qcsnlqczpvrjhfsx04g
