Hi all,

I have been thinking about proposing a public testing package for Airflow
users. Before I spend more time on it, I would like to hear your opinions.

The problem I see is that there is no good way to test a Dag with pytest
today. The docs [1] suggest calling the operator directly with a context
you build by yourself, or running the whole Dag in a way that was made for
debugging.

TBH I don't think either one is good enough.

Why I think we need this:

- If users build the context and fake the inputs by hand, they are testing
their own setup, not Airflow. The test still passes even when Airflow's
behavior changes.

- Users who want a real test have to use our private interfaces. They only
find out it is broken after they upgrade.

- A real test needs a prepared metadata database today, which is slow and
hard to set up in CI. A test for a single task should not need one. (This
might be related to the recent discussion on a faster `import airflow` [2].)

- Many Dags are written with AI agents now. I think a fast and real test is
what users will need most. It gives agents a quick way to verify their work.

I found an earlier discussion started by Jarek [3], where releasing our
“whole internal test utilities” to users was rejected because of the
compatibility burden. What I want to discuss here is different: a public
API designed for users, not our internal tools. And we already maintain the
hard part, because our own tests run tasks this way.

I would like to hear your thoughts. If it sounds reasonable, I would be
happy to write an AIP for it!

Thanks,
Andrew

[1]
https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#unit-tests
[2] https://lists.apache.org/thread/1nhdc8y1x7dclcpcck98q5chzh1tvodk
[3] https://lists.apache.org/thread/jonodvjnycv01qcsnlqczpvrjhfsx04g

Reply via email to