Hi all,

TL;DR: With default settings, a deprecation warning attributed to the user's 
Dag file is dropped. It shows up in neither the Dag processor log nor the task 
log. I'd like to agree on a fix. Reproduction is in [1].

Example:

    EksClusterStateSensor(task_id="eks", cluster_name="c", region="us-east-1")
    EmptyOperator(task_id="t", email="[email protected]")

Both `region` and `email` are deprecated, and neither warning is logged when 
the Dag file is parsed.

It looks like Airflow's warning filter only enables DeprecationWarning for 
modules named `airflow*`, and these warnings are attributed to the Dag file. 
Tests do not catch it because pytest.warns records warnings regardless of 
filters.

This has been worked around twice by adding FutureWarning subclasses for 
specific warnings [2][3]. Provider warnings came up in [4] and were left 
unchanged.

Potential approaches:

A. Add a filter in core for AirflowProviderDeprecationWarning and 
RemovedInAirflow4Warning. Small change, and the log points at the right line of 
the Dag file. Draft PR for provider warnings: [5].
B. Make those classes subclass FutureWarning, as in [2][3]. Code that catches 
or filters DeprecationWarning stops matching them.
C. Leave core alone and have providers attribute warnings to their own module. 
Needs no core change, but every provider has to do it and the log no longer 
points at the user's line.

A and B only take effect on Airflow versions that ship them. With either, a 
parse-time warning also shows up in each task log, as FutureWarnings do today.

I'd like to hear thoughts on:
1. For core: A, B, or something else?
2. Include RemovedInAirflow4Warning, or only provider warnings?

Thanks,
Ramit

[1] https://github.com/apache/airflow/issues/77273
[2] https://github.com/apache/airflow/pull/55881
[3] https://github.com/apache/airflow/pull/76007
[4] https://github.com/apache/airflow/issues/55889
[5] https://github.com/apache/airflow/pull/77276

Reply via email to