Hi all,
TL;DR: With default settings, a deprecation warning attributed to the user's
Dag file is dropped. It shows up in neither the Dag processor log nor the task
log. I'd like to agree on a fix. Reproduction is in [1].
Example:
EksClusterStateSensor(task_id="eks", cluster_name="c", region="us-east-1")
EmptyOperator(task_id="t", email="[email protected]")
Both `region` and `email` are deprecated, and neither warning is logged when
the Dag file is parsed.
It looks like Airflow's warning filter only enables DeprecationWarning for
modules named `airflow*`, and these warnings are attributed to the Dag file.
Tests do not catch it because pytest.warns records warnings regardless of
filters.
This has been worked around twice by adding FutureWarning subclasses for
specific warnings [2][3]. Provider warnings came up in [4] and were left
unchanged.
Potential approaches:
A. Add a filter in core for AirflowProviderDeprecationWarning and
RemovedInAirflow4Warning. Small change, and the log points at the right line of
the Dag file. Draft PR for provider warnings: [5].
B. Make those classes subclass FutureWarning, as in [2][3]. Code that catches
or filters DeprecationWarning stops matching them.
C. Leave core alone and have providers attribute warnings to their own module.
Needs no core change, but every provider has to do it and the log no longer
points at the user's line.
A and B only take effect on Airflow versions that ship them. With either, a
parse-time warning also shows up in each task log, as FutureWarnings do today.
I'd like to hear thoughts on:
1. For core: A, B, or something else?
2. Include RemovedInAirflow4Warning, or only provider warnings?
Thanks,
Ramit
[1] https://github.com/apache/airflow/issues/77273
[2] https://github.com/apache/airflow/pull/55881
[3] https://github.com/apache/airflow/pull/76007
[4] https://github.com/apache/airflow/issues/55889
[5] https://github.com/apache/airflow/pull/77276