ramitkataria opened a new issue, #77273:
URL: https://github.com/apache/airflow/issues/77273

   ### Under which category would you file this issue?
   
   Airflow Core
   
   ### Apache Airflow version
   
    main (3.5.0 development)
   
   ### What happened and how to reproduce it?
   
   **Issue description**
   
   `AirflowProviderDeprecationWarning`s do not reach users when the deprecated
   feature is used from a Dag file or other user code. They appear neither in 
the
   Dag processor's per-file log nor in the task log.
   
   `airflow-core/src/airflow/configuration.py` enables `DeprecationWarning` only
   for warnings attributed to a module whose name starts with `airflow`. A 
provider
   deprecation is attributed to the code using the deprecated feature:
   
   - Operator `__init__` warnings are attributed to the Dag file, because
     `fixup_decorator_warning_stack` shifts the stacklevel past the
     `apply_defaults` wrapper. The Dag module is named
     `unusual_prefix_<hash>_<file>`.
   - Hook method warnings with `stacklevel=2` are attributed to the caller, 
which
     is user code when a `@task` function or a custom operator calls the hook.
   
   Neither matches the `airflow` module filter, so Python's default
   `ignore::DeprecationWarning` applies. The Dag importer records warnings 
without
   changing filters, so nothing is captured.
   
   **Steps to reproduce**
   
   1. Add this Dag file (Amazon provider used as the example):
   
   ```python
   import warnings
   
   from airflow.providers.amazon.aws.hooks.kinesis import FirehoseHook
   from airflow.providers.amazon.aws.operators.glue_crawler import 
GlueCrawlerOperator
   from airflow.providers.amazon.aws.sensors.eks import EksClusterStateSensor
   from airflow.sdk import DAG, task
   
   warnings.warn("control UserWarning", UserWarning, stacklevel=1)
   
   with DAG("deprecation_parse", schedule=None):
       EksClusterStateSensor(task_id="eks", cluster_name="c", 
region="us-east-1")
       GlueCrawlerOperator(task_id="crawler", config={"Name": "x"})
   
   with DAG("deprecation_runtime", schedule=None):
   
       @task
       def use_deprecated_hook():
           FirehoseHook(delivery_stream="s")
   
       use_deprecated_hook()
   ```
   
   2. Run `airflow dag-processor --num-runs 1` and read the per-file log under
      `logs/dag_processor/`. It contains the control `UserWarning` and neither 
of
      the two provider deprecations.
   3. Run `airflow dags test deprecation_runtime`. The task output has no
      `FirehoseHook` deprecation.
   
   No `PYTHONWARNINGS` or `-W` option is set.
   
   ### What you think should happen instead?
   
   The deprecation warnings should be logged like the control warning, pointing 
at
   the Dag file and line. Providers raise them so users can migrate before a
   removal, and today users only find out when the feature is gone.
   
   ### Anything else?
   
   - The module filter dates from 1.7.1 (2016). Operator `__init__` warnings 
have
     been attributed to the Dag file since 2.1.0 (#15667), so they have been 
hidden
     since then.
   - `RemovedInAirflow4Warning` goes through the same filter. For example, the
     `email` deprecation from `EmptyOperator(task_id="t", 
email="[email protected]")`
     in a Dag file is hidden as well.
   - `airflow-core/src/airflow/dag_processing/dagbag.py` already special-cases
     `TaskGroupCycleDeprecationWarning` for this reason.
   - Tests do not catch this because pytest enables all deprecation warnings.
   - Proposed fix: add a `default` filter for the 
`AirflowProviderDeprecationWarning`
     category with no module restriction. Whether `RemovedInAirflow4Warning` 
should
     get the same treatment is a question for maintainers.
   
   ### Operating System
   
   Breeze on MacOS 26.7.3
   
   ### Deployment
   
   Other
   
   ### Apache Airflow Provider(s)
   
   _No response_
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Official Helm Chart version
   
   Not Applicable
   
   ### Kubernetes Version
   
   _No response_
   
   ### Helm Chart configuration
   
   _No response_
   
   ### Docker Image customizations
   
   _No response_
   
   ### Anything else?
   
   - Prior reports: #55875 reported the same symptom for deprecated imports.
     #55881 traced it to the same filter and fixed only imports, by adding
     `DeprecatedImportWarning` (a `FutureWarning`). #55889 discussed provider
     warnings and left `AirflowProviderDeprecationWarning` unchanged, so 
provider
     deprecations are still hidden.
   - #55881 avoided changing the filters so task logs would not show every
     deprecation warning. A filter on the `AirflowProviderDeprecationWarning`
     category alone does not do that. The other option, floated in #55889, is to
     make it a `FutureWarning` subclass.
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to