I think we can make this non-breaking entirely? We can simply raise a subclass of both AirflowException and whatever we want to migrate to.
If we want to, we can emit a deprecation warning if AirflowException is imported directly by the user, but I feel even that is particularly needed. Raising a custom exception that’s as specific as practical is always the best practice. Using a custom exception that subclasses both built-in and AirflowException (but not AirflowException itself) is a good idea on its own. TP > On 31 May 2026, at 21:25, Jens Scheffler <[email protected]> wrote: > > Hi, > > I do not have a stron opinion on this, with the past decision we mainly > wanted to ensure code is clean. > > For me also Option (3) would be viable: We leave existing code as it is and > just ensure that new code is clean. But I am also okay to join the other > majority opinion. > > Jens > > On 31.05.26 14:08, Sameer Mesiah wrote: >> Hi, >> >> Thanks for opening this thread. This was long overdue considering that >> current migration efforts for AirflowException are more scattered rather >> than a consolidated pre-planned wave. >> >> In general, I agree with your overall approach. But there are a few things >> I want to raise in response to your proposal: >> >> On 1) >> >> I think Amogh already expanded on thiss somewhat. But I believe it is >> important to understand that AirflowException is not constrained to >> the provider layer but often bubble up from Airflow core.For example >> task_runner.py explicitly catches AirflowException as part of task >> execution and retry handling, and there are also usages in the hook and >> connection layers. >> >> But depending on whether we scope this to core + providers vs providers >> only, the migration strategy looks quite different. If the intention is >> provider-only migration, users will continue to encounter AirflowException >> from core Airflow components, which may make the overall exception model >> feel somewhat inconsistent. If the intention is project-wide deprecation, I >> think we need a broader plan than provider-level migrations alone. >> >> Also, as Amogh already implied (he is free to correct me in case I >> misinterpreted his reservations), deprecation warnings may not be >> worthwhile here. Unlike a deprecated API or method, this is not something >> users are actively choosing to call. In most cases, Airflow itself is >> deciding which exception type to raise. I am leaning against warnings due >> to the potential for log pollution and the relatively limited value they >> provide. >> >> On 2) >> >> I agree with your per-package migration proposal. But as Amogh said, we >> should ensure new exceptions inherit from AirflowException to prevent >> breaking changes across the provider ecosystem. I must add that there will >> inevitably be a period where users run multiple providers, some migrated >> and some not. This may result in a mixture of exception types appearing in >> logs and stack traces. While not ideal, this is probably unavoidable given >> independent provider versioning. As long as migrations are clearly >> documented, I do not see this as a major issue. >> >> My answers to your questions: >> >> *Should we add a deprecation warning to AirflowException?* >> >> I am +0. Not strongly opposed, but I am leaning no due to the risk of log >> pollution and the fact that the warning would be emitted by Airflow itself >> rather than as a result of an explicit user action. >> >> *Is the per-package migration approach reasonable?* >> >> Yes. No blocking concerns from my side. >> >> *Should we define a project-wide exception taxonomy, or leave exception >> choices to individual provider teams?* >> >> I would leave this to individual provider teams. The providers have very >> different APIs, error models, and opinions around exception handling. >> Attempting to enforce a central taxonomy across all providers may be >> impractical and may not provide much practical value over what we currently >> have in place. >> >> Thanks, >> Sameer Mesiah. >> >> On Sat, 30 May 2026 at 10:59, Shahar Epstein <[email protected]> wrote: >> >>> Hi all, >>> >>> This follows up on the earlier lazy-consensus >>> <https://lists.apache.org/thread/m86gs4mqt8hq1n26g0pp0fq5h5g0x2q9> and >>> related discussions around deprecating AirflowException. >>> >>> While working on a cleanup PR for the Google Cloud Run operator in the >>> Google provider (#67769), Amogh pointed out that replacing AirflowException >>> with built-in or custom exceptions is a breaking change. Users may rely on >>> AirflowException in callbacks (on_failure_callback, on_retry_callback) or >>> in custom wrappers that catch it explicitly. >>> >>> Rather than making these changes incrementally across individual PRs, I'd >>> like to discuss a coordinated migration strategy. >>> Proposal >>> >>> *1. Emit a deprecation warning from **AirflowException.__init__**.* >>> >>> Although emitting warnings when exceptions are raised is uncommon, it is >>> justified here because users may depend on AirflowException in callbacks >>> and exception handlers. This provides advance notice before >>> package-specific migrations occur. >>> >>> *2. Migrate per package in the next breaking release of that package.* >>> >>> Require a single migration PR per package (airflow-core and each provider), >>> rather than spreading exception changes across multiple PRs and releases. >>> The migration would be included in that package's next breaking release and >>> documented accordingly. >>> Questions >>> >>> - Should we add a deprecation warning to AirflowException? >>> - Is the per-package migration approach reasonable? >>> - Should we define a project-wide exception taxonomy, or leave exception >>> choices to individual provider teams? >>> >>> If there's agreement, I'll follow up with a lazy-consensus proposal. >>> >>> Thanks, >>> >>> >>> Shahar >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
