Hi,
I do not have a stron opinion on this, with the past decision we mainly
wanted to ensure code is clean.
For me also Option (3) would be viable: We leave existing code as it is
and just ensure that new code is clean. But I am also okay to join the
other majority opinion.
Jens
On 31.05.26 14:08, Sameer Mesiah wrote:
Hi,
Thanks for opening this thread. This was long overdue considering that
current migration efforts for AirflowException are more scattered rather
than a consolidated pre-planned wave.
In general, I agree with your overall approach. But there are a few things
I want to raise in response to your proposal:
On 1)
I think Amogh already expanded on thiss somewhat. But I believe it is
important to understand that AirflowException is not constrained to
the provider layer but often bubble up from Airflow core.For example
task_runner.py explicitly catches AirflowException as part of task
execution and retry handling, and there are also usages in the hook and
connection layers.
But depending on whether we scope this to core + providers vs providers
only, the migration strategy looks quite different. If the intention is
provider-only migration, users will continue to encounter AirflowException
from core Airflow components, which may make the overall exception model
feel somewhat inconsistent. If the intention is project-wide deprecation, I
think we need a broader plan than provider-level migrations alone.
Also, as Amogh already implied (he is free to correct me in case I
misinterpreted his reservations), deprecation warnings may not be
worthwhile here. Unlike a deprecated API or method, this is not something
users are actively choosing to call. In most cases, Airflow itself is
deciding which exception type to raise. I am leaning against warnings due
to the potential for log pollution and the relatively limited value they
provide.
On 2)
I agree with your per-package migration proposal. But as Amogh said, we
should ensure new exceptions inherit from AirflowException to prevent
breaking changes across the provider ecosystem. I must add that there will
inevitably be a period where users run multiple providers, some migrated
and some not. This may result in a mixture of exception types appearing in
logs and stack traces. While not ideal, this is probably unavoidable given
independent provider versioning. As long as migrations are clearly
documented, I do not see this as a major issue.
My answers to your questions:
*Should we add a deprecation warning to AirflowException?*
I am +0. Not strongly opposed, but I am leaning no due to the risk of log
pollution and the fact that the warning would be emitted by Airflow itself
rather than as a result of an explicit user action.
*Is the per-package migration approach reasonable?*
Yes. No blocking concerns from my side.
*Should we define a project-wide exception taxonomy, or leave exception
choices to individual provider teams?*
I would leave this to individual provider teams. The providers have very
different APIs, error models, and opinions around exception handling.
Attempting to enforce a central taxonomy across all providers may be
impractical and may not provide much practical value over what we currently
have in place.
Thanks,
Sameer Mesiah.
On Sat, 30 May 2026 at 10:59, Shahar Epstein <[email protected]> wrote:
Hi all,
This follows up on the earlier lazy-consensus
<https://lists.apache.org/thread/m86gs4mqt8hq1n26g0pp0fq5h5g0x2q9> and
related discussions around deprecating AirflowException.
While working on a cleanup PR for the Google Cloud Run operator in the
Google provider (#67769), Amogh pointed out that replacing AirflowException
with built-in or custom exceptions is a breaking change. Users may rely on
AirflowException in callbacks (on_failure_callback, on_retry_callback) or
in custom wrappers that catch it explicitly.
Rather than making these changes incrementally across individual PRs, I'd
like to discuss a coordinated migration strategy.
Proposal
*1. Emit a deprecation warning from **AirflowException.__init__**.*
Although emitting warnings when exceptions are raised is uncommon, it is
justified here because users may depend on AirflowException in callbacks
and exception handlers. This provides advance notice before
package-specific migrations occur.
*2. Migrate per package in the next breaking release of that package.*
Require a single migration PR per package (airflow-core and each provider),
rather than spreading exception changes across multiple PRs and releases.
The migration would be included in that package's next breaking release and
documented accordingly.
Questions
- Should we add a deprecation warning to AirflowException?
- Is the per-package migration approach reasonable?
- Should we define a project-wide exception taxonomy, or leave exception
choices to individual provider teams?
If there's agreement, I'll follow up with a lazy-consensus proposal.
Thanks,
Shahar
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
