A quick update on what we have now. 1. Lazy Consensus reached. 1. Instruction now in https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst 2. prek hook PR https://github.com/apache/airflow/pull/55416 <https://github.com/apache/airflow/pull/55416> to block new AirflowException usage now under review 2. In https://github.com/apache/airflow/pull/54505, Ephraim might get us closer to removing AirflowException
Jarek Potiuk <[email protected]> 於 2025年9月6日 週六 上午4:13寫道： > cool > > On Fri, Sep 5, 2025 at 8:30 PM Ferruzzi, Dennis <[email protected]> > wrote: > > > Yup, I like it. Thanks for driving this one. > > > > > > - ferruzzi > > > > > > ________________________________ > > From: Wei Lee <[email protected]> > > Sent: Thursday, September 4, 2025 3:43 AM > > To: [email protected] > > Subject: RE: [EXT] [Discussion] Should We avoid using AirflowException > > direclty? > > > > CAUTION: This email originated from outside of the organization. Do not > > click links or open attachments unless you can confirm the sender and > know > > the content is safe. > > > > > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. > > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne > pouvez > > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain > que > > le contenu ne présente aucun risque. > > > > > > > > I believe we've reached a certain level of consensus, but we still have > > some things to discuss. I'll break it into two parts. > > > > — > > > > ## What we have concluded on. > > 1. In most cases, we should prioritize using Python’s standard exceptions > > (e.g., `ValueError`, `TypeError`, `OSError`) instead of wrapping > everything > > in `AirflowException`. > > 2. Within `airflow-core`, we should define and utilize more specific > > exception classes under `airflow.exceptions`. > > 3. For provider-specific implementations, exceptions should be defined > > within `airflow.providers.<provider>.exceptions`. > > The use of points 2 and 3 should only be considered when point 1 is > > inappropriate, which should be a rare occurrence. > > > > What we can do other than nail down these rules > > Add it to > > > https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst > > Add a prek hook to detect `raise AirflowException` > > > > If no one objects, I'm planning on sending a [Lazy Consensus] sometime > > tomorrow. > > > > — > > > > ## What we haven't yet had a consensus > > Whether we really can/want to remove or depreciate AirflowException > > If we want, how are we going to achieve it > > > > Best, > > Wei > > > > > On Sep 2, 2025, at 3:36 PM, Amogh Desai <[email protected]> wrote: > > > > > > Generally agree with this too. > > > > > > `AirflowException` is literally being overused almost to an extent that > > it > > > has > > > become like a dumping ground (along the lines of what `utils` has > become) > > > > > > I would be in favour of removing it too and addressing backcompat > > concerns > > > for user facing / custom provider / hooks etc facing classes. > > > > > > This could be run as a community exercise too once we reach that stage > to > > > clean > > > up providers from using / overusing it. > > > > > > Thanks & Regards, > > > Amogh Desai > > > > > > > > > On Tue, Sep 2, 2025 at 1:58 AM Jens Scheffler > <[email protected] > > > > > > wrote: > > > > > >> Also was a bit focussing on other stuff, late to the party. > > >> > > >> For deprecation we should consider RUFF rules as well. > > >> > > >> But on side of code changes I fear a lot of people wil have > > >> implementations based on AirflowException. I did not get the idea of > > >> backwards compatability. If we change Exception types then Operators > > >> implemented on top of Standard-Provider might fail if the receive > > >> different exceptions from inherited classes? > > >> > > >> Jens > > >> > > >> On 01.09.25 17:05, Buğra Öztürk wrote: > > >>> Great discussion! Joining delayed. We have this usage in airflowctl > too > > >>> where AirflowException is the main one even though it isn't imported > > from > > >>> core. > > >>> I agree with the consensus. Great idea to make it backward > compatible. > > >>> Maybe would be good to define a migration calendar roughly. > > >>> While we should prevent usage with some pre-commit hooks, we should > > also > > >>> document the practices we should follow on exception handling for > these > > >>> changes. > > >>> Even though we can manage to migrate seamlessly for the user, I think > > we > > >>> still should warn them about the exception handling usage and give > some > > >>> kind of deprecation warning if possible > > >>> > > >>> Bugra Ozturk > > >>> > > >>> On Mon, 1 Sept 2025, 16:04 Wei Lee, <[email protected]> wrote: > > >>> > > >>>> Thanks everyone for joining the discussion! > > >>>> > > >>>> I think we have some consensus. These should at least apply to new > > code. > > >>>> > > >>>> 1. Don’t use AirflowException > > >>>> 2. Use Python native whenever possible > > >>>> 3. Create customized exceptions instead of AirflowException > > >>>> > > >>>> But we’re worried about how and whether we should remove > > >> AirflowException. > > >>>> What I am thinking is removing `raise AirflowException`. > > >>>> As for AirflowException itself, it looks somewhat challenging. > > >>>> What if we do something like `CustomException(AirflowException)`, > > >>>> so `except AirflowException` blocks still work, but we can gain the > > >>>> benefit of making it descriptive. > > >>>> > > >>>> > > >>>> Best, > > >>>> Wei > > >>>> > > >>>>> On Aug 30, 2025, at 11:08 PM, Jarek Potiuk <[email protected]> > wrote: > > >>>>> > > >>>>>> I would say we should only worry about backcompat when there's > > >> something > > >>>>> to > > >>>>> worry about. > > >>>>> > > >>>>> I think Daniel is quite right. In this case what **would** happen > if > > >>>>> someone relies on `try AirflowException`, the issue would be > > immediate > > >> to > > >>>>> see and easy to fix (and we can describe it in release notes as > > >>>> significant > > >>>>> news. I can't easily think of a "reasonable" case where behaviour > > would > > >>>> be > > >>>>> changed without crashing. I think that would be a blocker if we can > > >> have > > >>>> a > > >>>>> reasonable way of using it that might change the behaviour in a way > > >> that > > >>>> is > > >>>>> difficult to notice by the user, but when I think of potential > cases > > I > > >>>>> cannot imagine usage that could lead to it. > > >>>>> > > >>>>> J > > >>>>> > > >>>>> > > >>>>> On Wed, Aug 27, 2025 at 3:31 PM Daniel Standish > > >>>>> <[email protected]> wrote: > > >>>>> > > >>>>>> Generally agree with this. > > >>>>>> > > >>>>>> I would say we should only worry about backcompat when there's > > >>>> something to > > >>>>>> worry about. > > >>>>>> > > >>>>>> Like, if there's just a remote possibility that a user is doing > > >>>> something > > >>>>>> weird (e.g. in some subclass they wrote of something) then, not > > really > > >>>> for > > >>>>>> us to worry about. Seems rather internal behavior generally. > > >>>>>> > > >>>>>> If we *know* for sure changing it will break airflow for certain > > >>>>>> combinations that should work, that's something to address. > > >>>>>> > > >>>> > > >>>> > --------------------------------------------------------------------- > > >>>> To unsubscribe, e-mail: [email protected] > > >>>> For additional commands, e-mail: [email protected] > > >>>> > > >>>> > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: [email protected] > > >> For additional commands, e-mail: [email protected] > > >> > > >> > > > > >
