I think yes - we should strive to have the delete "kinds" of operations to silently ignore already deleted content. We can agree on it as a "default" behaviour - and probably have it written somewhere in best practices for writing operators.
But - we also should be pragmatic in the sense that a) there might be (justified) exceptions b) it's not very likely we will be able to consistently and automatically enforce it, and it almost guarantees that at any point in time some of those will be following different semantics - because we forget about it over time or won't realise this is happening. I wish we had some way of enforcing or at least detecting it :) On Tue, Jan 13, 2026 at 7:17 PM Daniel Standish via dev < [email protected]> wrote: > Yeah just suppress the exception. > > Task should not fail cus object already deleted. > > The intent is that the thing is not there, not to verify that it was and > that you deleted it. > > If users want diff behavior they can subclass. > > Simpler is better than complex, re the flag idea. Does not seem like a case > for a global param. > > > > On Tue, Jan 13, 2026 at 9:49 AM Ash Berlin-Taylor <[email protected]> wrote: > > > Given then general rule of idempontency of Airflow operators, I think we > > should break with the API. The intent of a “Delete” operator is to ensure > > the resource no longer exists. So not found is not an error in that > model. > > This makes such operators safe to retry. If it threw an error when it was > > deleted then they are not safe to retry, and this is not the model > Airflow > > operators should have. > > > > No flag should be needed — if someone doesn’t want this behavoiur than I > > would say they should use the hook or API directly. The point of the > > pre-built operators is to package up behaviour like this in a consistent > > manner. > > > > My 2c: we should standardised on remove the “ignore_if_missing” flags on > > operators and always catching not exists style errors in operators and > > treating those as success. > > > > -ash > > > > > On 12 Jan 2026, at 22:06, Ferruzzi, Dennis <[email protected]> > wrote: > > > > > > Yeah, generally when I add operators to the Amazon provider package, my > > intent is to preserve the behavior of the API,. I'm not claiming that's > > necessarily the right answer, but that's what i do and I like that idea. > > > > > > I wouldn't be against the idea of a flag, but I think as a general rule > > we should assume the default behavior should be familiar to someone who > is > > used to using the API before moving to Airflow without surprises. Of > > course, his may be an XKCD1172 issue [1] and maybe we SHOULD insist on a > > standard expectation across providers.... > > > > > > [1] https://xkcd.com/1172/ > > > > > > > > > - ferruzzi > > > > > > > > > ________________________________ > > > From: Shahar Epstein <[email protected]> > > > Sent: Monday, January 12, 2026 12:51 PM > > > To: [email protected] > > > Subject: [EXT] [DISCUSS] Idempotency of "Delete" operators > > > > > > CAUTION: This email originated from outside of the organization. Do not > > click links or open attachments unless you can confirm the sender and > know > > the content is safe. > > > > > > > > > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > > externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si > vous > > ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas > > certain que le contenu ne présente aucun risque. > > > > > > > > > > > > Hey all, > > > > > > For the sake of this discussion, the idempotency of Delete operators > > means > > > that when an operator is applied to a non-existing resource, it > catches a > > > “not found” exception and returns success. By default, the operators I > > > refer to return an exception if the resource is not found. > > > > > > Following this PR #60083 <https://github.com/apache/airflow/pull/60083 > >, > > > I've realized that we don't handle idempotency consistently in > different > > > "Delete" operators across different providers. For example, in some GCP > > > operators there's a tendency to catch "NotFound" exception without any > > flag > > > (DeletePipelineJobOperator > > > < > > > https://github.com/apache/airflow/blob/bd133c0ebb1219b72f4cb7998b2f5f55c8aff200/providers/google/src/airflow/providers/google/cloud/operators/vertex_ai/pipeline_job.py#L522 > > >), > > > while in some Microsoft's operators we handle it by flag (like > > > "ignore_if_missing" in WASBDeleteBlob > > > < > > > https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/_api/airflow/providers/microsoft/azure/operators/wasb_delete_blob/index.html#module-contents > > > > > > ). > > > > > > My questions for discussion are as follows: > > > 1. Do we want to require that idempotency for operators (that normally > > > return an error) will be handled by a flag? > > > 2. If we agree on having a flag, what should be the default value of > this > > > flag? > > > 3. Given the answer to the last question, should we break the interface > > of > > > existing operators where applicable? > > > > > > My answers: > > > 1. If the original API usually returns an error, catching NotFound > error > > > and returning success "defies" the intent of the original API, and I > > think > > > that it might be confusing and even limiting in some cases. Therefore, > if > > > originally a "Delete" operator fails and we want to catch this > exception > > - > > > I think that we should at least define a flag for that (preferably > with a > > > consistent name across all operators). > > > 2. Default behavior should reflect the original intent of the API. > > > 3. Slowly, but carefully - yes: changing both the default value, and > the > > > name of the flag to be consistent. > > > > > > I'll be happy to hear more opinions! > > > > > > > > > Shahar > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
