Yes, I agree that delete operations should ignore already deleted targets, because of the idempotency approach being a general rule.
I am not a fan of a new flag for this. On Tue, Jan 13, 2026 at 11:06 AM Jarek Potiuk <[email protected]> wrote: > I think yes - we should strive to have the delete "kinds" of operations to > silently ignore already deleted content. We can agree on it as a "default" > behaviour - and probably have it written somewhere in best practices for > writing operators. > > But - we also should be pragmatic in the sense that a) there might be > (justified) exceptions b) it's not very likely we will be able to > consistently and automatically enforce it, and it almost guarantees that at > any point in time some of those will be following different semantics - > because we forget about it over time or won't realise this is happening. > > I wish we had some way of enforcing or at least detecting it :) > > On Tue, Jan 13, 2026 at 7:17 PM Daniel Standish via dev < > [email protected]> wrote: > > > Yeah just suppress the exception. > > > > Task should not fail cus object already deleted. > > > > The intent is that the thing is not there, not to verify that it was and > > that you deleted it. > > > > If users want diff behavior they can subclass. > > > > Simpler is better than complex, re the flag idea. Does not seem like a > case > > for a global param. > > > > > > > > On Tue, Jan 13, 2026 at 9:49 AM Ash Berlin-Taylor <[email protected]> > wrote: > > > > > Given then general rule of idempontency of Airflow operators, I think > we > > > should break with the API. The intent of a “Delete” operator is to > ensure > > > the resource no longer exists. So not found is not an error in that > > model. > > > This makes such operators safe to retry. If it threw an error when it > was > > > deleted then they are not safe to retry, and this is not the model > > Airflow > > > operators should have. > > > > > > No flag should be needed — if someone doesn’t want this behavoiur than > I > > > would say they should use the hook or API directly. The point of the > > > pre-built operators is to package up behaviour like this in a > consistent > > > manner. > > > > > > My 2c: we should standardised on remove the “ignore_if_missing” flags > on > > > operators and always catching not exists style errors in operators and > > > treating those as success. > > > > > > -ash > > > > > > > On 12 Jan 2026, at 22:06, Ferruzzi, Dennis <[email protected]> > > wrote: > > > > > > > > Yeah, generally when I add operators to the Amazon provider package, > my > > > intent is to preserve the behavior of the API,. I'm not claiming > that's > > > necessarily the right answer, but that's what i do and I like that > idea. > > > > > > > > I wouldn't be against the idea of a flag, but I think as a general > rule > > > we should assume the default behavior should be familiar to someone who > > is > > > used to using the API before moving to Airflow without surprises. Of > > > course, his may be an XKCD1172 issue [1] and maybe we SHOULD insist on > a > > > standard expectation across providers.... > > > > > > > > [1] https://xkcd.com/1172/ > > > > > > > > > > > > - ferruzzi > > > > > > > > > > > > ________________________________ > > > > From: Shahar Epstein <[email protected]> > > > > Sent: Monday, January 12, 2026 12:51 PM > > > > To: [email protected] > > > > Subject: [EXT] [DISCUSS] Idempotency of "Delete" operators > > > > > > > > CAUTION: This email originated from outside of the organization. Do > not > > > click links or open attachments unless you can confirm the sender and > > know > > > the content is safe. > > > > > > > > > > > > > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > > > externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si > > vous > > > ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes > pas > > > certain que le contenu ne présente aucun risque. > > > > > > > > > > > > > > > > Hey all, > > > > > > > > For the sake of this discussion, the idempotency of Delete operators > > > means > > > > that when an operator is applied to a non-existing resource, it > > catches a > > > > “not found” exception and returns success. By default, the operators > I > > > > refer to return an exception if the resource is not found. > > > > > > > > Following this PR #60083 < > https://github.com/apache/airflow/pull/60083 > > >, > > > > I've realized that we don't handle idempotency consistently in > > different > > > > "Delete" operators across different providers. For example, in some > GCP > > > > operators there's a tendency to catch "NotFound" exception without > any > > > flag > > > > (DeletePipelineJobOperator > > > > < > > > > > > https://github.com/apache/airflow/blob/bd133c0ebb1219b72f4cb7998b2f5f55c8aff200/providers/google/src/airflow/providers/google/cloud/operators/vertex_ai/pipeline_job.py#L522 > > > >), > > > > while in some Microsoft's operators we handle it by flag (like > > > > "ignore_if_missing" in WASBDeleteBlob > > > > < > > > > > > https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/_api/airflow/providers/microsoft/azure/operators/wasb_delete_blob/index.html#module-contents > > > > > > > > ). > > > > > > > > My questions for discussion are as follows: > > > > 1. Do we want to require that idempotency for operators (that > normally > > > > return an error) will be handled by a flag? > > > > 2. If we agree on having a flag, what should be the default value of > > this > > > > flag? > > > > 3. Given the answer to the last question, should we break the > interface > > > of > > > > existing operators where applicable? > > > > > > > > My answers: > > > > 1. If the original API usually returns an error, catching NotFound > > error > > > > and returning success "defies" the intent of the original API, and I > > > think > > > > that it might be confusing and even limiting in some cases. > Therefore, > > if > > > > originally a "Delete" operator fails and we want to catch this > > exception > > > - > > > > I think that we should at least define a flag for that (preferably > > with a > > > > consistent name across all operators). > > > > 2. Default behavior should reflect the original intent of the API. > > > > 3. Slowly, but carefully - yes: changing both the default value, and > > the > > > > name of the flag to be consistent. > > > > > > > > I'll be happy to hear more opinions! > > > > > > > > > > > > Shahar > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > >
