itay-raveh commented on issue #22040:
URL: https://github.com/apache/airflow/issues/22040#issuecomment-1500651337
> @eladkal That Makes sense. Since I don't have the entire context about the
motivation/use case for this. @itay-raveh can you please share more details?
>
> Also, an audit log should be sufficient for your use case, right?
For my use case, the need for the 'aborted' state is similar to the need for
the 'skipped' state. If someone triggered a dag, and decided based on the logs
that the rest of the tasks should not run, I would like to know that.
In that situation, there is a difference between "Something failed but did
not raise an error, so I stopped it manually" and "I realised that, although
everything is working, I actually don't need the dag to continue"
While 'skipped' indicates a programmatic decision to skip some task,
'aborted' indicates a human decision to skip the rest of the dag.
However, I concede that this may not be a widespread use case, and as such
might not warrant a core change. I have no knowledge on the scope of the
required changes or on the popularity of my suggestion.
Currently I use the success and failure callbacks for tasks and dags to send
updates to an external service. I would need a third callback, which I assume
requires a core change as discussed above.
The goal is to track the failure rate of dags and tasks, which is why
counting manual abortions as failures is problematic.
If I could, as you suggested, query the user (or `airflow`) which marked the
failure, I should be able to make do with that. It would be much less
convenient, but beggars can't be choosers :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
