hi Jason, here is an example below - in task0_python_callable i did 1/0 to raise an error and Airflow then called task0_failure_cb. I could see FAIL in the task log so my task0_failure_cb was definitely invoked.
The task would still have failed status if look at UI. Are you expecting a different behavior? def task0_python_callable(ds, **kwargs): print ds print 1/0 return 'Success!!! Execution time = {}'.format(kwargs['execution_date']) def task0_failure_cb(ds, **kwargs): logging.info("FAIL") task0 = PythonOperator( task_id='task0', python_callable=task0_python_callable, on_failure_callback=task0_failure_cb, provide_context=True, dag=dag) [2016-10-18 16:07:02,631] {models.py:154} INFO - Filling up the DagBag from /home/oracle/airflow/dags/fail_callback.py [2016-10-18 16:07:03,239] {models.py:154} INFO - Filling up the DagBag from /home/oracle/airflow/dags/fail_callback.py [2016-10-18 16:07:03,312] {models.py:1196} INFO - -------------------------------------------------------------------------------- Starting attempt 1 of 1 -------------------------------------------------------------------------------- [2016-10-18 16:07:03,317] {models.py:1219} INFO - Executing <Task(PythonOperator): task0> on 2016-10-18 16:06:02 [2016-10-18 16:07:03,322] {models.py:1286} ERROR - integer division or modulo by zero Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 1245, in run result = task_copy.execute(context=context) File "/usr/local/lib/python2.7/site-packages/airflow/operators/python_operator.py", line 66, in execute return_value = self.python_callable(*self.op_args, **self.op_kwargs) File "/home/oracle/airflow/dags/fail_callback.py", line 33, in task0_python_callable print 1/0 ZeroDivisionError: integer division or modulo by zero [2016-10-18 16:07:03,324] {models.py:1306} INFO - Marking task as FAILED. [2016-10-18 16:07:03,324] {fail_callback.py:38} INFO - FAIL [2016-10-18 16:07:03,327] {models.py:1327} ERROR - integer division or modulo by zero On Tue, Oct 18, 2016 at 2:59 PM, Jason Chen <chingchien.c...@gmail.com> wrote: > Hi airflow team, > > Is there any sample code to use "on_failure_callback" ? > I tried to use that as a callback to "post-process" when a task fails. > However, I cannot make it work. > > The definition of my task is as below (just one task in my dag). > > It invokes "task0_python_callable" which executes a command-line script and > it could drops exception. It in turns making task0 failed. > In that case, I am thinking it should trigger the "on_failure_callback" > function "task0_failure_cb", but it does not happen. > > Any suggestion? > Thanks. > > Jason > > ========= > task0 = PythonOperator( > task_id='task0', > python_callable=task0_python_callable, > on_failure_callback=task0_failure_cb, > dag=dag) >