lucafuji commented on a change in pull request #6870: [AIRFLOW-0578] Check
return code
URL: https://github.com/apache/airflow/pull/6870#discussion_r372673255
##########
File path: airflow/jobs/local_task_job.py
##########
@@ -95,6 +95,15 @@ def signal_handler(signum, frame):
# Monitor the task to see if it's done
return_code = self.task_runner.return_code()
if return_code is not None:
+ print(self.task_instance.state)
+ # there is one case we should not treat non zero return
+ # code as failed: the job has been killed externally.
+ if return_code != 0 and ((not self.terminating) or
+ self.task_instance.state ==
Review comment:
> TBH JobState isn't used anyway (so long as it's not still running).
>
> It's open to interpretation when a job should be marked as failed. If the
task ran to completion then the job did what it was told to. Is that a failure?
What about when the task is in "up_for_retry" -- what state should the Job be
in?
>
> My first though is that the job should be marked as success if the task is
in any state other than "running" at the end, and only failure if the Task
didn't update it's state.
Hmm, I see your point but I think we should better to be consistent about
interpreting the meaning of Job.state across or job types?
e.g, for BackfillJob, if one of the underlying tasks fail, it will be marked
as failure as well
on backfill_job.py:L792
"""
err = self._collect_errors(ti_status=ti_status, session=session)
if err:
raise AirflowException(err)
"""
So I think it might be better to have job state reflects the state of
underlying task instance
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services