It's been a while since I've looked at this code, but the exception thrown
there is thrown from a place where it should not be able to be caught by
your operator code, so the issue may be somewhere else.
On Wed, Oct 2, 2019 at 12:41 PM Shaw, Damian P. <
damian.sha...@credit-suisse.com> wrote:
> Today I had an issue where my operator caught the SIGTERM exception that
> Airflow throws. It seems to me that Airflow shouldn't be throwing a SIGTERM
> exception that subclasses Exception, particular as in arbitrary Python
> functions this might happen commonly.
>
> The code in my operator looked like this:
> try:
> optimistic_function()
> except Exception:
> self.log.exception('Optimistic function failed: ')
>
> When I terminated the task instance early in the Airflow GUI the exception
> was caught and the task carried on as normal:
> [2019-10-02 15:10:39,153] {my_operators.py:415} ERROR - Something went
> wrong with checksum validation:
> Traceback (most recent call last):
> ...
> File ".../python3.7/site-packages/airflow/models/__init__.py", line
> 1418, in signal_handler
> raise AirflowException("Task received SIGTERM signal")
> airflow.exceptions.AirflowException: Task received SIGTERM signal
> [2019-10-02 15:10:39,177] {my_operators.py:428} INFO - Carrying on with
> the operator like everything is good
>
> The above is from Airflow 1.10.3, on master branch the signal_handler code
> is located here:
> https://github.com/apache/airflow/blob/d719e1fd6705a93a0dfefef4b46478ade5e006ea/airflow/models/taskinstance.py#L898
>
> And the exception it throws is located here:
> https://github.com/apache/airflow/blob/d719e1fd6705a93a0dfefef4b46478ade5e006ea/airflow/exceptions.py#L25
>
> It seems to me that if Airflow is throwing an exception with the intent to
> kill the Python process it should be subclassing "SystemExit" not
> "Exception":
> https://docs.python.org/3/library/exceptions.html#exception-hierarchy .
> In the mean time I will rewrite my code for special handling when receiving
> AirflowException.
>
> Would everyone agree in general this is a bug or am I missing some use
> case?
>
> Damian
>
>
>
>
> ===============================================================================
>
> Please access the attached hyperlink for an important electronic
> communications disclaimer:
> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
> ===============================================================================
>
>
