[
https://issues.apache.org/jira/browse/AIRFLOW-2285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446118#comment-16446118
]
Gregory Dougherty edited comment on AIRFLOW-2285 at 4/20/18 6:16 PM:
---------------------------------------------------------------------
Further comments:
I set the DAG to failed while it was doing file processing: It worked, the DAG
stopped.
I then launched a new DAG and set it to "Failed" while it was doing the large
file upload.
It failed, with a mushroom cloud:
(Note: the "stop" failed". The DAG is still running, just fine.)
____/ ( ( ) ) ___ /( ( ( ) _ )) ) )\ (( ( )( ) ) ( ) ) ((/ ( _( ) ( _) ) ( () )
) ( ( ( (_) (( ( ) .((_ ) . )_ ( ( ) ( ( ) ) ) . ) ( ) ( ( ( ( ) ( _ ( _) ). )
. ) ) ( ) ( ( ( ) ( ) ( )) ) _)( ) ) ) ( ( ( \ ) ( (_ ( ) ( ) ) ) ) )) ( ) ( (
( ( (_ ( ) ( _ ) ) ( ) ) ) ( ( ( ( ( ) (_ ) ) ) _) ) _( ( ) (( ( )( ( _ ) _)
_(_ ( (_ ) (_((__(_(__(( ( ( | ) ) ) )_))__))_)___) ((__)_
_||lll|l||/// _)) ( /(/ ( ) ) )\ ) ( ( ( ( | | ) ) )\ ) ( /(| / ( )) ) ) )) )
( ( ((((_(|)_))))) ) ( ||(|(|)|/|| ) ( |(||(||)|||| ) ( //|/l|||)|
\ ) (/ / // /|//||||
\ \ \ _)
-------------------------------------------------------------------------------
Node: foo
-------------------------------------------------------------------------------
Traceback (most recent call last):
File "/apps/dist/python/lib/python2.7/site-packages/flask/app.py", line 1988,
in wsgi_app response = self.full_dispatch_request()
File "/apps/dist/python/lib/python2.7/site-packages/flask/app.py", line 1641,
in full_dispatch_request rv = self.handle_user_exception(e)
File "/apps/dist/python/lib/python2.7/site-packages/flask/app.py", line 1544,
in handle_user_exception reraise(exc_type, exc_value, tb)
File "/apps/dist/python/lib/python2.7/site-packages/flask/app.py", line 1639,
in full_dispatch_request rv = self.dispatch_request()
File "/apps/dist/python/lib/python2.7/site-packages/flask/app.py", line 1625,
in dispatch_request return self.view_functions[rule.endpoint](**req.view_args)
File "/apps/dist/python/lib/python2.7/site-packages/flask_admin/base.py", line
69, in inner return self._run_view(f, *args, **kwargs)
File "/apps/dist/python/lib/python2.7/site-packages/flask_admin/base.py", line
368, in _run_view return fn(self, *args, **kwargs)
File "/apps/dist/python/lib/python2.7/site-packages/flask_admin/model/base.py",
line 2068, in action_view return self.handle_action()
File "/apps/dist/python/lib/python2.7/site-packages/flask_admin/actions.py",
line 113, in handle_action response = handler[0](ids)
File "/apps/dist/python/lib/python2.7/site-packages/airflow/www/views.py", line
2498, in action_set_failed self.set_task_instance_state(ids, State.FAILED)
File "/apps/dist/python/lib/python2.7/site-packages/airflow/utils/db.py", line
50, in wrapper result = func(*args, **kwargs)
File "/apps/dist/python/lib/python2.7/site-packages/airflow/www/views.py", line
2540, in set_task_instance_state raise Exception("Ooops")
Exception: Ooops
was (Author: gregtd):
Further comments:
I set the DAG to failed while it was doing file processing: It worked, the DAG
stopped.
I then launched a new DAG and set it to "Failed" while it was doing the large
file upload.
It failed, with a mushroom cloud
:
____/ ( ( ) ) ___ /( ( ( ) _ )) ) )\ (( ( )( ) ) ( ) ) ((/ ( _( ) ( _) ) ( () )
) ( ( ( (_) (( ( ) .((_ ) . )_ ( ( ) ( ( ) ) ) . ) ( ) ( ( ( ( ) ( _ ( _) ). )
. ) ) ( ) ( ( ( ) ( ) ( )) ) _)( ) ) ) ( ( ( \ ) ( (_ ( ) ( ) ) ) ) )) ( ) ( (
( ( (_ ( ) ( _ ) ) ( ) ) ) ( ( ( ( ( ) (_ ) ) ) _) ) _( ( ) (( ( )( ( _ ) _)
_(_ ( (_ ) (_((__(_(__(( ( ( | ) ) ) )_))__))_)___) ((__)_
_||lll|l||/// _)) ( /(/ ( ) ) )\ ) ( ( ( ( | | ) ) )\ ) ( /(| / ( )) ) ) )) ) (
( ((((_(|)_))))) ) ( ||(|(|)|/|| ) ( |(||(||)|||| ) ( //|/l|||)|
\ ) (/ / // /|//||||
\ \ \ _)
-------------------------------------------------------------------------------
Node: foo
-------------------------------------------------------------------------------
Traceback (most recent call last):
File "/apps/dist/python/lib/python2.7/site-packages/flask/app.py", line 1988,
in wsgi_app response = self.full_dispatch_request()
File "/apps/dist/python/lib/python2.7/site-packages/flask/app.py", line 1641,
in full_dispatch_request rv = self.handle_user_exception(e)
File "/apps/dist/python/lib/python2.7/site-packages/flask/app.py", line 1544,
in handle_user_exception reraise(exc_type, exc_value, tb)
File "/apps/dist/python/lib/python2.7/site-packages/flask/app.py", line 1639,
in full_dispatch_request rv = self.dispatch_request()
File "/apps/dist/python/lib/python2.7/site-packages/flask/app.py", line 1625,
in dispatch_request return self.view_functions[rule.endpoint](**req.view_args)
File "/apps/dist/python/lib/python2.7/site-packages/flask_admin/base.py", line
69, in inner return self._run_view(f, *args, **kwargs)
File "/apps/dist/python/lib/python2.7/site-packages/flask_admin/base.py", line
368, in _run_view return fn(self, *args, **kwargs)
File "/apps/dist/python/lib/python2.7/site-packages/flask_admin/model/base.py",
line 2068, in action_view return self.handle_action()
File "/apps/dist/python/lib/python2.7/site-packages/flask_admin/actions.py",
line 113, in handle_action response = handler[0](ids)
File "/apps/dist/python/lib/python2.7/site-packages/airflow/www/views.py", line
2498, in action_set_failed self.set_task_instance_state(ids, State.FAILED)
File "/apps/dist/python/lib/python2.7/site-packages/airflow/utils/db.py", line
50, in wrapper result = func(*args, **kwargs)
File "/apps/dist/python/lib/python2.7/site-packages/airflow/www/views.py", line
2540, in set_task_instance_state raise Exception("Ooops")
Exception: Ooops
> Need to be able to stop a running DAG
> -------------------------------------
>
> Key: AIRFLOW-2285
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2285
> Project: Apache Airflow
> Issue Type: New Feature
> Components: DAG
> Affects Versions: 1.9.0
> Reporter: Gregory Dougherty
> Priority: Major
> Labels: features
>
> There is no way to stop a running DAG. As DAGs can have serious bugs (for
> example, uploading data to the wrong location, when being charged for every
> MB uploaded), need a way to force a DAG to stop running
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
