Re-attaching the image.. On Fri, Jun 29, 2018 at 4:54 PM Feng Lu <[email protected]> wrote:
> Hi all, > > Please take a look at our proposal to deprecate Run task in Airflow webUI. > > *What?* > Deprecate Run task support in the Airflow webUI and make it a no-op for > now. > > [image: xGAcOrcLJE4.png] > > *Why?* > > 1. It only works with CeleryExecutor > > <https://github.com/apache/incubator-airflow/blob/master/airflow/www/views.py#L1001-L1003> > and renders an inconsistent experience for other types of Executors. > 2. It requires Airflow webserver to have direct connection with the > message backend of CeleryExecutor, and opens more vulnerability in the > system. In many cases, users may want to restrict access to the celery > messaging backend as much as possible. > > *Mitigation:* > This Run task feature is mainly for the purpose of re-executing of a > previously running task which got stuck in running and deleted manually. > It's currently a two step process: > 1. Navigate to the task instance view page and delete the running task > that's stuck > 2. Go back to DAG/task view and click "Run" > > We proposed to combine the two steps, after a running task is deleted, the > Airflow scheduler will automatically re-schedule (which it does today) and > re-queue the task (there's a bug that needs to be fixed). > > *Fix:* > The scheduler currently doesn't not automatically re-queue the task > despite the task instance has changed from running to scheduled state. The > heartbeat check incorrectly returns a success in this case. The root cause > is that LocalTaskJob doesn't set the job state to failed (details > <https://github.com/apache/incubator-airflow/blob/master/airflow/jobs.py#L2674-L2681>) > when a running task is externally deleted and confuses the heartbeat check > <https://github.com/apache/incubator-airflow/blob/master/airflow/models.py#L443> > . > Once this is fixed, a killed running task instance will be > auto-scheduled/enqueued for execution, verified locally. > > Thank you. > > Feng > > >
