Please create a Jira and provide context when this happens. “REMOVED” marked means the taskinstance does not have a task equivalent anymore in the dag (or so it should :)).
Bolke > On 1 Mar 2017, at 19:55, Dan Davydov <[email protected]> wrote: > > We are seeing another major issue with backfills where task instances are > being deleted and marked as "removed", I am still investigating. Let's keep > discussion about these in https://issues.apache.org/jira/browse/AIRFLOW-921 > and the subtask comments to have it one place. I will look at the other > points you cc'd me on too. Thanks for continuing to drive this forward! > > On Wed, Mar 1, 2017 at 8:22 AM, Bolke de Bruin <[email protected]> wrote: > >> Hi, >> >> Just wanted to give an update about the progress getting to RC5. As >> reported we have 6 blockers listed. >> >> 1. Double run job should not terminate the existing running job. -> Patch >> Available >> 2. Parallelize dag runs in backfills -> Patch Available, Tests need to be >> updated, see below >> 3. Setting a task to running manually breaks a DAGs UI -> Patch merged >> 4. Can't mark non-existent tasks as successful from graph view -> >> Workaround available (t.b.c.), Patch Available unit tests to be added >> 5. (Named)HivePartitionSensor broken if hook attr not set -> Patch merged >> 6. Skipped tasks potentially cause a dagrun to be marked as >> failure/success prematurely -> see below >> >> On 2 I would like to have some more discussion of this would be acceptable >> (https://github.com/apache/incubator-airflow/pull/2107). I have written >> the patch for this, however we are not large backfill users. So I need >> feedback specifically on ripping out the “executor” part: @dan, @max. >> >> On 6 Alex has reported this earlier and written a PR for this ( >> https://github.com/apache/incubator-airflow/pull/1961). Maxime had some >> thoughts about this, which are currently blocking the integration. However, >> in testing it seems to solve the issue. Can we finalise the discussion >> please @max @dan @alex? >> >> Cheers >> Bolke
