Agreed, I created a JIRA a couple of minutes ago (it's a subtask in the JIRA I mentioned).
On Wed, Mar 1, 2017 at 10:58 AM, Bolke de Bruin <[email protected]> wrote: > Please create a Jira and provide context when this happens. “REMOVED” > marked means the taskinstance does not have a task equivalent anymore in > the dag (or so it should :)). > > Bolke > > > On 1 Mar 2017, at 19:55, Dan Davydov <[email protected]> > wrote: > > > > We are seeing another major issue with backfills where task instances are > > being deleted and marked as "removed", I am still investigating. Let's > keep > > discussion about these in https://issues.apache.org/ > jira/browse/AIRFLOW-921 > > and the subtask comments to have it one place. I will look at the other > > points you cc'd me on too. Thanks for continuing to drive this forward! > > > > On Wed, Mar 1, 2017 at 8:22 AM, Bolke de Bruin <[email protected]> > wrote: > > > >> Hi, > >> > >> Just wanted to give an update about the progress getting to RC5. As > >> reported we have 6 blockers listed. > >> > >> 1. Double run job should not terminate the existing running job. -> > Patch > >> Available > >> 2. Parallelize dag runs in backfills -> Patch Available, Tests need to > be > >> updated, see below > >> 3. Setting a task to running manually breaks a DAGs UI -> Patch merged > >> 4. Can't mark non-existent tasks as successful from graph view -> > >> Workaround available (t.b.c.), Patch Available unit tests to be added > >> 5. (Named)HivePartitionSensor broken if hook attr not set -> Patch > merged > >> 6. Skipped tasks potentially cause a dagrun to be marked as > >> failure/success prematurely -> see below > >> > >> On 2 I would like to have some more discussion of this would be > acceptable > >> (https://github.com/apache/incubator-airflow/pull/2107). I have written > >> the patch for this, however we are not large backfill users. So I need > >> feedback specifically on ripping out the “executor” part: @dan, @max. > >> > >> On 6 Alex has reported this earlier and written a PR for this ( > >> https://github.com/apache/incubator-airflow/pull/1961). Maxime had some > >> thoughts about this, which are currently blocking the integration. > However, > >> in testing it seems to solve the issue. Can we finalise the discussion > >> please @max @dan @alex? > >> > >> Cheers > >> Bolke > >
