Upon further investigation this was caused by a change in the semantics of ALL_SUCCESS, which I have these feelings about: Intuitively you would expect to skip any task that has dependencies that weren't run by default, i.e. the trigger rule is called ALL_SUCCESS and skipped tasks are not successful ones, and that was also the old behavior in 1.7.3.
This is going to break some use cases which could be alright, but I feel these new semantics make less sense than before so it's a bad reason to break existing use cases. I will get started on a PR for a new ALL_SUCCESS_NOT_SKIPPED trigger rule but again I feel this is hacky and really we should have the old ALL_SUCCESS (default) and a new ALL_SUCCESS_OR_SKIPPED trigger rule if desired. On Wed, Mar 15, 2017 at 6:25 PM, Dan Davydov <dan.davy...@airbnb.com> wrote: > Another issue we are seeing is https://issues.apache.org/ > jira/browse/AIRFLOW-992 - tasks that have both skipped children and > successful children are run instead of skipped. Not blocking the release on > this just letting you guys know for the release bug notes. We will be > cherrypicking a fix for this onto our production when we release 1.8 once > we come up with one. > > It's possibly thought not necessarily related to an incomplete/incorrect > fix of https://issues.apache.org/jira/browse/AIRFLOW-719 . > > On Wed, Mar 15, 2017 at 4:53 PM, siddharth anand <san...@apache.org> > wrote: > >> Confirmed that Bolke's PR above fixes the issue. >> >> Also, I agree this is not a blocker for the current airflow release, so my >> +1 (binding) stands. >> -s >> >> On Wed, Mar 15, 2017 at 3:11 PM, Bolke de Bruin <bdbr...@gmail.com> >> wrote: >> >> > PR is available: https://github.com/apache/incubator-airflow/pull/2154 >> > >> > But marked for 1.8.1. >> > >> > - Bolke >> > >> > > On 15 Mar 2017, at 14:37, Bolke de Bruin <bdbr...@gmail.com> wrote: >> > > >> > > On second thought I do consider it a bug and can have a fix out pretty >> > quickly, but I don’t consider it a blocker. >> > > >> > > - B. >> > > >> > >> On 15 Mar 2017, at 14:21, Bolke de Bruin <bdbr...@gmail.com> wrote: >> > >> >> > >> Just to be clear: Also in 1.7.1 the DagRun was marked successful, but >> > its tasks continued to be scheduled. So one could also consider 1.7.1 >> > behaviour a bug. I am not sure here, but I think it kind of makes sense >> to >> > consider the behaviour of 1.7.1 a bug. It has been present throughout >> all >> > the 1.8 rc/beta/apha series. >> > >> >> > >> So yes it is a change in behaviour whether it is a regression or an >> > integrity improvement is up for discussion. Either way I don’t consider >> it >> > a blocker. >> > >> >> > >> Bolke. >> > >> >> > >>> On 15 Mar 2017, at 14:06, siddharth anand <san...@apache.org> >> wrote: >> > >>> >> > >>> Here's the JIRA : >> > >>> https://issues.apache.org/jira/browse/AIRFLOW-989 >> > >>> >> > >>> I confirmed it is a regression from 1.7.1.3, which I installed via >> pip >> > and >> > >>> tested against the same DAG in the JIRA. >> > >>> >> > >>> The issue occurs if a leaf / last / terminal downstream task is not >> > >>> cleared. You won't see this issue if you clear the entire DAG Run or >> > clear >> > >>> a task and all of its downstream tasks. If you truly want to only >> > clear and >> > >>> rerun a task, but not its downstream tasks, you can use the CLI to >> > execute >> > >>> a specific task (e.g. vial airflow run). >> > >>> >> > >>> This is a change in behavior -- if we do go ahead with the release, >> > then >> > >>> this JIRA should be in a list of JIRAs of known issues related to >> the >> > new >> > >>> version. >> > >>> -s >> > >>> >> > >>> On Wed, Mar 15, 2017 at 9:17 AM, Chris Riccomini < >> > criccom...@apache.org> >> > >>> wrote: >> > >>> >> > >>>> @Sid, does this happen if you clear downstream as well? >> > >>>> >> > >>>> On Wed, Mar 15, 2017 at 9:04 AM, Chris Riccomini < >> > criccom...@apache.org> >> > >>>> wrote: >> > >>>> >> > >>>>> Has anyone been able to reproduce Sid's issue? >> > >>>>> >> > >>>>> On Tue, Mar 14, 2017 at 11:17 PM, Bolke de Bruin < >> bdbr...@gmail.com> >> > >>>>> wrote: >> > >>>>> >> > >>>>>> That is not an airflow error, but a Kerberos error. Try executing >> > the >> > >>>>>> kinit command on the command line by yourself. >> > >>>>>> >> > >>>>>> Bolke >> > >>>>>> >> > >>>>>> Sent from my iPhone >> > >>>>>> >> > >>>>>>> On 14 Mar 2017, at 23:11, Ruslan Dautkhanov < >> dautkha...@gmail.com> >> > >>>>>> wrote: >> > >>>>>>> >> > >>>>>>> `airflow kerberos` is broken in 1.8-rc5 >> > >>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-987 >> > >>>>>>> Hopefully fix can be part of the 1.8 release. >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> -- >> > >>>>>>> Ruslan Dautkhanov >> > >>>>>>> >> > >>>>>>>> On Tue, Mar 14, 2017 at 6:19 PM, siddharth anand < >> > san...@apache.org> >> > >>>>>> wrote: >> > >>>>>>>> >> > >>>>>>>> FYI, >> > >>>>>>>> I've just hit a major bug in the release candidate related to >> > "clear >> > >>>>>> task" >> > >>>>>>>> behavior. >> > >>>>>>>> >> > >>>>>>>> I've been running airflow in both stage and prod since >> yesterday >> > on >> > >>>>>> rc5 and >> > >>>>>>>> have reproduced this in both environments. I will file a JIRA >> for >> > >>>> this >> > >>>>>>>> tonight, but wanted to send a note over email as well. >> > >>>>>>>> >> > >>>>>>>> In my example, I have a 2 task DAG. For a given DAG run that >> has >> > >>>>>> completed >> > >>>>>>>> successfully, if I >> > >>>>>>>> 1) clear task2 (leaf task in this case), the >> previously-successful >> > >>>> DAG >> > >>>>>> Run >> > >>>>>>>> goes back to Running, requeues, and executes the task >> > successfully. >> > >>>>>> The DAG >> > >>>>>>>> Run the returns from Running to Success. >> > >>>>>>>> 2) clear task1 (root task in this case), the >> previously-successful >> > >>>> DAG >> > >>>>>> Run >> > >>>>>>>> goes back to Running, DOES NOT requeue or execute the task at >> all. >> > >>>> The >> > >>>>>> DAG >> > >>>>>>>> Run the returns from Running to Success though it never ran the >> > task. >> > >>>>>>>> >> > >>>>>>>> 1) is expected and previous behavior. 2) is a regression. >> > >>>>>>>> >> > >>>>>>>> The only workaround is to use the CLI to run the task cleared. >> > Here >> > >>>> are >> > >>>>>>>> some images : >> > >>>>>>>> *After Clearing the Tasks* >> > >>>>>>>> https://www.dropbox.com/s/wmuxt0krwx6wurr/Screenshot% >> > >>>>>>>> 202017-03-14%2014.09.34.png?dl=0 >> > >>>>>>>> >> > >>>>>>>> *After DAG Runs return to Success* >> > >>>>>>>> https://www.dropbox.com/s/qop933rzgdzchpd/Screenshot% >> > >>>>>>>> 202017-03-14%2014.09.49.png?dl=0 >> > >>>>>>>> >> > >>>>>>>> This is a major regression because it will force everyone to >> use >> > the >> > >>>>>> CLI >> > >>>>>>>> for things that they would normally use the UI for. >> > >>>>>>>> >> > >>>>>>>> -s >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> -s >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>>> On Tue, Mar 14, 2017 at 1:32 PM, Daniel Huang < >> dxhu...@gmail.com >> > > >> > >>>>>> wrote: >> > >>>>>>>>> >> > >>>>>>>>> +1 (non-binding)! >> > >>>>>>>>> >> > >>>>>>>>> On Tue, Mar 14, 2017 at 11:35 AM, siddharth anand < >> > >>>> san...@apache.org> >> > >>>>>>>>> wrote: >> > >>>>>>>>> >> > >>>>>>>>>> +1 (binding) >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> On Tue, Mar 14, 2017 at 8:42 AM, Maxime Beauchemin < >> > >>>>>>>>>> maximebeauche...@gmail.com> wrote: >> > >>>>>>>>>> >> > >>>>>>>>>>> +1 (binding) >> > >>>>>>>>>>> >> > >>>>>>>>>>> On Tue, Mar 14, 2017 at 3:59 AM, Alex Van Boxel < >> > a...@vanboxel.be >> > >>>>> >> > >>>>>>>>>> wrote: >> > >>>>>>>>>>> >> > >>>>>>>>>>>> +1 (binding) >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> Note: we had to revert all our ONE_SUCCESS with ALL_SUCCESS >> > >>>> trigger >> > >>>>>>>>>> rules >> > >>>>>>>>>>>> where the parent nodes where joining with a SKIP. But I >> can of >> > >>>>>>>> should >> > >>>>>>>>>>> have >> > >>>>>>>>>>>> known this was coming. Apart of that I had a successful run >> > last >> > >>>>>>>>> night. >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> On Tue, Mar 14, 2017 at 1:37 AM siddharth anand < >> > >>>> san...@apache.org >> > >>>>>>>>> >> > >>>>>>>>>>> wrote: >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> I'm going to deploy this to staging now. Fab work Bolke! >> > >>>>>>>>>>>> -s >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> On Mon, Mar 13, 2017 at 2:16 PM, Dan Davydov < >> > >>>>>>>> dan.davy...@airbnb.com >> > >>>>>>>>> . >> > >>>>>>>>>>>> invalid >> > >>>>>>>>>>>>> wrote: >> > >>>>>>>>>>>> >> > >>>>>>>>>>>>> I'll test this on staging as soon as I get a chance (the >> > testing >> > >>>>>>>> is >> > >>>>>>>>>>>>> non-blocking on the rc5). Bolke very much in particular >> :). >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> On Mon, Mar 13, 2017 at 10:46 AM, Jeremiah Lowin < >> > >>>>>>>>> jlo...@apache.org> >> > >>>>>>>>>>>>> wrote: >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>>> +1 (binding) extremely impressed by the work and >> diligence >> > all >> > >>>>>>>>>>>>> contributors >> > >>>>>>>>>>>>>> have put in to getting these blockers fixed, Bolke in >> > >>>>>>>> particular. >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> On Mon, Mar 13, 2017 at 1:07 AM Arthur Wiedmer < >> > >>>>>>>>> art...@apache.org> >> > >>>>>>>>>>>>> wrote: >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> +1 (binding) >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> Thanks again for steering us through Bolke. >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> Best, >> > >>>>>>>>>>>>>>> Arthur >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> On Sun, Mar 12, 2017 at 9:59 PM, Bolke de Bruin < >> > >>>>>>>>>> bdbr...@gmail.com >> > >>>>>>>>>>>> >> > >>>>>>>>>>>>>> wrote: >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> Dear All, >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> Finally, I have been able to make the FIFTH RELEASE >> > >>>>>>>> CANDIDATE >> > >>>>>>>>>> of >> > >>>>>>>>>>>>>> Airflow >> > >>>>>>>>>>>>>>>> 1.8.0 available at: https://dist.apache.org/repos/ >> > >>>>>>>>>>>>>>>> dist/dev/incubator/airflow/ <https://dist.apache.org/ >> > >>>>>>>>>>>>>>>> repos/dist/dev/incubator/airflow/> , public keys are >> > >>>>>>>>> available >> > >>>>>>>>>>> at >> > >>>>>>>>>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/ >> > >>>>>>>>> airflow/ >> > >>>>>>>>>> < >> > >>>>>>>>>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/ >> > >>>>>>>>> airflow/> >> > >>>>>>>>>> . >> > >>>>>>>>>>>> It >> > >>>>>>>>>>>>> is >> > >>>>>>>>>>>>>>>> tagged with a local version “apache.incubating” so it >> > >>>>>>>> allows >> > >>>>>>>>>>>>> upgrading >> > >>>>>>>>>>>>>>> from >> > >>>>>>>>>>>>>>>> earlier releases. >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> Issues fixed since rc4: >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> [AIRFLOW-900] Double trigger should not kill original >> task >> > >>>>>>>>>>> instance >> > >>>>>>>>>>>>>>>> [AIRFLOW-900] Fixes bugs in LocalTaskJob for double run >> > >>>>>>>>>>> protection >> > >>>>>>>>>>>>>>>> [AIRFLOW-932] Do not mark tasks removed when >> backfilling >> > >>>>>>>>>>>>>>>> [AIRFLOW-961] run onkill when SIGTERMed >> > >>>>>>>>>>>>>>>> [AIRFLOW-910] Use parallel task execution for backfills >> > >>>>>>>>>>>>>>>> [AIRFLOW-967] Wrap strings in native for py2 ldap >> > >>>>>>>>> compatibility >> > >>>>>>>>>>>>>>>> [AIRFLOW-941] Use defined parameters for psycopg2 >> > >>>>>>>>>>>>>>>> [AIRFLOW-719] Prevent DAGs from ending prematurely >> > >>>>>>>>>>>>>>>> [AIRFLOW-938] Use test for True in task_stats queries >> > >>>>>>>>>>>>>>>> [AIRFLOW-937] Improve performance of task_stats >> > >>>>>>>>>>>>>>>> [AIRFLOW-933] use ast.literal_eval rather eval because >> > >>>>>>>>>>>>> ast.literal_eval >> > >>>>>>>>>>>>>>>> does not execute input. >> > >>>>>>>>>>>>>>>> [AIRFLOW-919] Running tasks with no start date >> shouldn't >> > >>>>>>>>> break >> > >>>>>>>>>> a >> > >>>>>>>>>>>> DAGs >> > >>>>>>>>>>>>>> UI >> > >>>>>>>>>>>>>>>> [AIRFLOW-897] Prevent dagruns from failing with >> unfinished >> > >>>>>>>>>> tasks >> > >>>>>>>>>>>>>>>> [AIRFLOW-861] make pickle_info endpoint be >> login_required >> > >>>>>>>>>>>>>>>> [AIRFLOW-853] use utf8 encoding for stdout line decode >> > >>>>>>>>>>>>>>>> [AIRFLOW-856] Make sure execution date is set for local >> > >>>>>>>>> client >> > >>>>>>>>>>>>>>>> [AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis >> log >> > >>>>>>>>>>> verbosity >> > >>>>>>>>>>>>>>>> [AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN >> > >>>>>>>>>> exclusively >> > >>>>>>>>>>>>> from >> > >>>>>>>>>>>>>>>> settings >> > >>>>>>>>>>>>>>>> [AIRFLOW-694] Fix config behaviour for empty envvar >> > >>>>>>>>>>>>>>>> [AIRFLOW-365] Set dag.fileloc explicitly and use for >> Code >> > >>>>>>>>> view >> > >>>>>>>>>>>>>>>> [AIRFLOW-931] Do not set QUEUED in TaskInstances >> > >>>>>>>>>>>>>>>> [AIRFLOW-899] Tasks in SCHEDULED state should be white >> in >> > >>>>>>>> the >> > >>>>>>>>>> UI >> > >>>>>>>>>>>>>> instead >> > >>>>>>>>>>>>>>>> of black >> > >>>>>>>>>>>>>>>> [AIRFLOW-895] Address Apache release incompliancies >> > >>>>>>>>>>>>>>>> [AIRFLOW-893][AIRFLOW-510] Fix crashing webservers >> when a >> > >>>>>>>>>> dagrun >> > >>>>>>>>>>>> has >> > >>>>>>>>>>>>> no >> > >>>>>>>>>>>>>>>> start date >> > >>>>>>>>>>>>>>>> [AIRFLOW-793] Enable compressed loading in >> > S3ToHiveTransfer >> > >>>>>>>>>>>>>>>> [AIRFLOW-863] Example DAGs should have recent start >> dates >> > >>>>>>>>>>>>>>>> [AIRFLOW-869] Refactor mark success functionality >> > >>>>>>>>>>>>>>>> [AIRFLOW-856] Make sure execution date is set for local >> > >>>>>>>>> client >> > >>>>>>>>>>>>>>>> [AIRFLOW-814] Fix Presto*CheckOperator.__init__ >> > >>>>>>>>>>>>>>>> [AIRFLOW-844] Fix cgroups directory creation >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> No known issues anymore. >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> I would also like to raise a VOTE for releasing 1.8.0 >> > based >> > >>>>>>>>> on >> > >>>>>>>>>>>>> release >> > >>>>>>>>>>>>>>>> candidate 5, i.e. just renaming release candidate 5 to >> > >>>>>>>> 1.8.0 >> > >>>>>>>>>>>> release. >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> Please respond to this email by: >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> +1,0,-1 with *binding* if you are a PMC member or >> > >>>>>>>>> *non-binding* >> > >>>>>>>>>>> if >> > >>>>>>>>>>>>> you >> > >>>>>>>>>>>>>>> are >> > >>>>>>>>>>>>>>>> not. >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> Thanks! >> > >>>>>>>>>>>>>>>> Bolke >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> My VOTE: +1 (binding) >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> -- >> > >>>>>>>>>>>> _/ >> > >>>>>>>>>>>> _/ Alex Van Boxel >> > >>>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>> >> > >>>>>> >> > >>>>> >> > >>>>> >> > >>>> >> > >> >> > > >> > >> > >> > >
