On second thought I do consider it a bug and can have a fix out pretty quickly, but I don’t consider it a blocker.
- B. > On 15 Mar 2017, at 14:21, Bolke de Bruin <bdbr...@gmail.com> wrote: > > Just to be clear: Also in 1.7.1 the DagRun was marked successful, but its > tasks continued to be scheduled. So one could also consider 1.7.1 behaviour a > bug. I am not sure here, but I think it kind of makes sense to consider the > behaviour of 1.7.1 a bug. It has been present throughout all the 1.8 > rc/beta/apha series. > > So yes it is a change in behaviour whether it is a regression or an integrity > improvement is up for discussion. Either way I don’t consider it a blocker. > > Bolke. > >> On 15 Mar 2017, at 14:06, siddharth anand <san...@apache.org> wrote: >> >> Here's the JIRA : >> https://issues.apache.org/jira/browse/AIRFLOW-989 >> >> I confirmed it is a regression from 1.7.1.3, which I installed via pip and >> tested against the same DAG in the JIRA. >> >> The issue occurs if a leaf / last / terminal downstream task is not >> cleared. You won't see this issue if you clear the entire DAG Run or clear >> a task and all of its downstream tasks. If you truly want to only clear and >> rerun a task, but not its downstream tasks, you can use the CLI to execute >> a specific task (e.g. vial airflow run). >> >> This is a change in behavior -- if we do go ahead with the release, then >> this JIRA should be in a list of JIRAs of known issues related to the new >> version. >> -s >> >> On Wed, Mar 15, 2017 at 9:17 AM, Chris Riccomini <criccom...@apache.org> >> wrote: >> >>> @Sid, does this happen if you clear downstream as well? >>> >>> On Wed, Mar 15, 2017 at 9:04 AM, Chris Riccomini <criccom...@apache.org> >>> wrote: >>> >>>> Has anyone been able to reproduce Sid's issue? >>>> >>>> On Tue, Mar 14, 2017 at 11:17 PM, Bolke de Bruin <bdbr...@gmail.com> >>>> wrote: >>>> >>>>> That is not an airflow error, but a Kerberos error. Try executing the >>>>> kinit command on the command line by yourself. >>>>> >>>>> Bolke >>>>> >>>>> Sent from my iPhone >>>>> >>>>>> On 14 Mar 2017, at 23:11, Ruslan Dautkhanov <dautkha...@gmail.com> >>>>> wrote: >>>>>> >>>>>> `airflow kerberos` is broken in 1.8-rc5 >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-987 >>>>>> Hopefully fix can be part of the 1.8 release. >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Ruslan Dautkhanov >>>>>> >>>>>>> On Tue, Mar 14, 2017 at 6:19 PM, siddharth anand <san...@apache.org> >>>>> wrote: >>>>>>> >>>>>>> FYI, >>>>>>> I've just hit a major bug in the release candidate related to "clear >>>>> task" >>>>>>> behavior. >>>>>>> >>>>>>> I've been running airflow in both stage and prod since yesterday on >>>>> rc5 and >>>>>>> have reproduced this in both environments. I will file a JIRA for >>> this >>>>>>> tonight, but wanted to send a note over email as well. >>>>>>> >>>>>>> In my example, I have a 2 task DAG. For a given DAG run that has >>>>> completed >>>>>>> successfully, if I >>>>>>> 1) clear task2 (leaf task in this case), the previously-successful >>> DAG >>>>> Run >>>>>>> goes back to Running, requeues, and executes the task successfully. >>>>> The DAG >>>>>>> Run the returns from Running to Success. >>>>>>> 2) clear task1 (root task in this case), the previously-successful >>> DAG >>>>> Run >>>>>>> goes back to Running, DOES NOT requeue or execute the task at all. >>> The >>>>> DAG >>>>>>> Run the returns from Running to Success though it never ran the task. >>>>>>> >>>>>>> 1) is expected and previous behavior. 2) is a regression. >>>>>>> >>>>>>> The only workaround is to use the CLI to run the task cleared. Here >>> are >>>>>>> some images : >>>>>>> *After Clearing the Tasks* >>>>>>> https://www.dropbox.com/s/wmuxt0krwx6wurr/Screenshot% >>>>>>> 202017-03-14%2014.09.34.png?dl=0 >>>>>>> >>>>>>> *After DAG Runs return to Success* >>>>>>> https://www.dropbox.com/s/qop933rzgdzchpd/Screenshot% >>>>>>> 202017-03-14%2014.09.49.png?dl=0 >>>>>>> >>>>>>> This is a major regression because it will force everyone to use the >>>>> CLI >>>>>>> for things that they would normally use the UI for. >>>>>>> >>>>>>> -s >>>>>>> >>>>>>> >>>>>>> -s >>>>>>> >>>>>>> >>>>>>>> On Tue, Mar 14, 2017 at 1:32 PM, Daniel Huang <dxhu...@gmail.com> >>>>> wrote: >>>>>>>> >>>>>>>> +1 (non-binding)! >>>>>>>> >>>>>>>> On Tue, Mar 14, 2017 at 11:35 AM, siddharth anand < >>> san...@apache.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> +1 (binding) >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Mar 14, 2017 at 8:42 AM, Maxime Beauchemin < >>>>>>>>> maximebeauche...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> +1 (binding) >>>>>>>>>> >>>>>>>>>> On Tue, Mar 14, 2017 at 3:59 AM, Alex Van Boxel <a...@vanboxel.be >>>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> +1 (binding) >>>>>>>>>>> >>>>>>>>>>> Note: we had to revert all our ONE_SUCCESS with ALL_SUCCESS >>> trigger >>>>>>>>> rules >>>>>>>>>>> where the parent nodes where joining with a SKIP. But I can of >>>>>>> should >>>>>>>>>> have >>>>>>>>>>> known this was coming. Apart of that I had a successful run last >>>>>>>> night. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Mar 14, 2017 at 1:37 AM siddharth anand < >>> san...@apache.org >>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> I'm going to deploy this to staging now. Fab work Bolke! >>>>>>>>>>> -s >>>>>>>>>>> >>>>>>>>>>> On Mon, Mar 13, 2017 at 2:16 PM, Dan Davydov < >>>>>>> dan.davy...@airbnb.com >>>>>>>> . >>>>>>>>>>> invalid >>>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> I'll test this on staging as soon as I get a chance (the testing >>>>>>> is >>>>>>>>>>>> non-blocking on the rc5). Bolke very much in particular :). >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Mar 13, 2017 at 10:46 AM, Jeremiah Lowin < >>>>>>>> jlo...@apache.org> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> +1 (binding) extremely impressed by the work and diligence all >>>>>>>>>>>> contributors >>>>>>>>>>>>> have put in to getting these blockers fixed, Bolke in >>>>>>> particular. >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Mar 13, 2017 at 1:07 AM Arthur Wiedmer < >>>>>>>> art...@apache.org> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> +1 (binding) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks again for steering us through Bolke. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Best, >>>>>>>>>>>>>> Arthur >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sun, Mar 12, 2017 at 9:59 PM, Bolke de Bruin < >>>>>>>>> bdbr...@gmail.com >>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Dear All, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Finally, I have been able to make the FIFTH RELEASE >>>>>>> CANDIDATE >>>>>>>>> of >>>>>>>>>>>>> Airflow >>>>>>>>>>>>>>> 1.8.0 available at: https://dist.apache.org/repos/ >>>>>>>>>>>>>>> dist/dev/incubator/airflow/ <https://dist.apache.org/ >>>>>>>>>>>>>>> repos/dist/dev/incubator/airflow/> , public keys are >>>>>>>> available >>>>>>>>>> at >>>>>>>>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/ >>>>>>>> airflow/ >>>>>>>>> < >>>>>>>>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/ >>>>>>>> airflow/> >>>>>>>>> . >>>>>>>>>>> It >>>>>>>>>>>> is >>>>>>>>>>>>>>> tagged with a local version “apache.incubating” so it >>>>>>> allows >>>>>>>>>>>> upgrading >>>>>>>>>>>>>> from >>>>>>>>>>>>>>> earlier releases. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Issues fixed since rc4: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [AIRFLOW-900] Double trigger should not kill original task >>>>>>>>>> instance >>>>>>>>>>>>>>> [AIRFLOW-900] Fixes bugs in LocalTaskJob for double run >>>>>>>>>> protection >>>>>>>>>>>>>>> [AIRFLOW-932] Do not mark tasks removed when backfilling >>>>>>>>>>>>>>> [AIRFLOW-961] run onkill when SIGTERMed >>>>>>>>>>>>>>> [AIRFLOW-910] Use parallel task execution for backfills >>>>>>>>>>>>>>> [AIRFLOW-967] Wrap strings in native for py2 ldap >>>>>>>> compatibility >>>>>>>>>>>>>>> [AIRFLOW-941] Use defined parameters for psycopg2 >>>>>>>>>>>>>>> [AIRFLOW-719] Prevent DAGs from ending prematurely >>>>>>>>>>>>>>> [AIRFLOW-938] Use test for True in task_stats queries >>>>>>>>>>>>>>> [AIRFLOW-937] Improve performance of task_stats >>>>>>>>>>>>>>> [AIRFLOW-933] use ast.literal_eval rather eval because >>>>>>>>>>>> ast.literal_eval >>>>>>>>>>>>>>> does not execute input. >>>>>>>>>>>>>>> [AIRFLOW-919] Running tasks with no start date shouldn't >>>>>>>> break >>>>>>>>> a >>>>>>>>>>> DAGs >>>>>>>>>>>>> UI >>>>>>>>>>>>>>> [AIRFLOW-897] Prevent dagruns from failing with unfinished >>>>>>>>> tasks >>>>>>>>>>>>>>> [AIRFLOW-861] make pickle_info endpoint be login_required >>>>>>>>>>>>>>> [AIRFLOW-853] use utf8 encoding for stdout line decode >>>>>>>>>>>>>>> [AIRFLOW-856] Make sure execution date is set for local >>>>>>>> client >>>>>>>>>>>>>>> [AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis log >>>>>>>>>> verbosity >>>>>>>>>>>>>>> [AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN >>>>>>>>> exclusively >>>>>>>>>>>> from >>>>>>>>>>>>>>> settings >>>>>>>>>>>>>>> [AIRFLOW-694] Fix config behaviour for empty envvar >>>>>>>>>>>>>>> [AIRFLOW-365] Set dag.fileloc explicitly and use for Code >>>>>>>> view >>>>>>>>>>>>>>> [AIRFLOW-931] Do not set QUEUED in TaskInstances >>>>>>>>>>>>>>> [AIRFLOW-899] Tasks in SCHEDULED state should be white in >>>>>>> the >>>>>>>>> UI >>>>>>>>>>>>> instead >>>>>>>>>>>>>>> of black >>>>>>>>>>>>>>> [AIRFLOW-895] Address Apache release incompliancies >>>>>>>>>>>>>>> [AIRFLOW-893][AIRFLOW-510] Fix crashing webservers when a >>>>>>>>> dagrun >>>>>>>>>>> has >>>>>>>>>>>> no >>>>>>>>>>>>>>> start date >>>>>>>>>>>>>>> [AIRFLOW-793] Enable compressed loading in S3ToHiveTransfer >>>>>>>>>>>>>>> [AIRFLOW-863] Example DAGs should have recent start dates >>>>>>>>>>>>>>> [AIRFLOW-869] Refactor mark success functionality >>>>>>>>>>>>>>> [AIRFLOW-856] Make sure execution date is set for local >>>>>>>> client >>>>>>>>>>>>>>> [AIRFLOW-814] Fix Presto*CheckOperator.__init__ >>>>>>>>>>>>>>> [AIRFLOW-844] Fix cgroups directory creation >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> No known issues anymore. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I would also like to raise a VOTE for releasing 1.8.0 based >>>>>>>> on >>>>>>>>>>>> release >>>>>>>>>>>>>>> candidate 5, i.e. just renaming release candidate 5 to >>>>>>> 1.8.0 >>>>>>>>>>> release. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please respond to this email by: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> +1,0,-1 with *binding* if you are a PMC member or >>>>>>>> *non-binding* >>>>>>>>>> if >>>>>>>>>>>> you >>>>>>>>>>>>>> are >>>>>>>>>>>>>>> not. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>>> Bolke >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> My VOTE: +1 (binding) >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> _/ >>>>>>>>>>> _/ Alex Van Boxel >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>>> >>> >
