PR is available: https://github.com/apache/incubator-airflow/pull/2154
But marked for 1.8.1. - Bolke > On 15 Mar 2017, at 14:37, Bolke de Bruin <bdbr...@gmail.com> wrote: > > On second thought I do consider it a bug and can have a fix out pretty > quickly, but I don’t consider it a blocker. > > - B. > >> On 15 Mar 2017, at 14:21, Bolke de Bruin <bdbr...@gmail.com> wrote: >> >> Just to be clear: Also in 1.7.1 the DagRun was marked successful, but its >> tasks continued to be scheduled. So one could also consider 1.7.1 behaviour >> a bug. I am not sure here, but I think it kind of makes sense to consider >> the behaviour of 1.7.1 a bug. It has been present throughout all the 1.8 >> rc/beta/apha series. >> >> So yes it is a change in behaviour whether it is a regression or an >> integrity improvement is up for discussion. Either way I don’t consider it a >> blocker. >> >> Bolke. >> >>> On 15 Mar 2017, at 14:06, siddharth anand <san...@apache.org> wrote: >>> >>> Here's the JIRA : >>> https://issues.apache.org/jira/browse/AIRFLOW-989 >>> >>> I confirmed it is a regression from 1.7.1.3, which I installed via pip and >>> tested against the same DAG in the JIRA. >>> >>> The issue occurs if a leaf / last / terminal downstream task is not >>> cleared. You won't see this issue if you clear the entire DAG Run or clear >>> a task and all of its downstream tasks. If you truly want to only clear and >>> rerun a task, but not its downstream tasks, you can use the CLI to execute >>> a specific task (e.g. vial airflow run). >>> >>> This is a change in behavior -- if we do go ahead with the release, then >>> this JIRA should be in a list of JIRAs of known issues related to the new >>> version. >>> -s >>> >>> On Wed, Mar 15, 2017 at 9:17 AM, Chris Riccomini <criccom...@apache.org> >>> wrote: >>> >>>> @Sid, does this happen if you clear downstream as well? >>>> >>>> On Wed, Mar 15, 2017 at 9:04 AM, Chris Riccomini <criccom...@apache.org> >>>> wrote: >>>> >>>>> Has anyone been able to reproduce Sid's issue? >>>>> >>>>> On Tue, Mar 14, 2017 at 11:17 PM, Bolke de Bruin <bdbr...@gmail.com> >>>>> wrote: >>>>> >>>>>> That is not an airflow error, but a Kerberos error. Try executing the >>>>>> kinit command on the command line by yourself. >>>>>> >>>>>> Bolke >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>>> On 14 Mar 2017, at 23:11, Ruslan Dautkhanov <dautkha...@gmail.com> >>>>>> wrote: >>>>>>> >>>>>>> `airflow kerberos` is broken in 1.8-rc5 >>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-987 >>>>>>> Hopefully fix can be part of the 1.8 release. >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Ruslan Dautkhanov >>>>>>> >>>>>>>> On Tue, Mar 14, 2017 at 6:19 PM, siddharth anand <san...@apache.org> >>>>>> wrote: >>>>>>>> >>>>>>>> FYI, >>>>>>>> I've just hit a major bug in the release candidate related to "clear >>>>>> task" >>>>>>>> behavior. >>>>>>>> >>>>>>>> I've been running airflow in both stage and prod since yesterday on >>>>>> rc5 and >>>>>>>> have reproduced this in both environments. I will file a JIRA for >>>> this >>>>>>>> tonight, but wanted to send a note over email as well. >>>>>>>> >>>>>>>> In my example, I have a 2 task DAG. For a given DAG run that has >>>>>> completed >>>>>>>> successfully, if I >>>>>>>> 1) clear task2 (leaf task in this case), the previously-successful >>>> DAG >>>>>> Run >>>>>>>> goes back to Running, requeues, and executes the task successfully. >>>>>> The DAG >>>>>>>> Run the returns from Running to Success. >>>>>>>> 2) clear task1 (root task in this case), the previously-successful >>>> DAG >>>>>> Run >>>>>>>> goes back to Running, DOES NOT requeue or execute the task at all. >>>> The >>>>>> DAG >>>>>>>> Run the returns from Running to Success though it never ran the task. >>>>>>>> >>>>>>>> 1) is expected and previous behavior. 2) is a regression. >>>>>>>> >>>>>>>> The only workaround is to use the CLI to run the task cleared. Here >>>> are >>>>>>>> some images : >>>>>>>> *After Clearing the Tasks* >>>>>>>> https://www.dropbox.com/s/wmuxt0krwx6wurr/Screenshot% >>>>>>>> 202017-03-14%2014.09.34.png?dl=0 >>>>>>>> >>>>>>>> *After DAG Runs return to Success* >>>>>>>> https://www.dropbox.com/s/qop933rzgdzchpd/Screenshot% >>>>>>>> 202017-03-14%2014.09.49.png?dl=0 >>>>>>>> >>>>>>>> This is a major regression because it will force everyone to use the >>>>>> CLI >>>>>>>> for things that they would normally use the UI for. >>>>>>>> >>>>>>>> -s >>>>>>>> >>>>>>>> >>>>>>>> -s >>>>>>>> >>>>>>>> >>>>>>>>> On Tue, Mar 14, 2017 at 1:32 PM, Daniel Huang <dxhu...@gmail.com> >>>>>> wrote: >>>>>>>>> >>>>>>>>> +1 (non-binding)! >>>>>>>>> >>>>>>>>> On Tue, Mar 14, 2017 at 11:35 AM, siddharth anand < >>>> san...@apache.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> +1 (binding) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Mar 14, 2017 at 8:42 AM, Maxime Beauchemin < >>>>>>>>>> maximebeauche...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> +1 (binding) >>>>>>>>>>> >>>>>>>>>>> On Tue, Mar 14, 2017 at 3:59 AM, Alex Van Boxel <a...@vanboxel.be >>>>> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> +1 (binding) >>>>>>>>>>>> >>>>>>>>>>>> Note: we had to revert all our ONE_SUCCESS with ALL_SUCCESS >>>> trigger >>>>>>>>>> rules >>>>>>>>>>>> where the parent nodes where joining with a SKIP. But I can of >>>>>>>> should >>>>>>>>>>> have >>>>>>>>>>>> known this was coming. Apart of that I had a successful run last >>>>>>>>> night. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Mar 14, 2017 at 1:37 AM siddharth anand < >>>> san...@apache.org >>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> I'm going to deploy this to staging now. Fab work Bolke! >>>>>>>>>>>> -s >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Mar 13, 2017 at 2:16 PM, Dan Davydov < >>>>>>>> dan.davy...@airbnb.com >>>>>>>>> . >>>>>>>>>>>> invalid >>>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I'll test this on staging as soon as I get a chance (the testing >>>>>>>> is >>>>>>>>>>>>> non-blocking on the rc5). Bolke very much in particular :). >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Mar 13, 2017 at 10:46 AM, Jeremiah Lowin < >>>>>>>>> jlo...@apache.org> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> +1 (binding) extremely impressed by the work and diligence all >>>>>>>>>>>>> contributors >>>>>>>>>>>>>> have put in to getting these blockers fixed, Bolke in >>>>>>>> particular. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Mar 13, 2017 at 1:07 AM Arthur Wiedmer < >>>>>>>>> art...@apache.org> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> +1 (binding) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks again for steering us through Bolke. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>> Arthur >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sun, Mar 12, 2017 at 9:59 PM, Bolke de Bruin < >>>>>>>>>> bdbr...@gmail.com >>>>>>>>>>>> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Dear All, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Finally, I have been able to make the FIFTH RELEASE >>>>>>>> CANDIDATE >>>>>>>>>> of >>>>>>>>>>>>>> Airflow >>>>>>>>>>>>>>>> 1.8.0 available at: https://dist.apache.org/repos/ >>>>>>>>>>>>>>>> dist/dev/incubator/airflow/ <https://dist.apache.org/ >>>>>>>>>>>>>>>> repos/dist/dev/incubator/airflow/> , public keys are >>>>>>>>> available >>>>>>>>>>> at >>>>>>>>>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/ >>>>>>>>> airflow/ >>>>>>>>>> < >>>>>>>>>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/ >>>>>>>>> airflow/> >>>>>>>>>> . >>>>>>>>>>>> It >>>>>>>>>>>>> is >>>>>>>>>>>>>>>> tagged with a local version “apache.incubating” so it >>>>>>>> allows >>>>>>>>>>>>> upgrading >>>>>>>>>>>>>>> from >>>>>>>>>>>>>>>> earlier releases. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Issues fixed since rc4: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> [AIRFLOW-900] Double trigger should not kill original task >>>>>>>>>>> instance >>>>>>>>>>>>>>>> [AIRFLOW-900] Fixes bugs in LocalTaskJob for double run >>>>>>>>>>> protection >>>>>>>>>>>>>>>> [AIRFLOW-932] Do not mark tasks removed when backfilling >>>>>>>>>>>>>>>> [AIRFLOW-961] run onkill when SIGTERMed >>>>>>>>>>>>>>>> [AIRFLOW-910] Use parallel task execution for backfills >>>>>>>>>>>>>>>> [AIRFLOW-967] Wrap strings in native for py2 ldap >>>>>>>>> compatibility >>>>>>>>>>>>>>>> [AIRFLOW-941] Use defined parameters for psycopg2 >>>>>>>>>>>>>>>> [AIRFLOW-719] Prevent DAGs from ending prematurely >>>>>>>>>>>>>>>> [AIRFLOW-938] Use test for True in task_stats queries >>>>>>>>>>>>>>>> [AIRFLOW-937] Improve performance of task_stats >>>>>>>>>>>>>>>> [AIRFLOW-933] use ast.literal_eval rather eval because >>>>>>>>>>>>> ast.literal_eval >>>>>>>>>>>>>>>> does not execute input. >>>>>>>>>>>>>>>> [AIRFLOW-919] Running tasks with no start date shouldn't >>>>>>>>> break >>>>>>>>>> a >>>>>>>>>>>> DAGs >>>>>>>>>>>>>> UI >>>>>>>>>>>>>>>> [AIRFLOW-897] Prevent dagruns from failing with unfinished >>>>>>>>>> tasks >>>>>>>>>>>>>>>> [AIRFLOW-861] make pickle_info endpoint be login_required >>>>>>>>>>>>>>>> [AIRFLOW-853] use utf8 encoding for stdout line decode >>>>>>>>>>>>>>>> [AIRFLOW-856] Make sure execution date is set for local >>>>>>>>> client >>>>>>>>>>>>>>>> [AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis log >>>>>>>>>>> verbosity >>>>>>>>>>>>>>>> [AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN >>>>>>>>>> exclusively >>>>>>>>>>>>> from >>>>>>>>>>>>>>>> settings >>>>>>>>>>>>>>>> [AIRFLOW-694] Fix config behaviour for empty envvar >>>>>>>>>>>>>>>> [AIRFLOW-365] Set dag.fileloc explicitly and use for Code >>>>>>>>> view >>>>>>>>>>>>>>>> [AIRFLOW-931] Do not set QUEUED in TaskInstances >>>>>>>>>>>>>>>> [AIRFLOW-899] Tasks in SCHEDULED state should be white in >>>>>>>> the >>>>>>>>>> UI >>>>>>>>>>>>>> instead >>>>>>>>>>>>>>>> of black >>>>>>>>>>>>>>>> [AIRFLOW-895] Address Apache release incompliancies >>>>>>>>>>>>>>>> [AIRFLOW-893][AIRFLOW-510] Fix crashing webservers when a >>>>>>>>>> dagrun >>>>>>>>>>>> has >>>>>>>>>>>>> no >>>>>>>>>>>>>>>> start date >>>>>>>>>>>>>>>> [AIRFLOW-793] Enable compressed loading in S3ToHiveTransfer >>>>>>>>>>>>>>>> [AIRFLOW-863] Example DAGs should have recent start dates >>>>>>>>>>>>>>>> [AIRFLOW-869] Refactor mark success functionality >>>>>>>>>>>>>>>> [AIRFLOW-856] Make sure execution date is set for local >>>>>>>>> client >>>>>>>>>>>>>>>> [AIRFLOW-814] Fix Presto*CheckOperator.__init__ >>>>>>>>>>>>>>>> [AIRFLOW-844] Fix cgroups directory creation >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> No known issues anymore. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I would also like to raise a VOTE for releasing 1.8.0 based >>>>>>>>> on >>>>>>>>>>>>> release >>>>>>>>>>>>>>>> candidate 5, i.e. just renaming release candidate 5 to >>>>>>>> 1.8.0 >>>>>>>>>>>> release. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please respond to this email by: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> +1,0,-1 with *binding* if you are a PMC member or >>>>>>>>> *non-binding* >>>>>>>>>>> if >>>>>>>>>>>>> you >>>>>>>>>>>>>>> are >>>>>>>>>>>>>>>> not. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>>>> Bolke >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> My VOTE: +1 (binding) >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> _/ >>>>>>>>>>>> _/ Alex Van Boxel >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>>> >>>> >> >
