Gotcha. I can relate to that. With hindsight the backfill change should not have made it into the RC cycle. Any way I hope you guys can jump on the wagon pretty soon again: I do think 1.8.1 is in a lot better shape than 1.8.0. It would be nice if you guys could take on 1.8.2.
And as mentioned if it has to do with “backfill” for the moment I feel my name is tagged on that. - Bolke > On 4 May 2017, at 21:27, Dan Davydov <[email protected]> wrote: > > Thinking back it may have been 1.8.0rc5-> 1.8.0 regressions. I am still > worried about the large number of PRs in 1.8.1 even if they are all bug fixes > though (known issues that we already have patches for vs unknown new issues > introduced with the 1.8.1 patches) , but I agree with your sentiment that > these PRs should most likely make things more stable. > > On Thu, May 4, 2017 at 10:55 AM, Alex Guziel <[email protected] > <mailto:[email protected]>> wrote: > I don't think any of the fixes I did were regressions. > > On Thu, May 4, 2017 at 8:11 AM, Bolke de Bruin <[email protected] > <mailto:[email protected]>> wrote: > I know of one that Alex wanted to get in, but wasn’t targeted for 1.8.1 in > Jira and thus didn’t make the cut at RC time. There is is another one out > that seems to have stalled a bit > (https://github.com/apache/incubator-airflow/pull/2205 > <https://github.com/apache/incubator-airflow/pull/2205>). > > Reading the changelog of 1.8.1 I see bug fixes, apache requirements and one > “new” feature (UI lightning bolt). Regressions could have happened but we > have been quite vigilant on the fact that these bug fixes needed proper > tests, so I am very interested in 1.8.0 -> 1.8.1 regressions. If it is a > pre-backfill-change 1.8.0 to 1.8.1 regression then I would also like to know, > cause I made that change and feel responsible for it. > > Cheers > Bolke > > >> On 3 May 2017, at 22:13, Dan Davydov <[email protected] >> <mailto:[email protected]>> wrote: >> >> cc Alex and Rui who were working on fixes, I'm not sure if their commits got >> in before 1.8.1. >> >> On Wed, May 3, 2017 at 1:09 PM, Bolke de Bruin <[email protected] >> <mailto:[email protected]>> wrote: >> Hi Dan, >> >> (Thread renamed to make sure it does not clash, dev@ now added) >> >> It surprises me that you found regression from 1.8.0 to 1.8.1 as 1.8.1 is >> very much focused on bug fixes. Were the regressions shared yet? >> >> The whole 1.8.X release will be bug fix focused (per release management) and >> minor feature updates. The 1.9.0 release will be the first release with >> major feature updates. So what you want, more robustness and focus on >> stability, is now underway. I agree with beefing up tests and including the >> major operators in this. Executors should also be on this list btw. Turning >> on coverage reporting might be a first step in helping this (it isn’t the >> solution obviously). >> >> Cheers >> Bolke >> >> >>> On 3 May 2017, at 20:28, Dan Davydov <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> We saw several regressions moving from 1.8.0 to 1.8.1 the first time we >>> tried, and while I think we merged all our fixes to master (not sure if >>> they all made it into 1.8.1 however), we have put releasing on hold due to >>> stability issues from the last couple of releases. It's either the case >>> that: >>> A) Airbnb requires more robustness from new releases. >>> or >>> B) Most companies using Airflow require more robustness and we should halt >>> on feature work until we are more confident in our testing >>> >>> I think the biggest problem currently is the lack of unit testing coverage, >>> e.g. when the backfill framework was refactored (which was the right >>> long-term fix), it caused a lot of breakages that weren't caught by tests. >>> I think we need to audit the major operators/classes and beef up the unit >>> testing coverage. The coverage metric does not necessarily cover these >>> cases (e.g. cyclomatic complexity). Writing regression tests is good but we >>> shouldn't have so many new blocker issues in our releases. >>> >>> We are fighting some fires internally at the moment (not Airflow related), >>> but Alex and I have been working on some stuff that we will push to the >>> community once we are done. Alex is working on a good solution for python >>> package isolation, and I'm working on integration with Kubernetes at the >>> executor level. >>> >>> Feel free to forward any of my messages to the dev mailing list. >>> >>> On Wed, May 3, 2017 at 11:18 AM, Bolke de Bruin <[email protected] >>> <mailto:[email protected]>> wrote: >>> Grrr, I seriously dislike to send button on the touch bar…here goes again. >>> >>> Hi Dan, >>> >>> (Please note I would like to forward the next message to dev@, but let me >>> know if you don’t find it comfortable) >>> >>> I understand your point. The gap between 1.7.1 was large in terms of >>> functionality changes etc. It was going to be a (bit?) rough and as you >>> guys are using many of the edge cases you probably found more issues than >>> any of us. Still, between 1.8.0 and 1.8.1 we have added many tests >>> (coverage increased from 67% to close to 69%, which is a lot as you know). >>> It would be nice if you can share where your areas of concern are so we can >>> address those and a suggestion on how to proceed with integration tests is >>> also welcome. >>> >>> You guys (=Airbnb) have been a bit quiet over the past couple of days, so I >>> am getting a bit worried in terms of engagement. Is that warranted? >>> >>> Cheers >>> Bolke >>> >>> >>>> On 3 May 2017, at 20:13, Bolke de Bruin <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> Hi Dan, >>>> >>>> (Please note I would like to forward the next message to dev@, but let me >>>> know if you don’t find it comfortable) >>>> >>>> I understand your point. The gap between 1.7.1 was large in terms of >>>> functionality changes etc. It was going to be a (bit?) rough and as you >>>> guys are using many of the edge cases you probably found more issues than >>>> any of us. Still, between 1.8.0 and 1.8.1 we have added many tests >>>> (coverage increased from 67 >>>>> On 3 May 2017, at 19:41, Arthur Wiedmer <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> As a counterpoint, >>>>> >>>>> I am comfortable voting +1 on this release in the sense that it fixes >>>>> some of the issues with 1.8.0. It is unfortunate that we cannot test it >>>>> on the Airbnb production for now and we should definitely invest in >>>>> increasing testing coverage, but some of the fixes are needed for ease of >>>>> use/adoption (See for instance AIRFLOW-832), and this release is a step >>>>> in the right direction. >>>>> >>>>> Best, >>>>> Arthur >>>>> >>>>> On Wed, May 3, 2017 at 10:30 AM, Dan Davydov <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> I'm not comfortable voting without doing comprehensive staging and we >>>>> aren't comfortable doing an internal lease for now until we fix the state >>>>> of unit test coverage and integration tests. >>>>> >>>>> On May 3, 2017 8:42 AM, "Bolke de Bruin" <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> Hey Guys, >>>>> >>>>> Chris has been preparing the 1.8.1 release and the vote is running for >>>>> it. Only one day left though! Would you mind casting your vote? Only >>>>> Chris and I have voted binding until so far. >>>>> >>>>> (Please reply to the message on the list, not this message). >>>>> >>>>> Cheers! >>>>> Bolke >>>>> >>>>>> Begin forwarded message: >>>>>> >>>>>> From: Chris Riccomini <[email protected] >>>>>> <mailto:[email protected]>> >>>>>> Subject: [VOTE] Release Airflow 1.8.1 RC2 >>>>>> Date: 1 May 2017 at 19:58:41 GMT+2 >>>>>> To: [email protected] >>>>>> <mailto:[email protected]> >>>>>> Reply-To: [email protected] >>>>>> <mailto:[email protected]> >>>>>> >>>>>> Dear All, >>>>>> >>>>>> _WARN: The package version for this RC is 1.8.1 (does not include RC2 in >>>>>> version number). As such, any future 1.8.1 installatinos will have to be >>>>>> force installed. PIP will not be able to distinguish between RCs and >>>>>> final >>>>>> versions. Again, you'll have to force install the package. This can be >>>>>> done >>>>>> by adding `--force-reinstall` to your `pip install` commands._ >>>>>> >>>>>> I've made Airflow 1.8.1 RC2 available at: >>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow >>>>>> <https://dist.apache.org/repos/dist/dev/incubator/airflow>, public keys >>>>>> are >>>>>> available at >>>>>> https://dist.apache.org/repos/dist/release/incubator/airflow >>>>>> <https://dist.apache.org/repos/dist/release/incubator/airflow>. >>>>>> >>>>>> New issues fixed in 1.8.1 RC2: >>>>>> >>>>>> [AIRFLOW-1142] SubDAG Tasks Not Executed Even Though All Dependen >>>>>> [AIRFLOW-1004] `airflow webserver -D` runs in foreground >>>>>> [AIRFLOW-492] Insert into dag_stats table results into failed ta >>>>>> >>>>>> Issues fixed in 1.8.1 RC0/RC1, and included in RC2: >>>>>> >>>>>> [AIRFLOW-1138] Add licenses to files in scripts directory >>>>>> [AIRFLOW-1127] Move license notices to LICENSE instead of NOTICE >>>>>> [AIRFLOW-1124] Do not set all task instances to scheduled on back >>>>>> [AIRFLOW-1120] Update version view to include Apache prefix >>>>>> [AIRFLOW-1062] DagRun#find returns wrong result if external_trigg >>>>>> [AIRFLOW-1054] Fix broken import on test_dag >>>>>> [AIRFLOW-1050] Retries ignored - regression >>>>>> [AIRFLOW-1033] TypeError: can't compare datetime.datetime to None >>>>>> [AIRFLOW-1017] get_task_instance should return None instead of th >>>>>> [AIRFLOW-1011] Fix bug in BackfillJob._execute() for SubDAGs >>>>>> [AIRFLOW-1001] Landing Time shows "unsupported operand type(s) fo >>>>>> [AIRFLOW-1000] Rebrand to Apache Airflow instead of Airflow >>>>>> [AIRFLOW-989] Clear Task Regression >>>>>> [AIRFLOW-974] airflow.util.file mkdir has a race condition >>>>>> [AIRFLOW-906] Update Code icon from lightning bolt to file >>>>>> [AIRFLOW-858] Configurable database name for DB operators >>>>>> [AIRFLOW-853] ssh_execute_operator.py stdout decode default to A >>>>>> [AIRFLOW-832] Fix debug server >>>>>> [AIRFLOW-817] Trigger dag fails when using CLI + API >>>>>> [AIRFLOW-816] Make sure to pull nvd3 from local resources >>>>>> [AIRFLOW-815] Add previous/next execution dates to available def >>>>>> [AIRFLOW-813] Fix unterminated unit tests in tests.job (tests/jo >>>>>> [AIRFLOW-812] Scheduler job terminates when there is no dag file >>>>>> [AIRFLOW-806] UI should properly ignore DAG doc when it is None >>>>>> [AIRFLOW-794] Consistent access to DAGS_FOLDER and SQL_ALCHEMY_C >>>>>> [AIRFLOW-785] ImportError if cgroupspy is not installed >>>>>> [AIRFLOW-784] Cannot install with funcsigs > 1.0.0 >>>>>> [AIRFLOW-780] The UI no longer shows broken DAGs >>>>>> [AIRFLOW-777] dag_is_running is initlialized to True instead of >>>>>> [AIRFLOW-719] Skipped operations make DAG finish prematurely >>>>>> [AIRFLOW-694] Empty env vars do not overwrite non-empty config v >>>>>> [AIRFLOW-139] Executing VACUUM with PostgresOperator >>>>>> [AIRFLOW-111] DAG concurrency is not honored >>>>>> [AIRFLOW-88] Improve clarity Travis CI reports >>>>>> >>>>>> I would like to raise a VOTE for releasing 1.8.1 based on release >>>>>> candidate >>>>>> 2. >>>>>> >>>>>> Please respond to this email by: >>>>>> >>>>>> +1,0,-1 with *binding* if you are a PMC member or *non-binding* if you >>>>>> are >>>>>> not. >>>>>> >>>>>> Vote will run for 72 hours (ends this Thursday). >>>>>> >>>>>> Thanks! >>>>>> Chris >>>>>> >>>>>> My VOTE: +1 (binding) >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Arthur Wiedmer >>>>> (Pronouns: He, Him) >>>>> Data Engineering, Airbnb >>>>> www.airbnb.com <http://www.airbnb.com/> >>> >>> >> >> > > >
