Hey all, Current blockers for 1.9.0:
https://issues.apache.org/jira/browse/AIRFLOW-1711 https://issues.apache.org/jira/browse/AIRFLOW-1018 Cheers, Chris On Tue, Oct 24, 2017 at 11:06 AM, Chris Riccomini <[email protected]> wrote: > Hey all, > > Talked with @Bolke on Gitter. Here are the currently blocking issues for > 1.9.0: > > AIRFLOW-1744 |Bug ||Blocker ||Open |task.retries can > be False > AIRFLOW-1731 |Bug ||Blocker ||Open |Import custom > config on PYTHONPATH > AIRFLOW-1641 |Bug ||Blocker ||In Progress|Task gets stuck > in queued state > > PRs are out for them. After these three are merged, I'm planning to cut an > RC. > > Cheers, > Chris > > On Fri, Oct 20, 2017 at 7:49 AM, Bolke de Bruin <[email protected]> wrote: > >> I upgraded our production environment today. Some observations >> >> 1. Database migration (add max tries) didn’t workout of the box correctly >> for Postgres. “max_tries = False” didn’t work and needed to be casted to an >> int. Not sure how this gets to be false. This is a block imho >> 2. SSHExecuteOperator not having backwards compatibility sucks. We needed >> to rework quite a lot of dags >> 3. We use LdAP for logins. We havent configured the ‘superuser’ >> ‘data_profiler’ groups and thus should have automatic superuser privileges. >> We don’t at the moment (eg. we cannot manage connections or see the admin >> screens). >> >> After fixing #1, the rest seems quite normal. Happy with the new logging. >> >> Bolke >> >> >> > On 20 Oct 2017, at 12:16, Driesprong, Fokko <[email protected]> >> wrote: >> > >> > Hi Bolke, >> > >> > This one is a blocker: https://issues.apache.org/jira >> /browse/AIRFLOW-1731. >> > >> > This requirers that we are sure that the /plugin/ folder is on the path: >> > https://github.com/apache/incubator-airflow/blob/eb2f589099b >> 87743482c2eb16261b49e284dcd96/airflow/plugins_manager.py >> > >> > And maybe add an additional test and updating the docs a bit. Hopefully >> > somewhere in the next few days, however I'm a bit busy with the Spark >> > Summit. >> > >> > Cheers, Fokko >> > >> > 2017-10-19 21:03 GMT+02:00 Bolke de Bruin <[email protected]>: >> > >> >> Im planning to upgrade our production to 1.9.0alphaX tomorrow. >> >> >> >> What are the issues that are open? >> >> >> >> Cheers >> >> Bolke >> >> >> >>> On 9 Oct 2017, at 20:24, Chris Riccomini <[email protected]> >> wrote: >> >>> >> >>> K, I will plan on cutting an alpha1 later this week. Something for you >> >> guys to play with. >> >>> >> >>> On Mon, Oct 9, 2017 at 11:23 AM, Bolke de Bruin <[email protected] >> >> <mailto:[email protected]>> wrote: >> >>> hi Chris, >> >>> >> >>> We are still running pre alpha. I am a bit preoccupied with preparing >> >> for a conference and the team is readying a release of one of our core >> >> products. So it will probably will be after this week when I get my >> hands >> >> dirty again. >> >>> >> >>> Cheers >> >>> Bolke >> >>> >> >>> Verstuurd vanaf mijn iPad >> >>> >> >>>> Op 4 okt. 2017 om 23:35 heeft Chris Riccomini <[email protected] >> >> <mailto:[email protected]>> het volgende geschreven: >> >>>> >> >>>> Hey all, >> >>>> >> >>>> Checking in. Has anyone tested this in their environments? Stable >> >> releases >> >>>> depend on community involvement. >> >>>> >> >>>> Cheers, >> >>>> Chris >> >>>> >> >>>> On Tue, Oct 3, 2017 at 10:30 PM, Chris Riccomini < >> >> [email protected] <mailto:[email protected]>> >> >>>> wrote: >> >>>> >> >>>>> I've added AIRFLOW-1635 to the v1-9-test branch. It's not in alpha0, >> >> but >> >>>>> will be included in alpha1. >> >>>>> >> >>>>>> On Tue, Oct 3, 2017 at 4:13 PM, Feng Lu <[email protected] >> > >> >> wrote: >> >>>>>> >> >>>>>> Hi Chris, >> >>>>>> >> >>>>>> I know it's annoying to have last minute commit com in, but this >> is a >> >>>>>> highly desirable feature for folks using GCP operators, is it >> >> possible to >> >>>>>> include AIRFLOW-1635 >> >>>>>> <https://github.com/apache/incubator-airflow/commit/b3e985a3 < >> >> https://github.com/apache/incubator-airflow/commit/b3e985a3> >> >>>>>> 146272ecfd3ceaaa0d8567e4e9e117d4> >> >>>>>> in? >> >>>>>> More than happy to offer help if there's something I can do. >> >>>>>> Thanks a lot. >> >>>>>> >> >>>>>> Feng >> >>>>>> >> >>>>>> On Mon, Oct 2, 2017 at 3:24 PM, Chris Riccomini < >> >> [email protected] <mailto:[email protected]>> >> >>>>>> wrote: >> >>>>>> >> >>>>>>> Hey all, >> >>>>>>> >> >>>>>>> I have cut a 1.9.0alpha0 release of Airflow. You can download it >> >> here: >> >>>>>>> >> >>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1. >> >> 9.0alpha0/ <https://dist.apache.org/repos/dist/dev/incubator/ >> >> airflow/1.9.0alpha0/> >> >>>>>>> >> >>>>>>> The bin tarball can be installed with pip: >> >>>>>>> >> >>>>>>> pip install apache-airflow-1.9.0alpha0+incubating-bin.tar.gz >> >>>>>>> >> >>>>>>> The goal is to have the community install and run this to expose >> any >> >>>>>> bugs >> >>>>>>> before we move on to official release candidates. >> >>>>>>> >> >>>>>>> Here are the outstanding blocker bugs for 1.9.0: >> >>>>>>> >> >>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue >> >>>>>>> AIRFLOW-1258 |Bug |TaskInstances within SubDagOperator are >> >>>>>> marked >> >>>>>>> as >> >>>>>>> AIRFLOW-1055 |Bug |airflow/jobs.py:create_dag_run() >> >> exception >> >>>>>>> for >> >>>>>>> @on >> >>>>>>> AIRFLOW-1018 |Bug |Scheduler DAG processes can not log to >> >>>>>> stdout >> >>>>>>> AIRFLOW-1013 |Bug |airflow/jobs.py:manage_slas() >> exception >> >>>>>> for >> >>>>>>> @once >> >>>>>>> AIRFLOW-976 |Bug |Mark success running task causes it to >> >> fail >> >>>>>>> >> >>>>>>> Cheers, >> >>>>>>> Chris >> >>>>>>> >> >>>>>>> On Fri, Sep 29, 2017 at 3:54 PM, Chris Riccomini < >> >> [email protected] <mailto:[email protected]> >> >>>>>>> >> >>>>>>> wrote: >> >>>>>>> >> >>>>>>>> Welp. Work got in the way, so I'll cut the beta on Monday. :) >> >>>>>>>> >> >>>>>>>> On Thu, Sep 28, 2017 at 1:32 PM, Chris Riccomini < >> >>>>>> [email protected] <mailto:[email protected]>> >> >>>>>>>> wrote: >> >>>>>>>> >> >>>>>>>>> Works for me. Will try and cut a beta before end of week. >> >>>>>>>>> >> >>>>>>>>> Blockers for 1.9.0 are: >> >>>>>>>>> >> >>>>>>>>> AIRFLOW-1611 |Bug |Customize logging in Airflow >> >>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue >> >>>>>>>>> AIRFLOW-1258 |Bug |TaskInstances within SubDagOperator >> are >> >>>>>>> marked >> >>>>>>>>> as >> >>>>>>>>> AIRFLOW-976 |Bug |Mark success running task causes it >> to >> >>>>>> fail >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> On Thu, Sep 28, 2017 at 1:09 PM, Bolke de Bruin < >> >> [email protected] <mailto:[email protected]>> >> >>>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>>> Hi Chris >> >>>>>>>>>> >> >>>>>>>>>> Can I suggest releasing a beta? The stable branch is only cut >> at >> >> RC >> >>>>>>>>>> time. Betas allow us a broader exposure. It also gives us a >> >> point of >> >>>>>>>>>> reference. >> >>>>>>>>>> >> >>>>>>>>>> In addition the list below are mostly longer standing issues >> that >> >>>>>> are >> >>>>>>>>>> also part of the 1.8.x branch. Maybe only consider 1611, 1525, >> >> 1258, >> >>>>>>> and >> >>>>>>>>>> 976 as blocker? >> >>>>>>>>>> >> >>>>>>>>>> Cheers >> >>>>>>>>>> Bolke >> >>>>>>>>>> >> >>>>>>>>>> Verstuurd vanaf mijn iPad >> >>>>>>>>>> >> >>>>>>>>>>> Op 28 sep. 2017 om 19:49 heeft Chris Riccomini < >> >>>>>>> [email protected] <mailto:[email protected]>> >> >>>>>>>>>> het volgende geschreven: >> >>>>>>>>>>> >> >>>>>>>>>>> Hey all, >> >>>>>>>>>>> >> >>>>>>>>>>> I was planning to cut a 1.9.0 stable branch and 1.9.0 beta >> >>>>>> release, >> >>>>>>> but >> >>>>>>>>>>> seeing as there are several outstanding bugs, I'm going to >> >> delay. >> >>>>>>> Here >> >>>>>>>>>> are >> >>>>>>>>>>> the bugs that I'm tracking: >> >>>>>>>>>>> >> >>>>>>>>>>> AIRFLOW-1611 |Bug |Customize logging in Airflow >> >>>>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue >> >>>>>>>>>>> AIRFLOW-1258 |Bug |TaskInstances within SubDagOperator >> >> are >> >>>>>>>>>> marked as >> >>>>>>>>>>> AIRFLOW-1055 |Bug |airflow/jobs.py:create_dag_run() >> >>>>>>> exception >> >>>>>>>>>> for >> >>>>>>>>>>> @on >> >>>>>>>>>>> AIRFLOW-1018 |Bug |Scheduler DAG processes can not log >> >> to >> >>>>>>>>>> stdout >> >>>>>>>>>>> AIRFLOW-1013 |Bug |airflow/jobs.py:manage_slas() >> >>>>>> exception >> >>>>>>>>>> for @once >> >>>>>>>>>>> AIRFLOW-988 |Bug |SLA Miss Callbacks Are Repeated if >> >>>>>> Email >> >>>>>>> is >> >>>>>>>>>> Not >> >>>>>>>>>>> be >> >>>>>>>>>>> AIRFLOW-976 |Bug |Mark success running task causes it >> >> to >> >>>>>>> fail >> >>>>>>>>>>> >> >>>>>>>>>>> These are the priority issues. Once they're merged, I'll cut >> the >> >>>>>>>>>>> v1-9-stable and beta release. >> >>>>>>>>>>> >> >>>>>>>>>>> If you can help clean this up, that would be really >> appreciated. >> >>>>>>>>>>> >> >>>>>>>>>>> Cheers, >> >>>>>>>>>>> Chris >> >>>>>>>>>>> >> >>>>>>>>>>> On Thu, Sep 28, 2017 at 10:06 AM, Chris Riccomini < >> >>>>>>>>>> [email protected] <mailto:[email protected]>> >> >>>>>>>>>>> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>>> Marked it for 1.9.0. >> >>>>>>>>>>>> >> >>>>>>>>>>>>> On Thu, Sep 28, 2017 at 9:56 AM, Charlie Jones < >> >>>>>> [email protected] <mailto:[email protected]>> >> >>>>>>>>>> wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Is there any chance we could include AIRFLOW-988 in 1.9.0? >> SLA >> >>>>>>>>>> callbacks >> >>>>>>>>>>>>> are not working correctly without emails... Its not a major >> >> bug, >> >>>>>>> but >> >>>>>>>>>> it >> >>>>>>>>>>>>> does cause us some annoyance in our current deployment. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Link to Jira: >> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-988 < >> >> https://issues.apache.org/jira/browse/AIRFLOW-988> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Link to PR: >> >>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2415 < >> >> https://github.com/apache/incubator-airflow/pull/2415> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Thanks! >> >>>>>>>>>>>>> Charlie Jones >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> CHARLIE JONES >> >>>>>>>>>>>>> Data Engineer >> >>>>>>>>>>>>> [email protected] <mailto:[email protected]> | M: >> >> 972.821.7631 <tel:972.821.7631> >> >>>>>>>>>>>>> __________________________________________________ >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Programmatic Performance.* Localized.* >> >>>>>>>>>>>>> __________________________________________________ >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> 1407 Texas Street | Suite 202 | Fort Worth, TX 76102 >> >>>>>>>>>>>>> 800.840.0768 <tel:800.840.0768> | www.simpli.fi < >> >> http://www.simpli.fi/> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> On Wed, Sep 27, 2017 at 12:11 PM, Chris Riccomini < >> >>>>>>>>>> [email protected] <mailto:[email protected]>> >> >>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>>> Merged. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:25 PM, Ryan Buckley < >> >>>>>>>>>>>>> [email protected] <mailto:[email protected] >> >> >> >>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Would it be possible to include AIRFLOW-1587? >> >>>>>>>>>>>>>>> Running dags from the UI is currently broken on the 1.9.0 >> >>>>>> branch >> >>>>>>>>>> due >> >>>>>>>>>>>>> to >> >>>>>>>>>>>>>>> this issue. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2590 < >> >> https://github.com/apache/incubator-airflow/pull/2590> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Thanks, >> >>>>>>>>>>>>>>> Ryan >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 8:14 AM, Driesprong, Fokko >> >>>>>>>>>>>>> <[email protected] >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Hi All, >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> I would like to include AIRFLOW-1611 in the 1.9.0 >> release: >> >>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2631 < >> >> https://github.com/apache/incubator-airflow/pull/2631> >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Currently importing a custom logging configuration is not >> >>>>>> work >> >>>>>>> (as >> >>>>>>>>>>>>> far >> >>>>>>>>>>>>>>> as I >> >>>>>>>>>>>>>>>> know). Any feedback on the PR would also be appreciated. >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Cheers, Fokko >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> 2017-09-25 23:27 GMT+02:00 Chris Riccomini < >> >>>>>>> [email protected] <mailto:[email protected]> >> >>>>>>>>>>> : >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> Done! >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> On Mon, Sep 25, 2017 at 5:11 AM, Michael Crawford < >> >>>>>>>>>>>>>>>>> [email protected] <mailto: >> >> [email protected]>> wrote: >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> Can you slide the aws and emr connection type fix in? >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636 < >> >> https://issues.apache.org/jira/browse/AIRFLOW-1636> < >> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636 < >> >> https://issues.apache.org/jira/browse/AIRFLOW-1636>> >> >>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2626 >> < >> >> https://github.com/apache/incubator-airflow/pull/2626> < >> >>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2626 >> < >> >> https://github.com/apache/incubator-airflow/pull/2626>> >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> It keeps the connection type from getting blanked out >> on >> >>>>>> edit >> >>>>>>>>>>>>> for >> >>>>>>>>>>>>>>> these >> >>>>>>>>>>>>>>>>>> types. >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> Thanks, >> >>>>>>>>>>>>>>>>>> Mike >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>> On Sep 21, 2017, at 1:27 PM, Chris Riccomini < >> >>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>> >> >>>>>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>> Absolutely. Just cherry-picked. I've been looking >> >> forward >> >>>>>> to >> >>>>>>>>>>>>>> these >> >>>>>>>>>>>>>>>>> fixes! >> >>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 5:23 PM, Alex Guziel < >> >>>>>>>>>>>>>>> [email protected] <mailto:[email protected]> >> >>>>>>>>>>>>>>>> . >> >>>>>>>>>>>>>>>>>> invalid >> >>>>>>>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>> Can we get this in? >> >>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1519 < >> >> https://issues.apache.org/jira/browse/AIRFLOW-1519> >> >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1621 < >> >> https://issues.apache.org/jira/browse/AIRFLOW-1621> >> >>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/commit/ >> < >> >> https://github.com/apache/incubator-airflow/commit/> >> >>>>>>>>>>>>>>>>>>>> b6d2e0a46978e93e16576604624f57d1388814f2 >> >>>>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/commit/ >> < >> >> https://github.com/apache/incubator-airflow/commit/> >> >>>>>>>>>>>>>>>>>>>> 656d045e90bf67ca484a3778b2a07a419bfb324a >> >>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>> It speeds up loading times a lot, so it's a good >> thing >> >> to >> >>>>>>>>>>>>> have >> >>>>>>>>>>>>>> in >> >>>>>>>>>>>>>>>> 1.9. >> >>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 11:14 AM, Chris Riccomini < >> >>>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>> >> >>>>>>>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>> Sounds good. I'll plan on stable+beta next week, >> then. >> >>>>>>>>>>>>> Initial >> >>>>>>>>>>>>>>>>> warning >> >>>>>>>>>>>>>>>>>>>>> stands, that I will start locking down what can get >> >> into >> >>>>>>>>>>>>> 1.9.0 >> >>>>>>>>>>>>>> at >> >>>>>>>>>>>>>>>>> that >> >>>>>>>>>>>>>>>>>>>>> point. >> >>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 11:10 AM, Bolke de Bruin < >> >>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>> >> >>>>>>>>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>> No vote indeed, just to gather feedback on a >> >> particular >> >>>>>>>>>>>>> fixed >> >>>>>>>>>>>>>>>> point >> >>>>>>>>>>>>>>>>> in >> >>>>>>>>>>>>>>>>>>>>>> time. It also gives a bit more trust to a tarball >> >> than >> >>>>>> to >> >>>>>>> a >> >>>>>>>>>>>>>> git >> >>>>>>>>>>>>>>>>> pull. >> >>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>> Bolke >> >>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>> On 20 Sep 2017, at 20:09, Chris Riccomini < >> >>>>>>>>>>>>>>> [email protected] <mailto:[email protected]> >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>> I can do a beta. Is the process significantly >> >>>>>> different? >> >>>>>>>>>>>>>> IIRC, >> >>>>>>>>>>>>>>>> it's >> >>>>>>>>>>>>>>>>>>>>>>> basically the same, just no vote, right? >> >>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 10:56 AM, Bolke de Bruin < >> >>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>> >> >>>>>>>>>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>> Are you sure you want to go ahead and do RCs >> right >> >>>>>> away? >> >>>>>>>>>>>>>>> Isn’t a >> >>>>>>>>>>>>>>>>>>>> beta >> >>>>>>>>>>>>>>>>>>>>> a >> >>>>>>>>>>>>>>>>>>>>>>>> bit smarter? >> >>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>> - Bolke >> >>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>> On 20 Sep 2017, at 19:41, Chris Riccomini < >> >>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]> >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>> Hey all, >> >>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>> I want to send out a warning that I'm planning >> to >> >>>>>> cut >> >>>>>>>>>>>>> the >> >>>>>>>>>>>>>>>> stable >> >>>>>>>>>>>>>>>>>>>>> branch >> >>>>>>>>>>>>>>>>>>>>>>>>> next week, and begin the RC1 release vote. Once >> >> the >> >>>>>>>>>>>>> stable >> >>>>>>>>>>>>>>>> branch >> >>>>>>>>>>>>>>>>>>>> is >> >>>>>>>>>>>>>>>>>>>>>>>> cut, I >> >>>>>>>>>>>>>>>>>>>>>>>>> will be locking down what commits get cherry >> >> picked >> >>>>>>> into >> >>>>>>>>>>>>>> the >> >>>>>>>>>>>>>>>>>>>> branch, >> >>>>>>>>>>>>>>>>>>>>>> and >> >>>>>>>>>>>>>>>>>>>>>>>>> will only be doing PRs that are required to get >> >> the >> >>>>>>>>>>>>> release >> >>>>>>>>>>>>>>>> out. >> >>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >> >>>>>>>>>>>>>>>>>>>>>>>>> Chris >> >>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Sep 18, 2017 at 11:19 AM, Chris >> Riccomini >> >> < >> >>>>>>>>>>>>>>>>>>>>>> [email protected] <mailto: >> [email protected]> >> >>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>>> Hey all, >> >>>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>>> An update on the 1.9.0 release. Here are the >> >>>>>>>>>>>>> outstanding >> >>>>>>>>>>>>>> PRs >> >>>>>>>>>>>>>>>>> that >> >>>>>>>>>>>>>>>>>>>>> are >> >>>>>>>>>>>>>>>>>>>>>>>>>> slated to be included into 1.9.0: >> >>>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>>> ISSUE ID |STATUS |DESCRIPTION >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1617 |Open |XSS Vulnerability in >> >>>>>> Variable >> >>>>>>>>>>>>>>>> endpoint >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1611 |Open |Customize logging in >> >>>>>> Airflow >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1605 |Reopened |Fix log source of >> local >> >>>>>>>>>>>>> loggers >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1604 |Open |Rename the logger to >> log >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1525 |Open |Fix minor LICENSE & >> >> NOTICE >> >>>>>>>>>>>>> issue >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1499 |In Progres|Eliminate duplicate >> and >> >>>>>>>>>>>>> unneeded >> >>>>>>>>>>>>>>> code >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1198 |Open |HDFSOperator to >> operate >> >>>>>> HDFS >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1055 |Open >> >>>>>> |airflow/jobs.py:create_dag_ru >> >>>>>>>>>>>>> n() >> >>>>>>>>>>>>>>>>>>>> exception >> >>>>>>>>>>>>>>>>>>>>>> for >> >>>>>>>>>>>>>>>>>>>>>>>>>> @on >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1019 |Open |active_dagruns >> shouldn't >> >>>>>>>>>>>>> include >> >>>>>>>>>>>>>>>> paused >> >>>>>>>>>>>>>>>>>>>>> DAGs >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1018 |Open |Scheduler DAG >> processes >> >>>>>> can >> >>>>>>>>>>>>> not >> >>>>>>>>>>>>>> log >> >>>>>>>>>>>>>>>> to >> >>>>>>>>>>>>>>>>>>>>> stdout >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1015 |Open |TreeView displayed >> over >> >>>>>> task >> >>>>>>>>>>>>>>>> instances >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1013 |Open >> >>>>>> |airflow/jobs.py:manage_slas() >> >>>>>>>>>>>>>>>> exception >> >>>>>>>>>>>>>>>>>>>> for >> >>>>>>>>>>>>>>>>>>>>>>>>>> @once >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-976 |Open |Mark success running >> >> task >> >>>>>>>>>>>>> causes >> >>>>>>>>>>>>>> it >> >>>>>>>>>>>>>>>> to >> >>>>>>>>>>>>>>>>>>>> fail >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-914 |Open |Refactor >> >>>>>>>>>>>>>>>> BackfillJobTest.test_backfill_ >> >>>>>>>>>>>>>>>>>>>>>>>> examples >> >>>>>>>>>>>>>>>>>>>>>>>>>> to >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-913 |Open |Refactor >> >>>>>>>>>>>>>>>> tests.CoreTest.test_scheduler_ >> >>>>>>>>>>>>>>>>>>>> job >> >>>>>>>>>>>>>>>>>>>>>> to >> >>>>>>>>>>>>>>>>>>>>>>>>>> real >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-912 |Open |Refactor tests and >> build >> >>>>>>>>>>>>> matrix >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-888 |Open |Operators should not >> >> push >> >>>>>>>>>>>>> XComs >> >>>>>>>>>>>>>> by >> >>>>>>>>>>>>>>>>>>>> default >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-828 |Open |Add maximum size for >> >> XComs >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-825 |Open |Add Dataflow semantics >> >>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-788 |Open |Context unexpectedly >> >>>>>> added to >> >>>>>>>>>>>>>> hive >> >>>>>>>>>>>>>>>> conf >> >>>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>>> I will be locking down what can get >> cherry-picked >> >>>>>> into >> >>>>>>>>>>>>> the >> >>>>>>>>>>>>>>>> 1.9.0 >> >>>>>>>>>>>>>>>>>>>>>> branch >> >>>>>>>>>>>>>>>>>>>>>>>>>> shortly, so if you have something you want in, >> >>>>>> please >> >>>>>>>>>>>>> set >> >>>>>>>>>>>>>>> the >> >>>>>>>>>>>>>>>>> fix >> >>>>>>>>>>>>>>>>>>>>>>>> version >> >>>>>>>>>>>>>>>>>>>>>>>>>> to 1.9.0. >> >>>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>>> We (at WePay) have deployed 1.9.0 into our dev >> >>>>>>> cluster, >> >>>>>>>>>>>>>> and >> >>>>>>>>>>>>>>> it >> >>>>>>>>>>>>>>>>> has >> >>>>>>>>>>>>>>>>>>>>>> been >> >>>>>>>>>>>>>>>>>>>>>>>>>> running smoothly for several days. >> >>>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>>> ** I could really use help verifying stability. >> >> If >> >>>>>> you >> >>>>>>>>>>>>> run >> >>>>>>>>>>>>>>>>>>>> Airflow, >> >>>>>>>>>>>>>>>>>>>>>> it's >> >>>>>>>>>>>>>>>>>>>>>>>>>> in your best interest to deploy the 1.9.0 test >> >>>>>> branch >> >>>>>>>>>>>>>>>> somewhere, >> >>>>>>>>>>>>>>>>>>>> and >> >>>>>>>>>>>>>>>>>>>>>>>> verify >> >>>>>>>>>>>>>>>>>>>>>>>>>> it's working for your workload. ** >> >>>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >> >>>>>>>>>>>>>>>>>>>>>>>>>> Chris >> >>>>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>>>> >> >>> >> >> >> >> >> >> >
