Is this bug issue on DAG scheduling <https://issues.apache.org/jira/browse/AIRFLOW-1767> something that needs resolution before the RC?
On Tue, Oct 31, 2017 at 1:17 PM, Chris Riccomini <[email protected]> wrote: > Both blockers have been merged. Are we ready for an RC? > > On Mon, Oct 30, 2017 at 11:55 AM, Bolke de Bruin <[email protected]> > wrote: > > > AIRFLOW-1764 is design/integration mistake (maybe worth a CVE). But 1765 > > is not a vulnerability, although *maybe* unexpected behaviour. > > > > Bolke. > > > > > > > On 30 Oct 2017, at 19:51, Chris Riccomini <[email protected]> > wrote: > > > > > > I marked them as blocker. Still would like to understand how important > > this > > > is, though. The API is experimental, so I could see an argument for > them > > > not being blocker provided the vulnerability is limited to experimental > > > areas. > > > > > > On Mon, Oct 30, 2017 at 11:06 AM, Ash Berlin-Taylor < > [email protected]> > > > wrote: > > > > > >> I think we need to add > > >> > > >> https://issues.apache.org/jira/browse/AIRFLOW-1764 > > >> https://issues.apache.org/jira/browse/AIRFLOW-1765 > > >> > > >> to that list as critical security fixes. I'll tackled 1765 tomorrow > (UK > > >> time) if no one gets round to it before then. > > >> > > >> -ash > > >> > > >>> On 30 Oct 2017, at 17:58, Chris Riccomini <[email protected]> > > wrote: > > >>> > > >>> Hey all, > > >>> > > >>> Current blockers for 1.9.0: > > >>> > > >>> https://issues.apache.org/jira/browse/AIRFLOW-1711 > > >>> https://issues.apache.org/jira/browse/AIRFLOW-1018 > > >>> > > >>> Cheers, > > >>> Chris > > >>> > > >>> On Tue, Oct 24, 2017 at 11:06 AM, Chris Riccomini < > > [email protected] > > >>> > > >>> wrote: > > >>> > > >>>> Hey all, > > >>>> > > >>>> Talked with @Bolke on Gitter. Here are the currently blocking issues > > for > > >>>> 1.9.0: > > >>>> > > >>>> AIRFLOW-1744 |Bug ||Blocker ||Open |task.retries > > can > > >>>> be False > > >>>> AIRFLOW-1731 |Bug ||Blocker ||Open |Import > custom > > >>>> config on PYTHONPATH > > >>>> AIRFLOW-1641 |Bug ||Blocker ||In Progress|Task gets > > stuck > > >>>> in queued state > > >>>> > > >>>> PRs are out for them. After these three are merged, I'm planning to > > cut > > >> an > > >>>> RC. > > >>>> > > >>>> Cheers, > > >>>> Chris > > >>>> > > >>>> On Fri, Oct 20, 2017 at 7:49 AM, Bolke de Bruin <[email protected]> > > >> wrote: > > >>>> > > >>>>> I upgraded our production environment today. Some observations > > >>>>> > > >>>>> 1. Database migration (add max tries) didn’t workout of the box > > >> correctly > > >>>>> for Postgres. “max_tries = False” didn’t work and needed to be > casted > > >> to an > > >>>>> int. Not sure how this gets to be false. This is a block imho > > >>>>> 2. SSHExecuteOperator not having backwards compatibility sucks. We > > >> needed > > >>>>> to rework quite a lot of dags > > >>>>> 3. We use LdAP for logins. We havent configured the ‘superuser’ > > >>>>> ‘data_profiler’ groups and thus should have automatic superuser > > >> privileges. > > >>>>> We don’t at the moment (eg. we cannot manage connections or see the > > >> admin > > >>>>> screens). > > >>>>> > > >>>>> After fixing #1, the rest seems quite normal. Happy with the new > > >> logging. > > >>>>> > > >>>>> Bolke > > >>>>> > > >>>>> > > >>>>>> On 20 Oct 2017, at 12:16, Driesprong, Fokko <[email protected] > > > > >>>>> wrote: > > >>>>>> > > >>>>>> Hi Bolke, > > >>>>>> > > >>>>>> This one is a blocker: https://issues.apache.org/jira > > >>>>> /browse/AIRFLOW-1731. > > >>>>>> > > >>>>>> This requirers that we are sure that the /plugin/ folder is on the > > >> path: > > >>>>>> https://github.com/apache/incubator-airflow/blob/eb2f589099b > > >>>>> 87743482c2eb16261b49e284dcd96/airflow/plugins_manager.py > > >>>>>> > > >>>>>> And maybe add an additional test and updating the docs a bit. > > >> Hopefully > > >>>>>> somewhere in the next few days, however I'm a bit busy with the > > Spark > > >>>>>> Summit. > > >>>>>> > > >>>>>> Cheers, Fokko > > >>>>>> > > >>>>>> 2017-10-19 21:03 GMT+02:00 Bolke de Bruin <[email protected]>: > > >>>>>> > > >>>>>>> Im planning to upgrade our production to 1.9.0alphaX tomorrow. > > >>>>>>> > > >>>>>>> What are the issues that are open? > > >>>>>>> > > >>>>>>> Cheers > > >>>>>>> Bolke > > >>>>>>> > > >>>>>>>> On 9 Oct 2017, at 20:24, Chris Riccomini <[email protected] > > > > >>>>> wrote: > > >>>>>>>> > > >>>>>>>> K, I will plan on cutting an alpha1 later this week. Something > for > > >> you > > >>>>>>> guys to play with. > > >>>>>>>> > > >>>>>>>> On Mon, Oct 9, 2017 at 11:23 AM, Bolke de Bruin < > > [email protected] > > >>>>>>> <mailto:[email protected]>> wrote: > > >>>>>>>> hi Chris, > > >>>>>>>> > > >>>>>>>> We are still running pre alpha. I am a bit preoccupied with > > >> preparing > > >>>>>>> for a conference and the team is readying a release of one of our > > >> core > > >>>>>>> products. So it will probably will be after this week when I get > my > > >>>>> hands > > >>>>>>> dirty again. > > >>>>>>>> > > >>>>>>>> Cheers > > >>>>>>>> Bolke > > >>>>>>>> > > >>>>>>>> Verstuurd vanaf mijn iPad > > >>>>>>>> > > >>>>>>>>> Op 4 okt. 2017 om 23:35 heeft Chris Riccomini < > > >> [email protected] > > >>>>>>> <mailto:[email protected]>> het volgende geschreven: > > >>>>>>>>> > > >>>>>>>>> Hey all, > > >>>>>>>>> > > >>>>>>>>> Checking in. Has anyone tested this in their environments? > Stable > > >>>>>>> releases > > >>>>>>>>> depend on community involvement. > > >>>>>>>>> > > >>>>>>>>> Cheers, > > >>>>>>>>> Chris > > >>>>>>>>> > > >>>>>>>>> On Tue, Oct 3, 2017 at 10:30 PM, Chris Riccomini < > > >>>>>>> [email protected] <mailto:[email protected]>> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> I've added AIRFLOW-1635 to the v1-9-test branch. It's not in > > >> alpha0, > > >>>>>>> but > > >>>>>>>>>> will be included in alpha1. > > >>>>>>>>>> > > >>>>>>>>>>> On Tue, Oct 3, 2017 at 4:13 PM, Feng Lu > > >> <[email protected] > > >>>>>> > > >>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>> Hi Chris, > > >>>>>>>>>>> > > >>>>>>>>>>> I know it's annoying to have last minute commit com in, but > > this > > >>>>> is a > > >>>>>>>>>>> highly desirable feature for folks using GCP operators, is it > > >>>>>>> possible to > > >>>>>>>>>>> include AIRFLOW-1635 > > >>>>>>>>>>> <https://github.com/apache/incubator-airflow/commit/b3e985a3 > < > > >>>>>>> https://github.com/apache/incubator-airflow/commit/b3e985a3> > > >>>>>>>>>>> 146272ecfd3ceaaa0d8567e4e9e117d4> > > >>>>>>>>>>> in? > > >>>>>>>>>>> More than happy to offer help if there's something I can do. > > >>>>>>>>>>> Thanks a lot. > > >>>>>>>>>>> > > >>>>>>>>>>> Feng > > >>>>>>>>>>> > > >>>>>>>>>>> On Mon, Oct 2, 2017 at 3:24 PM, Chris Riccomini < > > >>>>>>> [email protected] <mailto:[email protected]>> > > >>>>>>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>>> Hey all, > > >>>>>>>>>>>> > > >>>>>>>>>>>> I have cut a 1.9.0alpha0 release of Airflow. You can > download > > it > > >>>>>>> here: > > >>>>>>>>>>>> > > >>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1. > > >>>>>>> 9.0alpha0/ <https://dist.apache.org/repos/dist/dev/incubator/ > > >>>>>>> airflow/1.9.0alpha0/> > > >>>>>>>>>>>> > > >>>>>>>>>>>> The bin tarball can be installed with pip: > > >>>>>>>>>>>> > > >>>>>>>>>>>> pip install apache-airflow-1.9.0alpha0+ > incubating-bin.tar.gz > > >>>>>>>>>>>> > > >>>>>>>>>>>> The goal is to have the community install and run this to > > expose > > >>>>> any > > >>>>>>>>>>> bugs > > >>>>>>>>>>>> before we move on to official release candidates. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Here are the outstanding blocker bugs for 1.9.0: > > >>>>>>>>>>>> > > >>>>>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue > > >>>>>>>>>>>> AIRFLOW-1258 |Bug |TaskInstances within > SubDagOperator > > >> are > > >>>>>>>>>>> marked > > >>>>>>>>>>>> as > > >>>>>>>>>>>> AIRFLOW-1055 |Bug |airflow/jobs.py:create_dag_run() > > >>>>>>> exception > > >>>>>>>>>>>> for > > >>>>>>>>>>>> @on > > >>>>>>>>>>>> AIRFLOW-1018 |Bug |Scheduler DAG processes can not > log > > >> to > > >>>>>>>>>>> stdout > > >>>>>>>>>>>> AIRFLOW-1013 |Bug |airflow/jobs.py:manage_slas() > > >>>>> exception > > >>>>>>>>>>> for > > >>>>>>>>>>>> @once > > >>>>>>>>>>>> AIRFLOW-976 |Bug |Mark success running task causes > it > > >> to > > >>>>>>> fail > > >>>>>>>>>>>> > > >>>>>>>>>>>> Cheers, > > >>>>>>>>>>>> Chris > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Fri, Sep 29, 2017 at 3:54 PM, Chris Riccomini < > > >>>>>>> [email protected] <mailto:[email protected]> > > >>>>>>>>>>>> > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>>> Welp. Work got in the way, so I'll cut the beta on Monday. > :) > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> On Thu, Sep 28, 2017 at 1:32 PM, Chris Riccomini < > > >>>>>>>>>>> [email protected] <mailto:[email protected]>> > > >>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> Works for me. Will try and cut a beta before end of week. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Blockers for 1.9.0 are: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> AIRFLOW-1611 |Bug |Customize logging in Airflow > > >>>>>>>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE > issue > > >>>>>>>>>>>>>> AIRFLOW-1258 |Bug |TaskInstances within > > SubDagOperator > > >>>>> are > > >>>>>>>>>>>> marked > > >>>>>>>>>>>>>> as > > >>>>>>>>>>>>>> AIRFLOW-976 |Bug |Mark success running task > causes > > it > > >>>>> to > > >>>>>>>>>>> fail > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> On Thu, Sep 28, 2017 at 1:09 PM, Bolke de Bruin < > > >>>>>>> [email protected] <mailto:[email protected]>> > > >>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Hi Chris > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Can I suggest releasing a beta? The stable branch is only > > cut > > >>>>> at > > >>>>>>> RC > > >>>>>>>>>>>>>>> time. Betas allow us a broader exposure. It also gives > us a > > >>>>>>> point of > > >>>>>>>>>>>>>>> reference. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> In addition the list below are mostly longer standing > > issues > > >>>>> that > > >>>>>>>>>>> are > > >>>>>>>>>>>>>>> also part of the 1.8.x branch. Maybe only consider 1611, > > >> 1525, > > >>>>>>> 1258, > > >>>>>>>>>>>> and > > >>>>>>>>>>>>>>> 976 as blocker? > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Cheers > > >>>>>>>>>>>>>>> Bolke > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Verstuurd vanaf mijn iPad > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Op 28 sep. 2017 om 19:49 heeft Chris Riccomini < > > >>>>>>>>>>>> [email protected] <mailto:[email protected]>> > > >>>>>>>>>>>>>>> het volgende geschreven: > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Hey all, > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> I was planning to cut a 1.9.0 stable branch and 1.9.0 > beta > > >>>>>>>>>>> release, > > >>>>>>>>>>>> but > > >>>>>>>>>>>>>>>> seeing as there are several outstanding bugs, I'm going > to > > >>>>>>> delay. > > >>>>>>>>>>>> Here > > >>>>>>>>>>>>>>> are > > >>>>>>>>>>>>>>>> the bugs that I'm tracking: > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> AIRFLOW-1611 |Bug |Customize logging in Airflow > > >>>>>>>>>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE > > issue > > >>>>>>>>>>>>>>>> AIRFLOW-1258 |Bug |TaskInstances within > > >> SubDagOperator > > >>>>>>> are > > >>>>>>>>>>>>>>> marked as > > >>>>>>>>>>>>>>>> AIRFLOW-1055 |Bug |airflow/jobs.py:create_dag_ > > run() > > >>>>>>>>>>>> exception > > >>>>>>>>>>>>>>> for > > >>>>>>>>>>>>>>>> @on > > >>>>>>>>>>>>>>>> AIRFLOW-1018 |Bug |Scheduler DAG processes can > not > > >> log > > >>>>>>> to > > >>>>>>>>>>>>>>> stdout > > >>>>>>>>>>>>>>>> AIRFLOW-1013 |Bug |airflow/jobs.py:manage_slas() > > >>>>>>>>>>> exception > > >>>>>>>>>>>>>>> for @once > > >>>>>>>>>>>>>>>> AIRFLOW-988 |Bug |SLA Miss Callbacks Are > Repeated > > >> if > > >>>>>>>>>>> Email > > >>>>>>>>>>>> is > > >>>>>>>>>>>>>>> Not > > >>>>>>>>>>>>>>>> be > > >>>>>>>>>>>>>>>> AIRFLOW-976 |Bug |Mark success running task > > causes > > >> it > > >>>>>>> to > > >>>>>>>>>>>> fail > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> These are the priority issues. Once they're merged, I'll > > cut > > >>>>> the > > >>>>>>>>>>>>>>>> v1-9-stable and beta release. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> If you can help clean this up, that would be really > > >>>>> appreciated. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Cheers, > > >>>>>>>>>>>>>>>> Chris > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> On Thu, Sep 28, 2017 at 10:06 AM, Chris Riccomini < > > >>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>> > > >>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> Marked it for 1.9.0. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> On Thu, Sep 28, 2017 at 9:56 AM, Charlie Jones < > > >>>>>>>>>>> [email protected] <mailto:[email protected]>> > > >>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Is there any chance we could include AIRFLOW-988 in > > 1.9.0? > > >>>>> SLA > > >>>>>>>>>>>>>>> callbacks > > >>>>>>>>>>>>>>>>>> are not working correctly without emails... Its not a > > >> major > > >>>>>>> bug, > > >>>>>>>>>>>> but > > >>>>>>>>>>>>>>> it > > >>>>>>>>>>>>>>>>>> does cause us some annoyance in our current > deployment. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Link to Jira: > > >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-988 < > > >>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-988> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Link to PR: > > >>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2415 > < > > >>>>>>> https://github.com/apache/incubator-airflow/pull/2415> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Thanks! > > >>>>>>>>>>>>>>>>>> Charlie Jones > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> CHARLIE JONES > > >>>>>>>>>>>>>>>>>> Data Engineer > > >>>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]> | M: > > >>>>>>> 972.821.7631 <tel:972.821.7631> > > >>>>>>>>>>>>>>>>>> __________________________________________________ > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Programmatic Performance.* Localized.* > > >>>>>>>>>>>>>>>>>> __________________________________________________ > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> 1407 Texas Street | Suite 202 | Fort Worth, TX > 76102 > > >>>>>>>>>>>>>>>>>> 800.840.0768 <tel:800.840.0768> | www.simpli.fi < > > >>>>>>> http://www.simpli.fi/> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> On Wed, Sep 27, 2017 at 12:11 PM, Chris Riccomini < > > >>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>> > > >>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> Merged. > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:25 PM, Ryan Buckley < > > >>>>>>>>>>>>>>>>>> [email protected] <mailto: > ryan.buckley@bluecore > > . > > >> com > > >>>>>>> > > >>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> Would it be possible to include AIRFLOW-1587? > > >>>>>>>>>>>>>>>>>>>> Running dags from the UI is currently broken on the > > >> 1.9.0 > > >>>>>>>>>>> branch > > >>>>>>>>>>>>>>> due > > >>>>>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>>>> this issue. > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> https://github.com/apache/ > incubator-airflow/pull/2590 > > < > > >>>>>>> https://github.com/apache/incubator-airflow/pull/2590> > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> Thanks, > > >>>>>>>>>>>>>>>>>>>> Ryan > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 8:14 AM, Driesprong, Fokko > > >>>>>>>>>>>>>>>>>> <[email protected] > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> Hi All, > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> I would like to include AIRFLOW-1611 in the 1.9.0 > > >>>>> release: > > >>>>>>>>>>>>>>>>>>>>> https://github.com/apache/ > > incubator-airflow/pull/2631 > > >> < > > >>>>>>> https://github.com/apache/incubator-airflow/pull/2631> > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> Currently importing a custom logging configuration > is > > >> not > > >>>>>>>>>>> work > > >>>>>>>>>>>> (as > > >>>>>>>>>>>>>>>>>> far > > >>>>>>>>>>>>>>>>>>>> as I > > >>>>>>>>>>>>>>>>>>>>> know). Any feedback on the PR would also be > > >> appreciated. > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> Cheers, Fokko > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> 2017-09-25 23:27 GMT+02:00 Chris Riccomini < > > >>>>>>>>>>>> [email protected] <mailto:[email protected]> > > >>>>>>>>>>>>>>>> : > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> Done! > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> On Mon, Sep 25, 2017 at 5:11 AM, Michael Crawford > < > > >>>>>>>>>>>>>>>>>>>>>> [email protected] <mailto: > > >>>>>>> [email protected]>> wrote: > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> Can you slide the aws and emr connection type fix > > in? > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/ > jira/browse/AIRFLOW-1636 > > < > > >>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636> < > > >>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/ > jira/browse/AIRFLOW-1636 > > < > > >>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636>> > > >>>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/ > > >> incubator-airflow/pull/2626 > > >>>>> < > > >>>>>>> https://github.com/apache/incubator-airflow/pull/2626> < > > >>>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/ > > >> incubator-airflow/pull/2626 > > >>>>> < > > >>>>>>> https://github.com/apache/incubator-airflow/pull/2626>> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> It keeps the connection type from getting blanked > > out > > >>>>> on > > >>>>>>>>>>> edit > > >>>>>>>>>>>>>>>>>> for > > >>>>>>>>>>>>>>>>>>>> these > > >>>>>>>>>>>>>>>>>>>>>>> types. > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> Thanks, > > >>>>>>>>>>>>>>>>>>>>>>> Mike > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>> On Sep 21, 2017, at 1:27 PM, Chris Riccomini < > > >>>>>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected] > >> > > >>>>>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>> Absolutely. Just cherry-picked. I've been > looking > > >>>>>>> forward > > >>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>>> these > > >>>>>>>>>>>>>>>>>>>>>> fixes! > > >>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 5:23 PM, Alex Guziel < > > >>>>>>>>>>>>>>>>>>>> [email protected] <mailto: > [email protected] > > > > > >>>>>>>>>>>>>>>>>>>>> . > > >>>>>>>>>>>>>>>>>>>>>>> invalid > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Can we get this in? > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/ > > jira/browse/AIRFLOW-1519 > > >> < > > >>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1519> > > >>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/ > > jira/browse/AIRFLOW-1621 > > >> < > > >>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1621> > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/ > > >> incubator-airflow/commit/ > > >>>>> < > > >>>>>>> https://github.com/apache/incubator-airflow/commit/> > > >>>>>>>>>>>>>>>>>>>>>>>>> b6d2e0a46978e93e16576604624f57d1388814f2 > > >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/ > > >> incubator-airflow/commit/ > > >>>>> < > > >>>>>>> https://github.com/apache/incubator-airflow/commit/> > > >>>>>>>>>>>>>>>>>>>>>>>>> 656d045e90bf67ca484a3778b2a07a419bfb324a > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> It speeds up loading times a lot, so it's a > good > > >>>>> thing > > >>>>>>> to > > >>>>>>>>>>>>>>>>>> have > > >>>>>>>>>>>>>>>>>>> in > > >>>>>>>>>>>>>>>>>>>>> 1.9. > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 11:14 AM, Chris > > Riccomini < > > >>>>>>>>>>>>>>>>>>>>>>> [email protected] <mailto: > > [email protected]> > > >>> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Sounds good. I'll plan on stable+beta next > week, > > >>>>> then. > > >>>>>>>>>>>>>>>>>> Initial > > >>>>>>>>>>>>>>>>>>>>>> warning > > >>>>>>>>>>>>>>>>>>>>>>>>>> stands, that I will start locking down what > can > > >> get > > >>>>>>> into > > >>>>>>>>>>>>>>>>>> 1.9.0 > > >>>>>>>>>>>>>>>>>>> at > > >>>>>>>>>>>>>>>>>>>>>> that > > >>>>>>>>>>>>>>>>>>>>>>>>>> point. > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 11:10 AM, Bolke de > > Bruin < > > >>>>>>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> No vote indeed, just to gather feedback on a > > >>>>>>> particular > > >>>>>>>>>>>>>>>>>> fixed > > >>>>>>>>>>>>>>>>>>>>> point > > >>>>>>>>>>>>>>>>>>>>>> in > > >>>>>>>>>>>>>>>>>>>>>>>>>>> time. It also gives a bit more trust to a > > tarball > > >>>>>>> than > > >>>>>>>>>>> to > > >>>>>>>>>>>> a > > >>>>>>>>>>>>>>>>>>> git > > >>>>>>>>>>>>>>>>>>>>>> pull. > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Bolke > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 20 Sep 2017, at 20:09, Chris Riccomini < > > >>>>>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected] > > > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I can do a beta. Is the process > significantly > > >>>>>>>>>>> different? > > >>>>>>>>>>>>>>>>>>> IIRC, > > >>>>>>>>>>>>>>>>>>>>> it's > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> basically the same, just no vote, right? > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 10:56 AM, Bolke de > > >> Bruin < > > >>>>>>>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you sure you want to go ahead and do > RCs > > >>>>> right > > >>>>>>>>>>> away? > > >>>>>>>>>>>>>>>>>>>> Isn’t a > > >>>>>>>>>>>>>>>>>>>>>>>>> beta > > >>>>>>>>>>>>>>>>>>>>>>>>>> a > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> bit smarter? > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Bolke > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 20 Sep 2017, at 19:41, Chris Riccomini > < > > >>>>>>>>>>>>>>>>>>>>> [email protected] <mailto: > [email protected]> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey all, > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I want to send out a warning that I'm > > planning > > >>>>> to > > >>>>>>>>>>> cut > > >>>>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>>>>>> stable > > >>>>>>>>>>>>>>>>>>>>>>>>>> branch > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> next week, and begin the RC1 release vote. > > >> Once > > >>>>>>> the > > >>>>>>>>>>>>>>>>>> stable > > >>>>>>>>>>>>>>>>>>>>> branch > > >>>>>>>>>>>>>>>>>>>>>>>>> is > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cut, I > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be locking down what commits get > cherry > > >>>>>>> picked > > >>>>>>>>>>>> into > > >>>>>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>>>>>>>>>> branch, > > >>>>>>>>>>>>>>>>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will only be doing PRs that are required > to > > >> get > > >>>>>>> the > > >>>>>>>>>>>>>>>>>> release > > >>>>>>>>>>>>>>>>>>>>> out. > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chris > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Sep 18, 2017 at 11:19 AM, Chris > > >>>>> Riccomini > > >>>>>>> < > > >>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected] <mailto: > > >>>>> [email protected]> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey all, > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> An update on the 1.9.0 release. Here are > > the > > >>>>>>>>>>>>>>>>>> outstanding > > >>>>>>>>>>>>>>>>>>> PRs > > >>>>>>>>>>>>>>>>>>>>>> that > > >>>>>>>>>>>>>>>>>>>>>>>>>> are > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> slated to be included into 1.9.0: > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISSUE ID |STATUS |DESCRIPTION > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1617 |Open |XSS > > Vulnerability in > > >>>>>>>>>>> Variable > > >>>>>>>>>>>>>>>>>>>>> endpoint > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1611 |Open |Customize > > logging in > > >>>>>>>>>>> Airflow > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1605 |Reopened |Fix log source > of > > >>>>> local > > >>>>>>>>>>>>>>>>>> loggers > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1604 |Open |Rename the > > logger to > > >>>>> log > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1525 |Open |Fix minor > > LICENSE & > > >>>>>>> NOTICE > > >>>>>>>>>>>>>>>>>> issue > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1499 |In Progres|Eliminate > > duplicate > > >>>>> and > > >>>>>>>>>>>>>>>>>> unneeded > > >>>>>>>>>>>>>>>>>>>> code > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1198 |Open |HDFSOperator to > > >>>>> operate > > >>>>>>>>>>> HDFS > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1055 |Open > > >>>>>>>>>>> |airflow/jobs.py:create_dag_ru > > >>>>>>>>>>>>>>>>>> n() > > >>>>>>>>>>>>>>>>>>>>>>>>> exception > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @on > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1019 |Open |active_dagruns > > >>>>> shouldn't > > >>>>>>>>>>>>>>>>>> include > > >>>>>>>>>>>>>>>>>>>>> paused > > >>>>>>>>>>>>>>>>>>>>>>>>>> DAGs > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1018 |Open |Scheduler DAG > > >>>>> processes > > >>>>>>>>>>> can > > >>>>>>>>>>>>>>>>>> not > > >>>>>>>>>>>>>>>>>>> log > > >>>>>>>>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>>>>>>>>>> stdout > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1015 |Open |TreeView > > displayed > > >>>>> over > > >>>>>>>>>>> task > > >>>>>>>>>>>>>>>>>>>>> instances > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1013 |Open > > >>>>>>>>>>> |airflow/jobs.py:manage_slas() > > >>>>>>>>>>>>>>>>>>>>> exception > > >>>>>>>>>>>>>>>>>>>>>>>>> for > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @once > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-976 |Open |Mark success > > running > > >>>>>>> task > > >>>>>>>>>>>>>>>>>> causes > > >>>>>>>>>>>>>>>>>>> it > > >>>>>>>>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>>>>>>>>> fail > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-914 |Open |Refactor > > >>>>>>>>>>>>>>>>>>>>> BackfillJobTest.test_backfill_ > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> examples > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-913 |Open |Refactor > > >>>>>>>>>>>>>>>>>>>>> tests.CoreTest.test_scheduler_ > > >>>>>>>>>>>>>>>>>>>>>>>>> job > > >>>>>>>>>>>>>>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> real > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-912 |Open |Refactor tests > > and > > >>>>> build > > >>>>>>>>>>>>>>>>>> matrix > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-888 |Open |Operators > should > > not > > >>>>>>> push > > >>>>>>>>>>>>>>>>>> XComs > > >>>>>>>>>>>>>>>>>>> by > > >>>>>>>>>>>>>>>>>>>>>>>>> default > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-828 |Open |Add maximum > size > > for > > >>>>>>> XComs > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-825 |Open |Add Dataflow > > >> semantics > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-788 |Open |Context > > unexpectedly > > >>>>>>>>>>> added to > > >>>>>>>>>>>>>>>>>>> hive > > >>>>>>>>>>>>>>>>>>>>> conf > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will be locking down what can get > > >>>>> cherry-picked > > >>>>>>>>>>> into > > >>>>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>>>>>> 1.9.0 > > >>>>>>>>>>>>>>>>>>>>>>>>>>> branch > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shortly, so if you have something you > want > > >> in, > > >>>>>>>>>>> please > > >>>>>>>>>>>>>>>>>> set > > >>>>>>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>>>>>>> fix > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> version > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to 1.9.0. > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We (at WePay) have deployed 1.9.0 into > our > > >> dev > > >>>>>>>>>>>> cluster, > > >>>>>>>>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>>>>>> it > > >>>>>>>>>>>>>>>>>>>>>> has > > >>>>>>>>>>>>>>>>>>>>>>>>>>> been > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> running smoothly for several days. > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ** I could really use help verifying > > >> stability. > > >>>>>>> If > > >>>>>>>>>>> you > > >>>>>>>>>>>>>>>>>> run > > >>>>>>>>>>>>>>>>>>>>>>>>> Airflow, > > >>>>>>>>>>>>>>>>>>>>>>>>>>> it's > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in your best interest to deploy the > 1.9.0 > > >> test > > >>>>>>>>>>> branch > > >>>>>>>>>>>>>>>>>>>>> somewhere, > > >>>>>>>>>>>>>>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> verify > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's working for your workload. ** > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chris > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>> > > >>>>> > > >>>> > > >> > > >> > > > > >
