Hi Chris Can I suggest releasing a beta? The stable branch is only cut at RC time. Betas allow us a broader exposure. It also gives us a point of reference.
In addition the list below are mostly longer standing issues that are also part of the 1.8.x branch. Maybe only consider 1611, 1525, 1258, and 976 as blocker? Cheers Bolke Verstuurd vanaf mijn iPad > Op 28 sep. 2017 om 19:49 heeft Chris Riccomini <[email protected]> het > volgende geschreven: > > Hey all, > > I was planning to cut a 1.9.0 stable branch and 1.9.0 beta release, but > seeing as there are several outstanding bugs, I'm going to delay. Here are > the bugs that I'm tracking: > > AIRFLOW-1611 |Bug |Customize logging in Airflow > AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue > AIRFLOW-1258 |Bug |TaskInstances within SubDagOperator are marked as > AIRFLOW-1055 |Bug |airflow/jobs.py:create_dag_run() exception for > @on > AIRFLOW-1018 |Bug |Scheduler DAG processes can not log to stdout > AIRFLOW-1013 |Bug |airflow/jobs.py:manage_slas() exception for @once > AIRFLOW-988 |Bug |SLA Miss Callbacks Are Repeated if Email is Not > be > AIRFLOW-976 |Bug |Mark success running task causes it to fail > > These are the priority issues. Once they're merged, I'll cut the > v1-9-stable and beta release. > > If you can help clean this up, that would be really appreciated. > > Cheers, > Chris > > On Thu, Sep 28, 2017 at 10:06 AM, Chris Riccomini <[email protected]> > wrote: > >> Marked it for 1.9.0. >> >>> On Thu, Sep 28, 2017 at 9:56 AM, Charlie Jones <[email protected]> wrote: >>> >>> Is there any chance we could include AIRFLOW-988 in 1.9.0? SLA callbacks >>> are not working correctly without emails... Its not a major bug, but it >>> does cause us some annoyance in our current deployment. >>> >>> Link to Jira: >>> https://issues.apache.org/jira/browse/AIRFLOW-988 >>> >>> Link to PR: >>> https://github.com/apache/incubator-airflow/pull/2415 >>> >>> Thanks! >>> Charlie Jones >>> >>> CHARLIE JONES >>> Data Engineer >>> [email protected] | M: 972.821.7631 >>> __________________________________________________ >>> >>> >>> Programmatic Performance.* Localized.* >>> __________________________________________________ >>> >>> 1407 Texas Street | Suite 202 | Fort Worth, TX 76102 >>> 800.840.0768 | www.simpli.fi >>> >>> >>> On Wed, Sep 27, 2017 at 12:11 PM, Chris Riccomini <[email protected]> >>> wrote: >>> >>>> Merged. >>>> >>>> On Tue, Sep 26, 2017 at 2:25 PM, Ryan Buckley < >>> [email protected]> >>>> wrote: >>>> >>>>> Would it be possible to include AIRFLOW-1587? >>>>> Running dags from the UI is currently broken on the 1.9.0 branch due >>> to >>>>> this issue. >>>>> >>>>> https://github.com/apache/incubator-airflow/pull/2590 >>>>> >>>>> Thanks, >>>>> Ryan >>>>> >>>>> On Tue, Sep 26, 2017 at 8:14 AM, Driesprong, Fokko >>> <[email protected] >>>>> >>>>> wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> I would like to include AIRFLOW-1611 in the 1.9.0 release: >>>>>> https://github.com/apache/incubator-airflow/pull/2631 >>>>>> >>>>>> Currently importing a custom logging configuration is not work (as >>> far >>>>> as I >>>>>> know). Any feedback on the PR would also be appreciated. >>>>>> >>>>>> Cheers, Fokko >>>>>> >>>>>> >>>>>> 2017-09-25 23:27 GMT+02:00 Chris Riccomini <[email protected]>: >>>>>> >>>>>>> Done! >>>>>>> >>>>>>> On Mon, Sep 25, 2017 at 5:11 AM, Michael Crawford < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Can you slide the aws and emr connection type fix in? >>>>>>>> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636 < >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636> >>>>>>>> https://github.com/apache/incubator-airflow/pull/2626 < >>>>>>>> https://github.com/apache/incubator-airflow/pull/2626> >>>>>>>> >>>>>>>> It keeps the connection type from getting blanked out on edit >>> for >>>>> these >>>>>>>> types. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Mike >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Sep 21, 2017, at 1:27 PM, Chris Riccomini < >>>>> [email protected]> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Absolutely. Just cherry-picked. I've been looking forward to >>>> these >>>>>>> fixes! >>>>>>>>> >>>>>>>>> On Wed, Sep 20, 2017 at 5:23 PM, Alex Guziel < >>>>> [email protected] >>>>>> . >>>>>>>> invalid >>>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Can we get this in? >>>>>>>>>> >>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1519 >>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1621 >>>>>>>>>> >>>>>>>>>> https://github.com/apache/incubator-airflow/commit/ >>>>>>>>>> b6d2e0a46978e93e16576604624f57d1388814f2 >>>>>>>>>> https://github.com/apache/incubator-airflow/commit/ >>>>>>>>>> 656d045e90bf67ca484a3778b2a07a419bfb324a >>>>>>>>>> >>>>>>>>>> It speeds up loading times a lot, so it's a good thing to >>> have >>>> in >>>>>> 1.9. >>>>>>>>>> >>>>>>>>>> On Wed, Sep 20, 2017 at 11:14 AM, Chris Riccomini < >>>>>>>> [email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Sounds good. I'll plan on stable+beta next week, then. >>> Initial >>>>>>> warning >>>>>>>>>>> stands, that I will start locking down what can get into >>> 1.9.0 >>>> at >>>>>>> that >>>>>>>>>>> point. >>>>>>>>>>> >>>>>>>>>>> On Wed, Sep 20, 2017 at 11:10 AM, Bolke de Bruin < >>>>>> [email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> No vote indeed, just to gather feedback on a particular >>> fixed >>>>>> point >>>>>>> in >>>>>>>>>>>> time. It also gives a bit more trust to a tarball than to a >>>> git >>>>>>> pull. >>>>>>>>>>>> >>>>>>>>>>>> Bolke >>>>>>>>>>>> >>>>>>>>>>>>> On 20 Sep 2017, at 20:09, Chris Riccomini < >>>>> [email protected] >>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> I can do a beta. Is the process significantly different? >>>> IIRC, >>>>>> it's >>>>>>>>>>>>> basically the same, just no vote, right? >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Sep 20, 2017 at 10:56 AM, Bolke de Bruin < >>>>>>> [email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Are you sure you want to go ahead and do RCs right away? >>>>> Isn’t a >>>>>>>>>> beta >>>>>>>>>>> a >>>>>>>>>>>>>> bit smarter? >>>>>>>>>>>>>> >>>>>>>>>>>>>> - Bolke >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 20 Sep 2017, at 19:41, Chris Riccomini < >>>>>> [email protected] >>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hey all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I want to send out a warning that I'm planning to cut >>> the >>>>>> stable >>>>>>>>>>> branch >>>>>>>>>>>>>>> next week, and begin the RC1 release vote. Once the >>> stable >>>>>> branch >>>>>>>>>> is >>>>>>>>>>>>>> cut, I >>>>>>>>>>>>>>> will be locking down what commits get cherry picked into >>>> the >>>>>>>>>> branch, >>>>>>>>>>>> and >>>>>>>>>>>>>>> will only be doing PRs that are required to get the >>> release >>>>>> out. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>> Chris >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Mon, Sep 18, 2017 at 11:19 AM, Chris Riccomini < >>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hey all, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> An update on the 1.9.0 release. Here are the >>> outstanding >>>> PRs >>>>>>> that >>>>>>>>>>> are >>>>>>>>>>>>>>>> slated to be included into 1.9.0: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ISSUE ID |STATUS |DESCRIPTION >>>>>>>>>>>>>>>> AIRFLOW-1617 |Open |XSS Vulnerability in Variable >>>>>> endpoint >>>>>>>>>>>>>>>> AIRFLOW-1611 |Open |Customize logging in Airflow >>>>>>>>>>>>>>>> AIRFLOW-1605 |Reopened |Fix log source of local >>> loggers >>>>>>>>>>>>>>>> AIRFLOW-1604 |Open |Rename the logger to log >>>>>>>>>>>>>>>> AIRFLOW-1525 |Open |Fix minor LICENSE & NOTICE >>> issue >>>>>>>>>>>>>>>> AIRFLOW-1499 |In Progres|Eliminate duplicate and >>> unneeded >>>>> code >>>>>>>>>>>>>>>> AIRFLOW-1198 |Open |HDFSOperator to operate HDFS >>>>>>>>>>>>>>>> AIRFLOW-1055 |Open |airflow/jobs.py:create_dag_ru >>> n() >>>>>>>>>> exception >>>>>>>>>>>> for >>>>>>>>>>>>>>>> @on >>>>>>>>>>>>>>>> AIRFLOW-1019 |Open |active_dagruns shouldn't >>> include >>>>>> paused >>>>>>>>>>> DAGs >>>>>>>>>>>>>>>> AIRFLOW-1018 |Open |Scheduler DAG processes can >>> not >>>> log >>>>>> to >>>>>>>>>>> stdout >>>>>>>>>>>>>>>> AIRFLOW-1015 |Open |TreeView displayed over task >>>>>> instances >>>>>>>>>>>>>>>> AIRFLOW-1013 |Open |airflow/jobs.py:manage_slas() >>>>>> exception >>>>>>>>>> for >>>>>>>>>>>>>>>> @once >>>>>>>>>>>>>>>> AIRFLOW-976 |Open |Mark success running task >>> causes >>>> it >>>>>> to >>>>>>>>>> fail >>>>>>>>>>>>>>>> AIRFLOW-914 |Open |Refactor >>>>>> BackfillJobTest.test_backfill_ >>>>>>>>>>>>>> examples >>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>> AIRFLOW-913 |Open |Refactor >>>>>> tests.CoreTest.test_scheduler_ >>>>>>>>>> job >>>>>>>>>>>> to >>>>>>>>>>>>>>>> real >>>>>>>>>>>>>>>> AIRFLOW-912 |Open |Refactor tests and build >>> matrix >>>>>>>>>>>>>>>> AIRFLOW-888 |Open |Operators should not push >>> XComs >>>> by >>>>>>>>>> default >>>>>>>>>>>>>>>> AIRFLOW-828 |Open |Add maximum size for XComs >>>>>>>>>>>>>>>> AIRFLOW-825 |Open |Add Dataflow semantics >>>>>>>>>>>>>>>> AIRFLOW-788 |Open |Context unexpectedly added to >>>> hive >>>>>> conf >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I will be locking down what can get cherry-picked into >>> the >>>>>> 1.9.0 >>>>>>>>>>>> branch >>>>>>>>>>>>>>>> shortly, so if you have something you want in, please >>> set >>>>> the >>>>>>> fix >>>>>>>>>>>>>> version >>>>>>>>>>>>>>>> to 1.9.0. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We (at WePay) have deployed 1.9.0 into our dev cluster, >>>> and >>>>> it >>>>>>> has >>>>>>>>>>>> been >>>>>>>>>>>>>>>> running smoothly for several days. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ** I could really use help verifying stability. If you >>> run >>>>>>>>>> Airflow, >>>>>>>>>>>> it's >>>>>>>>>>>>>>>> in your best interest to deploy the 1.9.0 test branch >>>>>> somewhere, >>>>>>>>>> and >>>>>>>>>>>>>> verify >>>>>>>>>>>>>>>> it's working for your workload. ** >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>> Chris >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >>
