Welp. Work got in the way, so I'll cut the beta on Monday. :) On Thu, Sep 28, 2017 at 1:32 PM, Chris Riccomini <[email protected]> wrote:
> Works for me. Will try and cut a beta before end of week. > > Blockers for 1.9.0 are: > > AIRFLOW-1611 |Bug |Customize logging in Airflow > AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue > AIRFLOW-1258 |Bug |TaskInstances within SubDagOperator are marked > as > AIRFLOW-976 |Bug |Mark success running task causes it to fail > > > On Thu, Sep 28, 2017 at 1:09 PM, Bolke de Bruin <[email protected]> wrote: > >> Hi Chris >> >> Can I suggest releasing a beta? The stable branch is only cut at RC time. >> Betas allow us a broader exposure. It also gives us a point of reference. >> >> In addition the list below are mostly longer standing issues that are >> also part of the 1.8.x branch. Maybe only consider 1611, 1525, 1258, and >> 976 as blocker? >> >> Cheers >> Bolke >> >> Verstuurd vanaf mijn iPad >> >> > Op 28 sep. 2017 om 19:49 heeft Chris Riccomini <[email protected]> >> het volgende geschreven: >> > >> > Hey all, >> > >> > I was planning to cut a 1.9.0 stable branch and 1.9.0 beta release, but >> > seeing as there are several outstanding bugs, I'm going to delay. Here >> are >> > the bugs that I'm tracking: >> > >> > AIRFLOW-1611 |Bug |Customize logging in Airflow >> > AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue >> > AIRFLOW-1258 |Bug |TaskInstances within SubDagOperator are >> marked as >> > AIRFLOW-1055 |Bug |airflow/jobs.py:create_dag_run() exception >> for >> > @on >> > AIRFLOW-1018 |Bug |Scheduler DAG processes can not log to stdout >> > AIRFLOW-1013 |Bug |airflow/jobs.py:manage_slas() exception for >> @once >> > AIRFLOW-988 |Bug |SLA Miss Callbacks Are Repeated if Email is >> Not >> > be >> > AIRFLOW-976 |Bug |Mark success running task causes it to fail >> > >> > These are the priority issues. Once they're merged, I'll cut the >> > v1-9-stable and beta release. >> > >> > If you can help clean this up, that would be really appreciated. >> > >> > Cheers, >> > Chris >> > >> > On Thu, Sep 28, 2017 at 10:06 AM, Chris Riccomini < >> [email protected]> >> > wrote: >> > >> >> Marked it for 1.9.0. >> >> >> >>> On Thu, Sep 28, 2017 at 9:56 AM, Charlie Jones <[email protected]> >> wrote: >> >>> >> >>> Is there any chance we could include AIRFLOW-988 in 1.9.0? SLA >> callbacks >> >>> are not working correctly without emails... Its not a major bug, but >> it >> >>> does cause us some annoyance in our current deployment. >> >>> >> >>> Link to Jira: >> >>> https://issues.apache.org/jira/browse/AIRFLOW-988 >> >>> >> >>> Link to PR: >> >>> https://github.com/apache/incubator-airflow/pull/2415 >> >>> >> >>> Thanks! >> >>> Charlie Jones >> >>> >> >>> CHARLIE JONES >> >>> Data Engineer >> >>> [email protected] | M: 972.821.7631 >> >>> __________________________________________________ >> >>> >> >>> >> >>> Programmatic Performance.* Localized.* >> >>> __________________________________________________ >> >>> >> >>> 1407 Texas Street | Suite 202 | Fort Worth, TX 76102 >> >>> 800.840.0768 | www.simpli.fi >> >>> >> >>> >> >>> On Wed, Sep 27, 2017 at 12:11 PM, Chris Riccomini < >> [email protected]> >> >>> wrote: >> >>> >> >>>> Merged. >> >>>> >> >>>> On Tue, Sep 26, 2017 at 2:25 PM, Ryan Buckley < >> >>> [email protected]> >> >>>> wrote: >> >>>> >> >>>>> Would it be possible to include AIRFLOW-1587? >> >>>>> Running dags from the UI is currently broken on the 1.9.0 branch due >> >>> to >> >>>>> this issue. >> >>>>> >> >>>>> https://github.com/apache/incubator-airflow/pull/2590 >> >>>>> >> >>>>> Thanks, >> >>>>> Ryan >> >>>>> >> >>>>> On Tue, Sep 26, 2017 at 8:14 AM, Driesprong, Fokko >> >>> <[email protected] >> >>>>> >> >>>>> wrote: >> >>>>> >> >>>>>> Hi All, >> >>>>>> >> >>>>>> I would like to include AIRFLOW-1611 in the 1.9.0 release: >> >>>>>> https://github.com/apache/incubator-airflow/pull/2631 >> >>>>>> >> >>>>>> Currently importing a custom logging configuration is not work (as >> >>> far >> >>>>> as I >> >>>>>> know). Any feedback on the PR would also be appreciated. >> >>>>>> >> >>>>>> Cheers, Fokko >> >>>>>> >> >>>>>> >> >>>>>> 2017-09-25 23:27 GMT+02:00 Chris Riccomini <[email protected] >> >: >> >>>>>> >> >>>>>>> Done! >> >>>>>>> >> >>>>>>> On Mon, Sep 25, 2017 at 5:11 AM, Michael Crawford < >> >>>>>>> [email protected]> wrote: >> >>>>>>> >> >>>>>>>> Can you slide the aws and emr connection type fix in? >> >>>>>>>> >> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636 < >> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636> >> >>>>>>>> https://github.com/apache/incubator-airflow/pull/2626 < >> >>>>>>>> https://github.com/apache/incubator-airflow/pull/2626> >> >>>>>>>> >> >>>>>>>> It keeps the connection type from getting blanked out on edit >> >>> for >> >>>>> these >> >>>>>>>> types. >> >>>>>>>> >> >>>>>>>> Thanks, >> >>>>>>>> Mike >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>>> On Sep 21, 2017, at 1:27 PM, Chris Riccomini < >> >>>>> [email protected]> >> >>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>> Absolutely. Just cherry-picked. I've been looking forward to >> >>>> these >> >>>>>>> fixes! >> >>>>>>>>> >> >>>>>>>>> On Wed, Sep 20, 2017 at 5:23 PM, Alex Guziel < >> >>>>> [email protected] >> >>>>>> . >> >>>>>>>> invalid >> >>>>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>>> Can we get this in? >> >>>>>>>>>> >> >>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1519 >> >>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1621 >> >>>>>>>>>> >> >>>>>>>>>> https://github.com/apache/incubator-airflow/commit/ >> >>>>>>>>>> b6d2e0a46978e93e16576604624f57d1388814f2 >> >>>>>>>>>> https://github.com/apache/incubator-airflow/commit/ >> >>>>>>>>>> 656d045e90bf67ca484a3778b2a07a419bfb324a >> >>>>>>>>>> >> >>>>>>>>>> It speeds up loading times a lot, so it's a good thing to >> >>> have >> >>>> in >> >>>>>> 1.9. >> >>>>>>>>>> >> >>>>>>>>>> On Wed, Sep 20, 2017 at 11:14 AM, Chris Riccomini < >> >>>>>>>> [email protected]> >> >>>>>>>>>> wrote: >> >>>>>>>>>> >> >>>>>>>>>>> Sounds good. I'll plan on stable+beta next week, then. >> >>> Initial >> >>>>>>> warning >> >>>>>>>>>>> stands, that I will start locking down what can get into >> >>> 1.9.0 >> >>>> at >> >>>>>>> that >> >>>>>>>>>>> point. >> >>>>>>>>>>> >> >>>>>>>>>>> On Wed, Sep 20, 2017 at 11:10 AM, Bolke de Bruin < >> >>>>>> [email protected]> >> >>>>>>>>>>> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>>> No vote indeed, just to gather feedback on a particular >> >>> fixed >> >>>>>> point >> >>>>>>> in >> >>>>>>>>>>>> time. It also gives a bit more trust to a tarball than to a >> >>>> git >> >>>>>>> pull. >> >>>>>>>>>>>> >> >>>>>>>>>>>> Bolke >> >>>>>>>>>>>> >> >>>>>>>>>>>>> On 20 Sep 2017, at 20:09, Chris Riccomini < >> >>>>> [email protected] >> >>>>>>> >> >>>>>>>>>>> wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> I can do a beta. Is the process significantly different? >> >>>> IIRC, >> >>>>>> it's >> >>>>>>>>>>>>> basically the same, just no vote, right? >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> On Wed, Sep 20, 2017 at 10:56 AM, Bolke de Bruin < >> >>>>>>> [email protected]> >> >>>>>>>>>>>> wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>>> Are you sure you want to go ahead and do RCs right away? >> >>>>> Isn’t a >> >>>>>>>>>> beta >> >>>>>>>>>>> a >> >>>>>>>>>>>>>> bit smarter? >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> - Bolke >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> On 20 Sep 2017, at 19:41, Chris Riccomini < >> >>>>>> [email protected] >> >>>>>>>> >> >>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Hey all, >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> I want to send out a warning that I'm planning to cut >> >>> the >> >>>>>> stable >> >>>>>>>>>>> branch >> >>>>>>>>>>>>>>> next week, and begin the RC1 release vote. Once the >> >>> stable >> >>>>>> branch >> >>>>>>>>>> is >> >>>>>>>>>>>>>> cut, I >> >>>>>>>>>>>>>>> will be locking down what commits get cherry picked into >> >>>> the >> >>>>>>>>>> branch, >> >>>>>>>>>>>> and >> >>>>>>>>>>>>>>> will only be doing PRs that are required to get the >> >>> release >> >>>>>> out. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Cheers, >> >>>>>>>>>>>>>>> Chris >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> On Mon, Sep 18, 2017 at 11:19 AM, Chris Riccomini < >> >>>>>>>>>>>> [email protected] >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Hey all, >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> An update on the 1.9.0 release. Here are the >> >>> outstanding >> >>>> PRs >> >>>>>>> that >> >>>>>>>>>>> are >> >>>>>>>>>>>>>>>> slated to be included into 1.9.0: >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> ISSUE ID |STATUS |DESCRIPTION >> >>>>>>>>>>>>>>>> AIRFLOW-1617 |Open |XSS Vulnerability in Variable >> >>>>>> endpoint >> >>>>>>>>>>>>>>>> AIRFLOW-1611 |Open |Customize logging in Airflow >> >>>>>>>>>>>>>>>> AIRFLOW-1605 |Reopened |Fix log source of local >> >>> loggers >> >>>>>>>>>>>>>>>> AIRFLOW-1604 |Open |Rename the logger to log >> >>>>>>>>>>>>>>>> AIRFLOW-1525 |Open |Fix minor LICENSE & NOTICE >> >>> issue >> >>>>>>>>>>>>>>>> AIRFLOW-1499 |In Progres|Eliminate duplicate and >> >>> unneeded >> >>>>> code >> >>>>>>>>>>>>>>>> AIRFLOW-1198 |Open |HDFSOperator to operate HDFS >> >>>>>>>>>>>>>>>> AIRFLOW-1055 |Open |airflow/jobs.py:create_dag_ru >> >>> n() >> >>>>>>>>>> exception >> >>>>>>>>>>>> for >> >>>>>>>>>>>>>>>> @on >> >>>>>>>>>>>>>>>> AIRFLOW-1019 |Open |active_dagruns shouldn't >> >>> include >> >>>>>> paused >> >>>>>>>>>>> DAGs >> >>>>>>>>>>>>>>>> AIRFLOW-1018 |Open |Scheduler DAG processes can >> >>> not >> >>>> log >> >>>>>> to >> >>>>>>>>>>> stdout >> >>>>>>>>>>>>>>>> AIRFLOW-1015 |Open |TreeView displayed over task >> >>>>>> instances >> >>>>>>>>>>>>>>>> AIRFLOW-1013 |Open |airflow/jobs.py:manage_slas() >> >>>>>> exception >> >>>>>>>>>> for >> >>>>>>>>>>>>>>>> @once >> >>>>>>>>>>>>>>>> AIRFLOW-976 |Open |Mark success running task >> >>> causes >> >>>> it >> >>>>>> to >> >>>>>>>>>> fail >> >>>>>>>>>>>>>>>> AIRFLOW-914 |Open |Refactor >> >>>>>> BackfillJobTest.test_backfill_ >> >>>>>>>>>>>>>> examples >> >>>>>>>>>>>>>>>> to >> >>>>>>>>>>>>>>>> AIRFLOW-913 |Open |Refactor >> >>>>>> tests.CoreTest.test_scheduler_ >> >>>>>>>>>> job >> >>>>>>>>>>>> to >> >>>>>>>>>>>>>>>> real >> >>>>>>>>>>>>>>>> AIRFLOW-912 |Open |Refactor tests and build >> >>> matrix >> >>>>>>>>>>>>>>>> AIRFLOW-888 |Open |Operators should not push >> >>> XComs >> >>>> by >> >>>>>>>>>> default >> >>>>>>>>>>>>>>>> AIRFLOW-828 |Open |Add maximum size for XComs >> >>>>>>>>>>>>>>>> AIRFLOW-825 |Open |Add Dataflow semantics >> >>>>>>>>>>>>>>>> AIRFLOW-788 |Open |Context unexpectedly added to >> >>>> hive >> >>>>>> conf >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> I will be locking down what can get cherry-picked into >> >>> the >> >>>>>> 1.9.0 >> >>>>>>>>>>>> branch >> >>>>>>>>>>>>>>>> shortly, so if you have something you want in, please >> >>> set >> >>>>> the >> >>>>>>> fix >> >>>>>>>>>>>>>> version >> >>>>>>>>>>>>>>>> to 1.9.0. >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> We (at WePay) have deployed 1.9.0 into our dev cluster, >> >>>> and >> >>>>> it >> >>>>>>> has >> >>>>>>>>>>>> been >> >>>>>>>>>>>>>>>> running smoothly for several days. >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> ** I could really use help verifying stability. If you >> >>> run >> >>>>>>>>>> Airflow, >> >>>>>>>>>>>> it's >> >>>>>>>>>>>>>>>> in your best interest to deploy the 1.9.0 test branch >> >>>>>> somewhere, >> >>>>>>>>>> and >> >>>>>>>>>>>>>> verify >> >>>>>>>>>>>>>>>> it's working for your workload. ** >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Cheers, >> >>>>>>>>>>>>>>>> Chris >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>>> >> >>> >> >> >> >> >> > >
