Hey all, I was planning to cut a 1.9.0 stable branch and 1.9.0 beta release, but seeing as there are several outstanding bugs, I'm going to delay. Here are the bugs that I'm tracking:
AIRFLOW-1611 |Bug |Customize logging in Airflow AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue AIRFLOW-1258 |Bug |TaskInstances within SubDagOperator are marked as AIRFLOW-1055 |Bug |airflow/jobs.py:create_dag_run() exception for @on AIRFLOW-1018 |Bug |Scheduler DAG processes can not log to stdout AIRFLOW-1013 |Bug |airflow/jobs.py:manage_slas() exception for @once AIRFLOW-988 |Bug |SLA Miss Callbacks Are Repeated if Email is Not be AIRFLOW-976 |Bug |Mark success running task causes it to fail These are the priority issues. Once they're merged, I'll cut the v1-9-stable and beta release. If you can help clean this up, that would be really appreciated. Cheers, Chris On Thu, Sep 28, 2017 at 10:06 AM, Chris Riccomini <[email protected]> wrote: > Marked it for 1.9.0. > > On Thu, Sep 28, 2017 at 9:56 AM, Charlie Jones <[email protected]> wrote: > >> Is there any chance we could include AIRFLOW-988 in 1.9.0? SLA callbacks >> are not working correctly without emails... Its not a major bug, but it >> does cause us some annoyance in our current deployment. >> >> Link to Jira: >> https://issues.apache.org/jira/browse/AIRFLOW-988 >> >> Link to PR: >> https://github.com/apache/incubator-airflow/pull/2415 >> >> Thanks! >> Charlie Jones >> >> CHARLIE JONES >> Data Engineer >> [email protected] | M: 972.821.7631 >> __________________________________________________ >> >> >> Programmatic Performance.* Localized.* >> __________________________________________________ >> >> 1407 Texas Street | Suite 202 | Fort Worth, TX 76102 >> 800.840.0768 | www.simpli.fi >> >> >> On Wed, Sep 27, 2017 at 12:11 PM, Chris Riccomini <[email protected]> >> wrote: >> >> > Merged. >> > >> > On Tue, Sep 26, 2017 at 2:25 PM, Ryan Buckley < >> [email protected]> >> > wrote: >> > >> > > Would it be possible to include AIRFLOW-1587? >> > > Running dags from the UI is currently broken on the 1.9.0 branch due >> to >> > > this issue. >> > > >> > > https://github.com/apache/incubator-airflow/pull/2590 >> > > >> > > Thanks, >> > > Ryan >> > > >> > > On Tue, Sep 26, 2017 at 8:14 AM, Driesprong, Fokko >> <[email protected] >> > > >> > > wrote: >> > > >> > > > Hi All, >> > > > >> > > > I would like to include AIRFLOW-1611 in the 1.9.0 release: >> > > > https://github.com/apache/incubator-airflow/pull/2631 >> > > > >> > > > Currently importing a custom logging configuration is not work (as >> far >> > > as I >> > > > know). Any feedback on the PR would also be appreciated. >> > > > >> > > > Cheers, Fokko >> > > > >> > > > >> > > > 2017-09-25 23:27 GMT+02:00 Chris Riccomini <[email protected]>: >> > > > >> > > > > Done! >> > > > > >> > > > > On Mon, Sep 25, 2017 at 5:11 AM, Michael Crawford < >> > > > > [email protected]> wrote: >> > > > > >> > > > > > Can you slide the aws and emr connection type fix in? >> > > > > > >> > > > > > https://issues.apache.org/jira/browse/AIRFLOW-1636 < >> > > > > > https://issues.apache.org/jira/browse/AIRFLOW-1636> >> > > > > > https://github.com/apache/incubator-airflow/pull/2626 < >> > > > > > https://github.com/apache/incubator-airflow/pull/2626> >> > > > > > >> > > > > > It keeps the connection type from getting blanked out on edit >> for >> > > these >> > > > > > types. >> > > > > > >> > > > > > Thanks, >> > > > > > Mike >> > > > > > >> > > > > > >> > > > > > >> > > > > > > On Sep 21, 2017, at 1:27 PM, Chris Riccomini < >> > > [email protected]> >> > > > > > wrote: >> > > > > > > >> > > > > > > Absolutely. Just cherry-picked. I've been looking forward to >> > these >> > > > > fixes! >> > > > > > > >> > > > > > > On Wed, Sep 20, 2017 at 5:23 PM, Alex Guziel < >> > > [email protected] >> > > > . >> > > > > > invalid >> > > > > > >> wrote: >> > > > > > > >> > > > > > >> Can we get this in? >> > > > > > >> >> > > > > > >> https://issues.apache.org/jira/browse/AIRFLOW-1519 >> > > > > > >> https://issues.apache.org/jira/browse/AIRFLOW-1621 >> > > > > > >> >> > > > > > >> https://github.com/apache/incubator-airflow/commit/ >> > > > > > >> b6d2e0a46978e93e16576604624f57d1388814f2 >> > > > > > >> https://github.com/apache/incubator-airflow/commit/ >> > > > > > >> 656d045e90bf67ca484a3778b2a07a419bfb324a >> > > > > > >> >> > > > > > >> It speeds up loading times a lot, so it's a good thing to >> have >> > in >> > > > 1.9. >> > > > > > >> >> > > > > > >> On Wed, Sep 20, 2017 at 11:14 AM, Chris Riccomini < >> > > > > > [email protected]> >> > > > > > >> wrote: >> > > > > > >> >> > > > > > >>> Sounds good. I'll plan on stable+beta next week, then. >> Initial >> > > > > warning >> > > > > > >>> stands, that I will start locking down what can get into >> 1.9.0 >> > at >> > > > > that >> > > > > > >>> point. >> > > > > > >>> >> > > > > > >>> On Wed, Sep 20, 2017 at 11:10 AM, Bolke de Bruin < >> > > > [email protected]> >> > > > > > >>> wrote: >> > > > > > >>> >> > > > > > >>>> No vote indeed, just to gather feedback on a particular >> fixed >> > > > point >> > > > > in >> > > > > > >>>> time. It also gives a bit more trust to a tarball than to a >> > git >> > > > > pull. >> > > > > > >>>> >> > > > > > >>>> Bolke >> > > > > > >>>> >> > > > > > >>>>> On 20 Sep 2017, at 20:09, Chris Riccomini < >> > > [email protected] >> > > > > >> > > > > > >>> wrote: >> > > > > > >>>>> >> > > > > > >>>>> I can do a beta. Is the process significantly different? >> > IIRC, >> > > > it's >> > > > > > >>>>> basically the same, just no vote, right? >> > > > > > >>>>> >> > > > > > >>>>> On Wed, Sep 20, 2017 at 10:56 AM, Bolke de Bruin < >> > > > > [email protected]> >> > > > > > >>>> wrote: >> > > > > > >>>>> >> > > > > > >>>>>> Are you sure you want to go ahead and do RCs right away? >> > > Isn’t a >> > > > > > >> beta >> > > > > > >>> a >> > > > > > >>>>>> bit smarter? >> > > > > > >>>>>> >> > > > > > >>>>>> - Bolke >> > > > > > >>>>>> >> > > > > > >>>>>>> On 20 Sep 2017, at 19:41, Chris Riccomini < >> > > > [email protected] >> > > > > > >> > > > > > >>>> wrote: >> > > > > > >>>>>>> >> > > > > > >>>>>>> Hey all, >> > > > > > >>>>>>> >> > > > > > >>>>>>> I want to send out a warning that I'm planning to cut >> the >> > > > stable >> > > > > > >>> branch >> > > > > > >>>>>>> next week, and begin the RC1 release vote. Once the >> stable >> > > > branch >> > > > > > >> is >> > > > > > >>>>>> cut, I >> > > > > > >>>>>>> will be locking down what commits get cherry picked into >> > the >> > > > > > >> branch, >> > > > > > >>>> and >> > > > > > >>>>>>> will only be doing PRs that are required to get the >> release >> > > > out. >> > > > > > >>>>>>> >> > > > > > >>>>>>> Cheers, >> > > > > > >>>>>>> Chris >> > > > > > >>>>>>> >> > > > > > >>>>>>> On Mon, Sep 18, 2017 at 11:19 AM, Chris Riccomini < >> > > > > > >>>> [email protected] >> > > > > > >>>>>>> >> > > > > > >>>>>>> wrote: >> > > > > > >>>>>>> >> > > > > > >>>>>>>> Hey all, >> > > > > > >>>>>>>> >> > > > > > >>>>>>>> An update on the 1.9.0 release. Here are the >> outstanding >> > PRs >> > > > > that >> > > > > > >>> are >> > > > > > >>>>>>>> slated to be included into 1.9.0: >> > > > > > >>>>>>>> >> > > > > > >>>>>>>> ISSUE ID |STATUS |DESCRIPTION >> > > > > > >>>>>>>> AIRFLOW-1617 |Open |XSS Vulnerability in Variable >> > > > endpoint >> > > > > > >>>>>>>> AIRFLOW-1611 |Open |Customize logging in Airflow >> > > > > > >>>>>>>> AIRFLOW-1605 |Reopened |Fix log source of local >> loggers >> > > > > > >>>>>>>> AIRFLOW-1604 |Open |Rename the logger to log >> > > > > > >>>>>>>> AIRFLOW-1525 |Open |Fix minor LICENSE & NOTICE >> issue >> > > > > > >>>>>>>> AIRFLOW-1499 |In Progres|Eliminate duplicate and >> unneeded >> > > code >> > > > > > >>>>>>>> AIRFLOW-1198 |Open |HDFSOperator to operate HDFS >> > > > > > >>>>>>>> AIRFLOW-1055 |Open |airflow/jobs.py:create_dag_ru >> n() >> > > > > > >> exception >> > > > > > >>>> for >> > > > > > >>>>>>>> @on >> > > > > > >>>>>>>> AIRFLOW-1019 |Open |active_dagruns shouldn't >> include >> > > > paused >> > > > > > >>> DAGs >> > > > > > >>>>>>>> AIRFLOW-1018 |Open |Scheduler DAG processes can >> not >> > log >> > > > to >> > > > > > >>> stdout >> > > > > > >>>>>>>> AIRFLOW-1015 |Open |TreeView displayed over task >> > > > instances >> > > > > > >>>>>>>> AIRFLOW-1013 |Open |airflow/jobs.py:manage_slas() >> > > > exception >> > > > > > >> for >> > > > > > >>>>>>>> @once >> > > > > > >>>>>>>> AIRFLOW-976 |Open |Mark success running task >> causes >> > it >> > > > to >> > > > > > >> fail >> > > > > > >>>>>>>> AIRFLOW-914 |Open |Refactor >> > > > BackfillJobTest.test_backfill_ >> > > > > > >>>>>> examples >> > > > > > >>>>>>>> to >> > > > > > >>>>>>>> AIRFLOW-913 |Open |Refactor >> > > > tests.CoreTest.test_scheduler_ >> > > > > > >> job >> > > > > > >>>> to >> > > > > > >>>>>>>> real >> > > > > > >>>>>>>> AIRFLOW-912 |Open |Refactor tests and build >> matrix >> > > > > > >>>>>>>> AIRFLOW-888 |Open |Operators should not push >> XComs >> > by >> > > > > > >> default >> > > > > > >>>>>>>> AIRFLOW-828 |Open |Add maximum size for XComs >> > > > > > >>>>>>>> AIRFLOW-825 |Open |Add Dataflow semantics >> > > > > > >>>>>>>> AIRFLOW-788 |Open |Context unexpectedly added to >> > hive >> > > > conf >> > > > > > >>>>>>>> >> > > > > > >>>>>>>> I will be locking down what can get cherry-picked into >> the >> > > > 1.9.0 >> > > > > > >>>> branch >> > > > > > >>>>>>>> shortly, so if you have something you want in, please >> set >> > > the >> > > > > fix >> > > > > > >>>>>> version >> > > > > > >>>>>>>> to 1.9.0. >> > > > > > >>>>>>>> >> > > > > > >>>>>>>> We (at WePay) have deployed 1.9.0 into our dev cluster, >> > and >> > > it >> > > > > has >> > > > > > >>>> been >> > > > > > >>>>>>>> running smoothly for several days. >> > > > > > >>>>>>>> >> > > > > > >>>>>>>> ** I could really use help verifying stability. If you >> run >> > > > > > >> Airflow, >> > > > > > >>>> it's >> > > > > > >>>>>>>> in your best interest to deploy the 1.9.0 test branch >> > > > somewhere, >> > > > > > >> and >> > > > > > >>>>>> verify >> > > > > > >>>>>>>> it's working for your workload. ** >> > > > > > >>>>>>>> >> > > > > > >>>>>>>> Cheers, >> > > > > > >>>>>>>> Chris >> > > > > > >>>>>>>> >> > > > > > >>>>>> >> > > > > > >>>>>> >> > > > > > >>>> >> > > > > > >>>> >> > > > > > >>> >> > > > > > >> >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
