I upgraded our production environment today. Some observations

1. Database migration (add max tries) didn’t workout of the box correctly for 
Postgres. “max_tries = False” didn’t work and needed to be casted to an int. 
Not sure how this gets to be false. This is a block imho
2. SSHExecuteOperator not having backwards compatibility sucks. We needed to 
rework quite a lot of dags
3. We use LdAP for logins. We havent configured the ‘superuser’ ‘data_profiler’ 
groups and thus should have automatic superuser privileges. We don’t at the 
moment (eg. we cannot manage connections or see the admin screens).

After fixing #1, the rest seems quite normal. Happy with the new logging.

Bolke


> On 20 Oct 2017, at 12:16, Driesprong, Fokko <[email protected]> wrote:
> 
> Hi Bolke,
> 
> This one is a blocker: https://issues.apache.org/jira/browse/AIRFLOW-1731.
> 
> This requirers that we are sure that the /plugin/ folder is on the path:
> https://github.com/apache/incubator-airflow/blob/eb2f589099b87743482c2eb16261b49e284dcd96/airflow/plugins_manager.py
> 
> And maybe add an additional test and updating the docs a bit. Hopefully
> somewhere in the next few days, however I'm a bit busy with the Spark
> Summit.
> 
> Cheers, Fokko
> 
> 2017-10-19 21:03 GMT+02:00 Bolke de Bruin <[email protected]>:
> 
>> Im planning to upgrade our production to 1.9.0alphaX tomorrow.
>> 
>> What are the issues that are open?
>> 
>> Cheers
>> Bolke
>> 
>>> On 9 Oct 2017, at 20:24, Chris Riccomini <[email protected]> wrote:
>>> 
>>> K, I will plan on cutting an alpha1 later this week. Something for you
>> guys to play with.
>>> 
>>> On Mon, Oct 9, 2017 at 11:23 AM, Bolke de Bruin <[email protected]
>> <mailto:[email protected]>> wrote:
>>> hi Chris,
>>> 
>>> We are still running pre alpha. I am a bit preoccupied with preparing
>> for a conference and the team is readying a release of one of our core
>> products. So it will probably will be after this week when I get my hands
>> dirty again.
>>> 
>>> Cheers
>>> Bolke
>>> 
>>> Verstuurd vanaf mijn iPad
>>> 
>>>> Op 4 okt. 2017 om 23:35 heeft Chris Riccomini <[email protected]
>> <mailto:[email protected]>> het volgende geschreven:
>>>> 
>>>> Hey all,
>>>> 
>>>> Checking in. Has anyone tested this in their environments? Stable
>> releases
>>>> depend on community involvement.
>>>> 
>>>> Cheers,
>>>> Chris
>>>> 
>>>> On Tue, Oct 3, 2017 at 10:30 PM, Chris Riccomini <
>> [email protected] <mailto:[email protected]>>
>>>> wrote:
>>>> 
>>>>> I've added AIRFLOW-1635 to the v1-9-test branch. It's not in alpha0,
>> but
>>>>> will be included in alpha1.
>>>>> 
>>>>>> On Tue, Oct 3, 2017 at 4:13 PM, Feng Lu <[email protected]>
>> wrote:
>>>>>> 
>>>>>> Hi Chris,
>>>>>> 
>>>>>> I know it's annoying to have last minute commit com in, but this is a
>>>>>> highly desirable feature for folks using GCP operators, is it
>> possible to
>>>>>> include AIRFLOW-1635
>>>>>> <https://github.com/apache/incubator-airflow/commit/b3e985a3 <
>> https://github.com/apache/incubator-airflow/commit/b3e985a3>
>>>>>> 146272ecfd3ceaaa0d8567e4e9e117d4>
>>>>>> in?
>>>>>> More than happy to offer help if there's something I can do.
>>>>>> Thanks a lot.
>>>>>> 
>>>>>> Feng
>>>>>> 
>>>>>> On Mon, Oct 2, 2017 at 3:24 PM, Chris Riccomini <
>> [email protected] <mailto:[email protected]>>
>>>>>> wrote:
>>>>>> 
>>>>>>> Hey all,
>>>>>>> 
>>>>>>> I have cut a 1.9.0alpha0 release of Airflow. You can download it
>> here:
>>>>>>> 
>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.
>> 9.0alpha0/ <https://dist.apache.org/repos/dist/dev/incubator/
>> airflow/1.9.0alpha0/>
>>>>>>> 
>>>>>>> The bin tarball can be installed with pip:
>>>>>>> 
>>>>>>> pip install apache-airflow-1.9.0alpha0+incubating-bin.tar.gz
>>>>>>> 
>>>>>>> The goal is to have the community install and run this to expose any
>>>>>> bugs
>>>>>>> before we move on to official release candidates.
>>>>>>> 
>>>>>>> Here are the outstanding blocker bugs for 1.9.0:
>>>>>>> 
>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue
>>>>>>> AIRFLOW-1258 |Bug         |TaskInstances within SubDagOperator are
>>>>>> marked
>>>>>>> as
>>>>>>> AIRFLOW-1055 |Bug         |airflow/jobs.py:create_dag_run()
>> exception
>>>>>>> for
>>>>>>> @on
>>>>>>> AIRFLOW-1018 |Bug         |Scheduler DAG processes can not log to
>>>>>> stdout
>>>>>>> AIRFLOW-1013 |Bug         |airflow/jobs.py:manage_slas() exception
>>>>>> for
>>>>>>> @once
>>>>>>> AIRFLOW-976  |Bug         |Mark success running task causes it to
>> fail
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Chris
>>>>>>> 
>>>>>>> On Fri, Sep 29, 2017 at 3:54 PM, Chris Riccomini <
>> [email protected] <mailto:[email protected]>
>>>>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Welp. Work got in the way, so I'll cut the beta on Monday. :)
>>>>>>>> 
>>>>>>>> On Thu, Sep 28, 2017 at 1:32 PM, Chris Riccomini <
>>>>>> [email protected] <mailto:[email protected]>>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Works for me. Will try and cut a beta before end of week.
>>>>>>>>> 
>>>>>>>>> Blockers for 1.9.0 are:
>>>>>>>>> 
>>>>>>>>> AIRFLOW-1611 |Bug         |Customize logging in Airflow
>>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue
>>>>>>>>> AIRFLOW-1258 |Bug         |TaskInstances within SubDagOperator are
>>>>>>> marked
>>>>>>>>> as
>>>>>>>>> AIRFLOW-976  |Bug         |Mark success running task causes it to
>>>>>> fail
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Thu, Sep 28, 2017 at 1:09 PM, Bolke de Bruin <
>> [email protected] <mailto:[email protected]>>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hi Chris
>>>>>>>>>> 
>>>>>>>>>> Can I suggest releasing a beta? The stable branch is only cut at
>> RC
>>>>>>>>>> time. Betas allow us a broader exposure. It also gives us a
>> point of
>>>>>>>>>> reference.
>>>>>>>>>> 
>>>>>>>>>> In addition the list below are mostly longer standing issues that
>>>>>> are
>>>>>>>>>> also part of the 1.8.x branch. Maybe only consider 1611, 1525,
>> 1258,
>>>>>>> and
>>>>>>>>>> 976 as blocker?
>>>>>>>>>> 
>>>>>>>>>> Cheers
>>>>>>>>>> Bolke
>>>>>>>>>> 
>>>>>>>>>> Verstuurd vanaf mijn iPad
>>>>>>>>>> 
>>>>>>>>>>> Op 28 sep. 2017 om 19:49 heeft Chris Riccomini <
>>>>>>> [email protected] <mailto:[email protected]>>
>>>>>>>>>> het volgende geschreven:
>>>>>>>>>>> 
>>>>>>>>>>> Hey all,
>>>>>>>>>>> 
>>>>>>>>>>> I was planning to cut a 1.9.0 stable branch and 1.9.0 beta
>>>>>> release,
>>>>>>> but
>>>>>>>>>>> seeing as there are several outstanding bugs, I'm going to
>> delay.
>>>>>>> Here
>>>>>>>>>> are
>>>>>>>>>>> the bugs that I'm tracking:
>>>>>>>>>>> 
>>>>>>>>>>> AIRFLOW-1611 |Bug         |Customize logging in Airflow
>>>>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue
>>>>>>>>>>> AIRFLOW-1258 |Bug         |TaskInstances within SubDagOperator
>> are
>>>>>>>>>> marked as
>>>>>>>>>>> AIRFLOW-1055 |Bug         |airflow/jobs.py:create_dag_run()
>>>>>>> exception
>>>>>>>>>> for
>>>>>>>>>>> @on
>>>>>>>>>>> AIRFLOW-1018 |Bug         |Scheduler DAG processes can not log
>> to
>>>>>>>>>> stdout
>>>>>>>>>>> AIRFLOW-1013 |Bug         |airflow/jobs.py:manage_slas()
>>>>>> exception
>>>>>>>>>> for @once
>>>>>>>>>>> AIRFLOW-988  |Bug         |SLA Miss Callbacks Are Repeated if
>>>>>> Email
>>>>>>> is
>>>>>>>>>> Not
>>>>>>>>>>> be
>>>>>>>>>>> AIRFLOW-976  |Bug         |Mark success running task causes it
>> to
>>>>>>> fail
>>>>>>>>>>> 
>>>>>>>>>>> These are the priority issues. Once they're merged, I'll cut the
>>>>>>>>>>> v1-9-stable and beta release.
>>>>>>>>>>> 
>>>>>>>>>>> If you can help clean this up, that would be really appreciated.
>>>>>>>>>>> 
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Chris
>>>>>>>>>>> 
>>>>>>>>>>> On Thu, Sep 28, 2017 at 10:06 AM, Chris Riccomini <
>>>>>>>>>> [email protected] <mailto:[email protected]>>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Marked it for 1.9.0.
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Thu, Sep 28, 2017 at 9:56 AM, Charlie Jones <
>>>>>> [email protected] <mailto:[email protected]>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Is there any chance we could include AIRFLOW-988 in 1.9.0? SLA
>>>>>>>>>> callbacks
>>>>>>>>>>>>> are not working correctly without emails... Its not a major
>> bug,
>>>>>>> but
>>>>>>>>>> it
>>>>>>>>>>>>> does cause us some annoyance in our current deployment.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Link to Jira:
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-988 <
>> https://issues.apache.org/jira/browse/AIRFLOW-988>
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Link to PR:
>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2415 <
>> https://github.com/apache/incubator-airflow/pull/2415>
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>> Charlie Jones
>>>>>>>>>>>>> 
>>>>>>>>>>>>> CHARLIE JONES
>>>>>>>>>>>>> Data Engineer
>>>>>>>>>>>>> [email protected] <mailto:[email protected]>  |  M:
>> 972.821.7631 <tel:972.821.7631>
>>>>>>>>>>>>> __________________________________________________
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Programmatic Performance.* Localized.*
>>>>>>>>>>>>> __________________________________________________
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 1407 Texas Street  |  Suite 202  |  Fort Worth, TX 76102
>>>>>>>>>>>>> 800.840.0768 <tel:800.840.0768>  |  www.simpli.fi <
>> http://www.simpli.fi/>
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Wed, Sep 27, 2017 at 12:11 PM, Chris Riccomini <
>>>>>>>>>> [email protected] <mailto:[email protected]>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Merged.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:25 PM, Ryan Buckley <
>>>>>>>>>>>>> [email protected] <mailto:[email protected]>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Would it be possible to include AIRFLOW-1587?
>>>>>>>>>>>>>>> Running dags from the UI is currently broken on the 1.9.0
>>>>>> branch
>>>>>>>>>> due
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> this issue.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2590 <
>> https://github.com/apache/incubator-airflow/pull/2590>
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 8:14 AM, Driesprong, Fokko
>>>>>>>>>>>>> <[email protected]
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I would like to include AIRFLOW-1611 in the 1.9.0 release:
>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2631 <
>> https://github.com/apache/incubator-airflow/pull/2631>
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Currently importing a custom logging configuration is not
>>>>>> work
>>>>>>> (as
>>>>>>>>>>>>> far
>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>> know). Any feedback on the PR would also be appreciated.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Cheers, Fokko
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 2017-09-25 23:27 GMT+02:00 Chris Riccomini <
>>>>>>> [email protected] <mailto:[email protected]>
>>>>>>>>>>> :
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Done!
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Mon, Sep 25, 2017 at 5:11 AM, Michael Crawford <
>>>>>>>>>>>>>>>>> [email protected] <mailto:
>> [email protected]>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Can you slide the aws and emr connection type fix in?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636 <
>> https://issues.apache.org/jira/browse/AIRFLOW-1636> <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636 <
>> https://issues.apache.org/jira/browse/AIRFLOW-1636>>
>>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2626 <
>> https://github.com/apache/incubator-airflow/pull/2626> <
>>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2626 <
>> https://github.com/apache/incubator-airflow/pull/2626>>
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> It keeps the connection type from getting blanked out on
>>>>>> edit
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>> types.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Sep 21, 2017, at 1:27 PM, Chris Riccomini <
>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Absolutely. Just cherry-picked. I've been looking
>> forward
>>>>>> to
>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>> fixes!
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 5:23 PM, Alex Guziel <
>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>
>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>>> invalid
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Can we get this in?
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1519 <
>> https://issues.apache.org/jira/browse/AIRFLOW-1519>
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1621 <
>> https://issues.apache.org/jira/browse/AIRFLOW-1621>
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/commit/ <
>> https://github.com/apache/incubator-airflow/commit/>
>>>>>>>>>>>>>>>>>>>> b6d2e0a46978e93e16576604624f57d1388814f2
>>>>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/commit/ <
>> https://github.com/apache/incubator-airflow/commit/>
>>>>>>>>>>>>>>>>>>>> 656d045e90bf67ca484a3778b2a07a419bfb324a
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> It speeds up loading times a lot, so it's a good thing
>> to
>>>>>>>>>>>>> have
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> 1.9.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 11:14 AM, Chris Riccomini <
>>>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Sounds good. I'll plan on stable+beta next week, then.
>>>>>>>>>>>>> Initial
>>>>>>>>>>>>>>>>> warning
>>>>>>>>>>>>>>>>>>>>> stands, that I will start locking down what can get
>> into
>>>>>>>>>>>>> 1.9.0
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> point.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 11:10 AM, Bolke de Bruin <
>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> No vote indeed, just to gather feedback on a
>> particular
>>>>>>>>>>>>> fixed
>>>>>>>>>>>>>>>> point
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> time. It also gives a bit more trust to a tarball
>> than
>>>>>> to
>>>>>>> a
>>>>>>>>>>>>>> git
>>>>>>>>>>>>>>>>> pull.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Bolke
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On 20 Sep 2017, at 20:09, Chris Riccomini <
>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> I can do a beta. Is the process significantly
>>>>>> different?
>>>>>>>>>>>>>> IIRC,
>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>> basically the same, just no vote, right?
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 10:56 AM, Bolke de Bruin <
>>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Are you sure you want to go ahead and do RCs right
>>>>>> away?
>>>>>>>>>>>>>>> Isn’t a
>>>>>>>>>>>>>>>>>>>> beta
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> bit smarter?
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> - Bolke
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On 20 Sep 2017, at 19:41, Chris Riccomini <
>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> I want to send out a warning that I'm planning to
>>>>>> cut
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> stable
>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>> next week, and begin the RC1 release vote. Once
>> the
>>>>>>>>>>>>> stable
>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> cut, I
>>>>>>>>>>>>>>>>>>>>>>>>> will be locking down what commits get cherry
>> picked
>>>>>>> into
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> branch,
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> will only be doing PRs that are required to get
>> the
>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>> out.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Sep 18, 2017 at 11:19 AM, Chris Riccomini
>> <
>>>>>>>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> An update on the 1.9.0 release. Here are the
>>>>>>>>>>>>> outstanding
>>>>>>>>>>>>>> PRs
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>> slated to be included into 1.9.0:
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> ISSUE ID     |STATUS    |DESCRIPTION
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1617 |Open      |XSS Vulnerability in
>>>>>> Variable
>>>>>>>>>>>>>>>> endpoint
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1611 |Open      |Customize logging in
>>>>>> Airflow
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1605 |Reopened  |Fix log source of local
>>>>>>>>>>>>> loggers
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1604 |Open      |Rename the logger to log
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1525 |Open      |Fix minor LICENSE &
>> NOTICE
>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1499 |In Progres|Eliminate duplicate and
>>>>>>>>>>>>> unneeded
>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1198 |Open      |HDFSOperator to operate
>>>>>> HDFS
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1055 |Open
>>>>>> |airflow/jobs.py:create_dag_ru
>>>>>>>>>>>>> n()
>>>>>>>>>>>>>>>>>>>> exception
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> @on
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1019 |Open      |active_dagruns shouldn't
>>>>>>>>>>>>> include
>>>>>>>>>>>>>>>> paused
>>>>>>>>>>>>>>>>>>>>> DAGs
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1018 |Open      |Scheduler DAG processes
>>>>>> can
>>>>>>>>>>>>> not
>>>>>>>>>>>>>> log
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> stdout
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1015 |Open      |TreeView displayed over
>>>>>> task
>>>>>>>>>>>>>>>> instances
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1013 |Open
>>>>>> |airflow/jobs.py:manage_slas()
>>>>>>>>>>>>>>>> exception
>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> @once
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-976  |Open      |Mark success running
>> task
>>>>>>>>>>>>> causes
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> fail
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-914  |Open      |Refactor
>>>>>>>>>>>>>>>> BackfillJobTest.test_backfill_
>>>>>>>>>>>>>>>>>>>>>>>> examples
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-913  |Open      |Refactor
>>>>>>>>>>>>>>>> tests.CoreTest.test_scheduler_
>>>>>>>>>>>>>>>>>>>> job
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> real
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-912  |Open      |Refactor tests and build
>>>>>>>>>>>>> matrix
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-888  |Open      |Operators should not
>> push
>>>>>>>>>>>>> XComs
>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>> default
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-828  |Open      |Add maximum size for
>> XComs
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-825  |Open      |Add Dataflow semantics
>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-788  |Open      |Context unexpectedly
>>>>>> added to
>>>>>>>>>>>>>> hive
>>>>>>>>>>>>>>>> conf
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> I will be locking down what can get cherry-picked
>>>>>> into
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> 1.9.0
>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>> shortly, so if you have something you want in,
>>>>>> please
>>>>>>>>>>>>> set
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>> to 1.9.0.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> We (at WePay) have deployed 1.9.0 into our dev
>>>>>>> cluster,
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>>> running smoothly for several days.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> ** I could really use help verifying stability.
>> If
>>>>>> you
>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>>>>>> Airflow,
>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>> in your best interest to deploy the 1.9.0 test
>>>>>> branch
>>>>>>>>>>>>>>>> somewhere,
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> verify
>>>>>>>>>>>>>>>>>>>>>>>>>> it's working for your workload. **
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>> 
>> 
>> 

Reply via email to