Re: [VOTE] Graduate the Apache Airflow as a TLP

2018-11-30 Thread Chris Riccomini
> >>> used to author and manage data pipelines; and be it further > >>> > >>> RESOLVED, that the office of "Vice President, Apache Airflow" be > >>> and hereby is created, the person holding such office to > >>> serve at the direc

Re: [DISCUSS] Apache Airflow graduation from the incubator

2018-11-26 Thread Chris Riccomini
+1 as well. I’m glad to see the community is thriving. On Mon, Nov 26, 2018 at 6:11 PM Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > This is great to see happen, it's been a long time coming! Also count me > in, I'll be happy to help on that last push! > > On Mon, Nov 26, 2018 at 5:21

Re: Airflow: Apache Graduation

2018-09-25 Thread Chris Riccomini
nd correct is here: > > http://incubator.apache.org/projects/airflow.html > > > > This is usually done by the Champion, Chris Riccomini. Chris - are > > you still interested to do this? > > > > After that and the maturity questionnaire are in good shape, it's: > &

Re: Sep Airflow Bay Area Meetup @ Google

2018-07-23 Thread Chris Riccomini
@Feng Lu apparently you're listed as an EVENT ORGANIZER on the group. I believe that should allow you to create meetups. If not, can you let me know? On Sun, Jul 22, 2018 at 12:36 PM Ben Gregory wrote: > Will do Feng! > > Also - is there an approximate date we'll know if the hackathon is

Re: [RESULT][VOTE] Airflow 1.10.0rc2

2018-07-18 Thread Chris Riccomini
Thank you for running this one, Bolke. Sincerely! On Wed, Jul 18, 2018 at 11:30 AM Bolke de Bruin wrote: > Hello, > > (Just making sure this works with the IPMC) > > Apache Airflow (incubating) 1.10.0 (based on RC2) has been accepted. > > 4 “+1” binding votes received: > > - Joy Gao (binding) >

Re: Airflow 1.10.0

2018-06-13 Thread Chris Riccomini
Hey Fokko, Sorry that I've been MIA for a while. Just wanted to check in on 1.10, and its current status. Cheers, Chris On Tue, May 1, 2018 at 11:18 PM Driesprong, Fokko wrote: > Hi Bryon, > > We'll be releasing the RC's soon. If there aren't much issues with the > RC's, it will be released

Re: Managed Apache Airflow Service on Google Cloud Platform

2018-05-03 Thread Chris Riccomini
Awesome work, Feng! Very cool. :) On Tue, May 1, 2018 at 10:10 PM, Feng Lu wrote: > Thank you all and looking forward to the many collaborations to come! > > On Tue, May 1, 2018, 17:37 Alex Tronchin-James 949-412-7220 < > alex.n.ja...@gmail.com> wrote: > > > Bravo! >

Re: 1.10.0beta1 now available for download

2018-04-23 Thread Chris Riccomini
Woot! Thanks for taking this on. :) On Mon, Apr 23, 2018 at 12:00 PM, Gerard Toonstra wrote: > fantastic effort, much appreciated! go go go > > On Mon, Apr 23, 2018 at 11:08 AM, Ace Haidrey > wrote: > > > Great work Fokko and Bolke! > > > > Sent from

Re: Give it up for Fokko!

2018-04-13 Thread Chris Riccomini
+1 ! On Fri, Apr 13, 2018 at 10:27 AM, Tao Feng wrote: > Thanks Fokko. Really appreciate your great feedback and your hard work on > helping airflow project thrive:) > > On Fri, Apr 13, 2018 at 9:55 AM, Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > > > Hey all,

Re: Slides / Presentations for Airflow

2018-04-12 Thread Chris Riccomini
> I suspect we will be hearing from Joy, Chris R., et al shortly. Correct. Video editing is going on as we speak. :) On Thu, Apr 12, 2018 at 9:30 AM, Sid Anand wrote: > Great question.. currently, we don't. > > After meet-ups, usually the host of the meetup sends an email to

Re: Podling Report Reminder - April 2018

2018-03-29 Thread Chris Riccomini
Thanks, Sid! On Wed, Mar 28, 2018 at 11:08 PM, Sid Anand wrote: > FYI! > I filled out the podling report due in April. > > https://wiki.apache.org/incubator/April2018 > > Mentors, please don't forget to sign-off!! > > -s > > On Wed, Mar 28, 2018 at 3:19 AM,

Re: Airflow 1.10

2018-03-26 Thread Chris Riccomini
> > I think 1.10 is a good idea. I'm working on this refactoring of the > > > sensor > > > > structure: https://github.com/apache/incubator-airflow/pull/2875 > > > > > > > > Would be awesome to get this in. At my current project we use sensors >

Re: [VOTE] [RESULT] Migrate to Github as primary repo (a.k.a. Gitbox)

2018-03-21 Thread Chris Riccomini
Great, thanks for doing this Ash! On Wed, Mar 21, 2018 at 8:14 AM, Ash Berlin-Taylor < ash_airflowl...@firemirror.com> wrote: > The vote passed with the following +1's, no -1. > > 4+ Binding (PPMC) votes > Ash Berlin-Taylor, Chris Riccomini, Joy Gao, Maxime Beauchemin

Re: Merging PRs, closing Jira tickets (a.k.a New Committer) guide?

2018-03-12 Thread Chris Riccomini
Done! On Sat, Mar 10, 2018 at 6:14 AM, Ash Berlin-Taylor < ash_airflowl...@firemirror.com> wrote: > Could someone add my Confluence user (ashb) to the space as well please? I > can't seem to edit/create pages right now. > > Thanks, > Ash > > > > On 4 Mar 2018, at 00:59, Sid Anand

Re: [VOTE] Migrate to Github as primary repo (a.k.a. Gitbox)

2018-03-12 Thread Chris Riccomini
+1 On Sat, Mar 10, 2018 at 11:18 PM, Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > +1 (binding) > > On Sat, Mar 10, 2018 at 2:03 PM, Beau Barker > wrote: > > > +1 for Github. > > > > Also think that moving to Github issues would be a step in the right > >

Re: Ash Berlin-Taylor joins Apache Airflow as committer and PPMC member

2018-02-28 Thread Chris Riccomini
Welcome! :) On Sun, Feb 25, 2018 at 5:16 PM, Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > Congrats and welcome! > > On Sat, Feb 24, 2018 at 1:21 PM, Naik Kaxil wrote: > > > Congrats Ash ( > > > > On 24/02/2018, 20:23, "fo...@driesprongen.nl on behalf of

Help with podling status report?

2018-02-07 Thread Chris Riccomini
Hey all, If someone has an extra 15m today, could you please fill out: https://wiki.apache.org/incubator/February2018 Anyone is welcome to fill it out (via https://incubator.apache.org/guides/ppmc.html#podling_status_reports): > The PPMC does not have to fill out the report itself; the PPMC is

Re: 答复: About multi user support in airflow

2018-02-02 Thread Chris Riccomini
@song, you might also be interested in the work that Joy Gao is doing to add more access controls to the UI: https://github.com/wepay/airflow-webserver On Thu, Feb 1, 2018 at 8:12 PM, Song Liu wrote: > Hi Trent, > > One way is to deploy one standalone airflow components

Airflow 1.10

2018-01-12 Thread Chris Riccomini
Hey all, After some past discussion on Airflow 1.10 vs 2.0, I think we've converged on a 1.10 as the next step. 1.10 will include: * Timezone changes * Kubernetes support * New UI The first two have been merged in, as I saw Bolke just merged K8s (I saw a few follow-on patches coming, though),

Re: Airflow documentation update

2018-01-03 Thread Chris Riccomini
Hey all, I've updated the docs here: https://airflow.incubator.apache.org/ The documentation publication is here: https://cwiki.apache.org/confluence/display/AIRFLOW/Building+and+deploying+the+docs Cheers, Chris On Wed, Dec 27, 2017 at 11:11 AM, Sid Anand wrote: > Thx

Re: Airflow 1.9.0 is released

2018-01-03 Thread Chris Riccomini
Hey all, I have updated the docs as well: https://airflow.incubator.apache.org/ Cheers, Chris On Wed, Jan 3, 2018 at 9:30 AM, Sid Anand <san...@apache.org> wrote: > Grazzi! > -s > > On Wed, Jan 3, 2018 at 9:03 AM, Chris Riccomini <criccom...@apache.org> > wrote:

Re: Airflow 1.9.0 is released

2018-01-03 Thread Chris Riccomini
t; > > > On Tue, Jan 2, 2018 at 2:52 PM, Andy Loughran <a...@zrmt.com> wrote: > > > > > > > Congratulations guys - lots and lots of voting and you got it over > the > > > > line. > > > > > > > > happy new year! > > > > &

Airflow 1.9.0 is released

2018-01-02 Thread Chris Riccomini
Dear Airflow community, Airflow 1.9.0 was just released. The source release as well as the binary "sdist" release are available here: https://dist.apache.org/repos/dist/release/incubator/ airflow/1.9.0-incubating/ We also made this version available on PyPi for convenience (`pip install

Re: Airflow 2.0

2017-12-19 Thread Chris Riccomini
hion. > > > > --George > > > >> On Thu, Dec 14, 2017 at 2:20 PM Alek Storm <alek.st...@gmail.com> > wrote: > >> > >> We use sqlite for developing Airflow DAGs locally, and doing basic > checks > >> for syntax/import errors in our CI/

Re: [RESULT][VOTE] Airflow 1.9.0rc8

2017-12-18 Thread Chris Riccomini
Thanks to all for helping validate it. :) On Mon, Dec 18, 2017 at 12:48 PM, Ash Berlin-Taylor < ash_airflowl...@firemirror.com> wrote: > Woot! Great news, and thanks for all the time you've spent preparing all > the RCs. > > -ash > > > On 18 Dec 2017, at 20:1

[RESULT][VOTE] Airflow 1.9.0rc8

2017-12-18 Thread Chris Riccomini
Hello, Apache Airflow (incubating) 1.9.0 (based on RC8) has been accepted. 5 “+1” binding votes received: - Maxime Beauchemin (binding) - Chris Riccomini (binding) - Bolke de Bruin (binding) - Joy Gao (binding) - Fokko Driesprong (binding) My next step is to open a thread with the IPMC

Re: [VOTE] Airflow 1.9.0rc8

2017-12-18 Thread Chris Riccomini
om my iPhone > > > On 18 Dec 2017, at 18:51, Chris Riccomini <criccom...@apache.org> wrote: > > > > https://github.com/apache/incubator-airflow/pull/2889 > > > > On Mon, Dec 18, 2017 at 9:44 AM, Chris Riccomini <criccom...@apache.org> > > wrote: &

Re: [VOTE] Airflow 1.9.0rc8

2017-12-18 Thread Chris Riccomini
reef Bolke de Bruin <bdbr...@gmail.com> > >>> > >>>> +1, binding > >>>> > >>>> Checked sigs, version, source is there (did not check build), bin is > >>> there. > >>>> > >>>> Bolke > >>>&

[VOTE] Airflow 1.9.0rc8

2017-12-15 Thread Chris Riccomini
Hey all, (Last time, I hope)^2 I have cut Airflow 1.9.0 RC8. This email is calling a vote on the release, which will last for 72 hours. Consider this my (binding) +1. Airflow 1.9.0 RC8 is available at: https://dist.apache.org/repos/dist/dev/incubator/airflow/1.9.0rc8/

Re: [CANCEL][VOTE] Airflow 1.9.0rc7

2017-12-15 Thread Chris Riccomini
vanaf mijn iPad > > > Op 15 dec. 2017 om 23:19 heeft Chris Riccomini <criccom...@apache.org> > het volgende geschreven: > > > > FML. Apparently +incubating should not be there due to Pypi issues. So > > sorry about this. What a mess. > > > > Canc

Re: Release question

2017-12-15 Thread Chris Riccomini
anaf mijn iPad > > > Op 15 dec. 2017 om 22:32 heeft Chris Riccomini <criccom...@apache.org> > het volgende geschreven: > > > > No worries. It gives me a chance to fix this: > > > > https://github.com/apache/incubator-airflow/pull/2884 > > > > C

[CANCEL][VOTE] Airflow 1.9.0rc7

2017-12-15 Thread Chris Riccomini
FML. Apparently +incubating should not be there due to Pypi issues. So sorry about this. What a mess. Canceling. On Fri, Dec 15, 2017 at 2:18 PM, Chris Riccomini <criccom...@apache.org> wrote: > Hey all, > > (Last time, I hope) > > I have cut Airflow 1.9.0 RC7. This em

Re: Release question

2017-12-15 Thread Chris Riccomini
mijn iPad > > > Op 15 dec. 2017 om 22:17 heeft Chris Riccomini <criccom...@apache.org> > het volgende geschreven: > > > > The artifact version has an rc6 in it. I suppose we need to re-vote. > Sigh. > > > >> On Fri, Dec 15, 2017 at 1:02 PM, Bolke de B

Re: Release question

2017-12-15 Thread Chris Riccomini
ght be off a little. > > Cheers > Bolke > > Verstuurd vanaf mijn iPad > > > Op 15 dec. 2017 om 21:43 heeft Chris Riccomini <criccom...@apache.org> > het volgende geschreven: > > > > Hey all, > > > > For 1.9.0, RC6 has passed the vote. Does anyon

Release question

2017-12-15 Thread Chris Riccomini
Hey all, For 1.9.0, RC6 has passed the vote. Does anyone recall what the protocol is for releasing it? I believe I need to re-build the artifacts to remove the 'rc6' from the release. IIRC, we don't need to re-vote for this, do we? Cheers, Chris

Re: Airflow 2.0

2017-12-14 Thread Chris Riccomini
mport side of things, > isn't > > Airflow 2 the best moment to change to a pip plugin system for imports of > > third party stuff? > > I.E being able to add a new type of credentials, operator etc.. without > > touching to the airflow code itself or having them in a specia

Re: Airflow 2.0

2017-12-14 Thread Chris Riccomini
I have created a wiki here: https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0 To track features and progress. On Thu, Dec 14, 2017 at 11:08 AM, Chris Riccomini <criccom...@apache.org> wrote: > > Re: #2: Is there a current ticket out for removing the legacy im

Re: Airflow 2.0

2017-12-14 Thread Chris Riccomini
cated features I imagine the docs will change > substantially. > > Re: #2: Is there a current ticket out for removing the legacy import style? > I'm happy to help drive that forward. > > - Andy > > On Thu, Dec 14, 2017 at 1:55 PM, Chris Riccomini <criccom...@apache.org&

Re: Airflow 2.0

2017-12-14 Thread Chris Riccomini
eprecated features? > > Verstuurd vanaf mijn iPad > > > Op 14 dec. 2017 om 19:55 heeft Chris Riccomini <criccom...@apache.org> > het volgende geschreven: > > > > Hey all, > > > > With 1.9.0 wrapping up soon (hopefully), there's been some discussion o

Airflow 2.0

2017-12-14 Thread Chris Riccomini
Hey all, With 1.9.0 wrapping up soon (hopefully), there's been some discussion on the having the next release be Airflow 2.0 (rather than 1.10). This would allow us to break compatibility, and clean up some stuff. Proposed things to include in 2.0 are: 1) New webserver that Joy Gao has been

Re: SAML2.0 authentication for airflow

2017-12-14 Thread Chris Riccomini
> > scheduled: new web ui, time zone support I think it would make sense. > Maybe > > we should move some api stuff out of experimental as well. > > > > B. > > > > Verstuurd vanaf mijn iPad > > > > > Op 13 dec. 2017 om 20:14 heeft Ch

Re: SAML2.0 authentication for airflow

2017-12-13 Thread Chris Riccomini
When you say "we're going to be moving over to it", do you mean that it's > > in the official roadmap? > > > > Thanks, > > Vincent > > > > On 2017-12-12 17:53, Chris Riccomini <criccom...@apache.org> wrote: > > > (Note, FAB = https://g

Re: SAML2.0 authentication for airflow

2017-12-12 Thread Chris Riccomini
(Note, FAB = https://github.com/dpgaspar/Flask-AppBuilder) On Tue, Dec 12, 2017 at 8:52 AM, Chris Riccomini <criccom...@apache.org> wrote: > If you are looking at implementing this, I'd suggest looking at FAB and > https://github.com/wepay/airflow-webserver. FAB has a much cleane

Re: SAML2.0 authentication for airflow

2017-12-12 Thread Chris Riccomini
If you are looking at implementing this, I'd suggest looking at FAB and https://github.com/wepay/airflow-webserver. FAB has a much cleaner security model than the current UI, and we're going to be moving over to it. If FAB doesn't support SAML 2.0 (I don't think it does), I think patching it to do

[VOTE] Airflow 1.9.0rc6

2017-12-11 Thread Chris Riccomini
Hey all, I have cut Airflow 1.9.0 RC6. This email is calling a vote on the release, which will last for 72 hours. Consider this my (binding) +1. Airflow 1.9.0 RC6 is available at: https://dist.apache.org/repos/dist/dev/incubator/airflow/1.9.0rc6/

[CANCEL][VOTE] Airflow 1.9.0rc5

2017-12-10 Thread Chris Riccomini
gt; > > Will do not done yet. > > > > B. > > > > Sent from my iPhone > > > >> On 7 Dec 2017, at 23:47, Chris Riccomini <criccom...@apache.org> wrote: > >> > >> Sure, LMK when the patch is merged, and I can start RC6. :) > >

Re: [VOTE] Airflow 1.9.0rc5

2017-12-07 Thread Chris Riccomini
hone > > > On 7 Dec 2017, at 18:57, Chris Riccomini <criccom...@apache.org> wrote: > > > > Hey all, > > > > I have cut Airflow 1.9.0 RC5. This email is calling a vote on the > release, > > which will last for 72 hours. Consider this my (binding) +1. &g

Re: [VOTE] Airflow 1.9.0rc5

2017-12-07 Thread Chris Riccomini
d to know if they passed. > > > > B. > > > > Sent from my iPhone > > > >> On 7 Dec 2017, at 18:57, Chris Riccomini <criccom...@apache.org> wrote: > >> > >> Hey all, > >> > >> I have cut Airflow 1.9.0 RC5. This

[CANCEL][VOTE] Airflow 1.9.0rc4

2017-12-04 Thread Chris Riccomini
. > > Cheers > Bolke > > Verstuurd vanaf mijn iPad > > > Op 1 dec. 2017 om 20:09 heeft Chris Riccomini <criccom...@apache.org> > het volgende geschreven: > > > > Hey all, > > > > My current understanding is that we're going to ca

Re: [VOTE] Airflow 1.9.0rc4

2017-12-01 Thread Chris Riccomini
-airflow/pull/2831 > >> > >> Bolke > >> > >>> On 30 Nov 2017, at 20:53, Bolke de Bruin <bdbr...@gmail.com> wrote: > >>> > >>> Ill have a patch in an hour or so > >>> > >>> > >>>> On 30 Nov 2017, a

Re: [VOTE] Airflow 1.9.0rc4

2017-11-30 Thread Chris Riccomini
Bolke de Bruin <bdbr...@gmail.com> > wrote: > >> > >>> We saw this issue around RC2, so I’m surprised that it turns up again. > We > >>> discussed it at lengths with Fokko and Joy. It might be that we somehow > >>> missed a commit, but I need

Re: [VOTE] Airflow 1.9.0rc4

2017-11-30 Thread Chris Riccomini
ook with fresh eyes > and a clean slate tomorrow. Someone confirming if they do or don't see logs > from inside an operator would help me. > > -ash > > > > On 30 Nov 2017, at 17:03, Chris Riccomini <criccom...@apache.org> wrote: > > > > @Ash, do you

Re: [VOTE] Airflow 1.9.0rc4

2017-11-30 Thread Chris Riccomini
is in turn means that any log calls end up not writing anything as > our FileTaskHandler's self.handler property is None, meaning it doesn't > emit anything. > >>> > >>> Does this ring any bells to anyone? > >>> > >>> > >>>> On 30 Nov

Re: Kubernetes Executor

2017-11-29 Thread Chris Riccomini
https://github.com/apache/incubator-airflow/pull/2414 On Tue, Nov 28, 2017 at 10:44 PM, Niranda Perera wrote: > Hi all, > > I am interested in this discussion on the Kubernetes executor [1]. > > Is there any way to access the current work that has been done on this >

Re: Data lineage and data portal

2017-11-27 Thread Chris Riccomini
Interested On Mon, Nov 27, 2017 at 12:07 PM, Kerr Shireman wrote: > I am interested. I remember being pretty excited when I read that blog > post. > On Mon, Nov 27, 2017 at 2:00 PM Arthur Wiedmer > wrote: > > > Likewise! > > > > Best, > > Arthur >

[CANCEL][VOTE] Airflow 1.9.0rc3

2017-11-27 Thread Chris Riccomini
Canceling due to S3Hook bug AIRFLOW-1839 On Mon, Nov 27, 2017 at 10:49 AM, Chris Riccomini <criccom...@apache.org> wrote: > I am going to cancel and start a new vote. Broken S3 hook is not > acceptable for a 1.X release, IMO. Hopefully we can get the release out > this week. >

Re: [VOTE] Airflow 1.9.0rc3

2017-11-27 Thread Chris Riccomini
ey for k in response['Contents']] if > response.get('Contents') else None > > AttributeError: 'dict' object has no attribute 'Key' > > > > Reported as https://issues.apache.org/jira/browse/AIRFLOW-1839 > > > >> On 18 Nov 2017, at 17:36, Chris Riccomini <criccom...@apache.o

Re: [notice] merging time zone awareness

2017-11-27 Thread Chris Riccomini
:thumbsup: On Mon, Nov 27, 2017 at 9:40 AM, Bolke de Bruin wrote: > Reminder, I’ll be merging today if no concerns, remarks etc. > > Cheers > Bolke > > > On 22 Nov 2017, at 20:54, Bolke de Bruin wrote: > > > > All, > > > > Just wanted to make you aware

Re: RBAC Update

2017-11-18 Thread Chris Riccomini
Bolke > > > On 18 Nov 2017, at 00:47, Chris Riccomini <criccom...@apache.org> wrote: > > > > Hey all, > > > > I know the weekend is coming up, and for those of us in the US, next week > > is a bit of a slow holiday week. Would love to get some feed

Re: RBAC Update

2017-11-17 Thread Chris Riccomini
Hey all, I know the weekend is coming up, and for those of us in the US, next week is a bit of a slow holiday week. Would love to get some feedback from everyone on this. The goal would ideally to be to converge on this and eventually replace the existing Airflow UI with this one. Cheers, Chris

Re: [VOTE] Airflow 1.9.0rc2

2017-11-16 Thread Chris Riccomini
our changelog seems incomplete (some more fixes were included in RC2). > > > > But apart from that > > > > +1, binding > > > > Bolke > > > > > >> On 13 Nov 2017, at 19:03, Chris Riccomini <criccom...@apache.org> > wrote: > >&g

[CANCEL][VOTE] Airflow 1.9.0rc1

2017-11-10 Thread Chris Riccomini
Hey all, I'm going to cancel the RC1 vote due to the -1's. I'm tracking the following issues for RC2: https://issues.apache.org/jira/browse/AIRFLOW-1776 https://issues.apache.org/jira/browse/AIRFLOW-1794 https://issues.apache.org/jira/browse/AIRFLOW-1789

Re: [VOTE] Airflow 1.9.0rc1

2017-11-08 Thread Chris Riccomini
Anyone? :/ On Mon, Nov 6, 2017 at 1:22 PM, Chris Riccomini <criccom...@apache.org> wrote: > Hey all, > > I have cut Airflow 1.9.0 RC1. This email is calling a vote on the release, > which will last fo 72 hours. Consider this my (binding) +1. > > Airflow 1.9.0 RC1 is

[VOTE] Airflow 1.9.0rc1

2017-11-06 Thread Chris Riccomini
Hey all, I have cut Airflow 1.9.0 RC1. This email is calling a vote on the release, which will last fo 72 hours. Consider this my (binding) +1. Airflow 1.9.0 RC1 is available at: https://dist.apache.org/repos/dist/dev/incubator/airflow/1.9.0rc1/ apache-airflow-1.9.0rc1+incubating-source.tar.gz

Re: Airflow 1.9.0 status

2017-11-03 Thread Chris Riccomini
ll we get it before the weekend? Oh the suspense ;-) > > Sent from my iPhone > > > On 2 Nov 2017, at 17:17, Chris Riccomini <criccom...@apache.org> wrote: > > > > @mC, yes it will. > > > > @Bolke, awesome! Will cut an RC today/tomorrow. :) Very excit

Re: Airflow 1.9.0 status

2017-11-02 Thread Chris Riccomini
> Subject: Re: Airflow 1.9.0 status > > I think we are ready for RC. > > Bolke > > > > On 31 Oct 2017, at 21:30, Chris Riccomini <criccom...@apache.org> wrote: > > > > Great, sounds like this is the last remaining JIRA, then? > > > > https

Re: Airflow 1.9.0 status

2017-10-31 Thread Chris Riccomini
..@gmail.com> > wrote: > > > > Is this bug issue on DAG scheduling > > <https://issues.apache.org/jira/browse/AIRFLOW-1767> something that > needs > > resolution before the RC? > > > > On Tue, Oct 31, 2017 at 1:17 PM, Chris Riccomini <criccom...@apache.org&

Re: DAG logging

2017-10-31 Thread Chris Riccomini
. On Tue, Oct 31, 2017 at 1:11 PM, Chris Riccomini <criccom...@apache.org> wrote: > @Bolke, I want them to end up in the DAG processor log. > > On Tue, Oct 31, 2017 at 1:03 PM, Bolke de Bruin <bdbr...@gmail.com> wrote: > >> If the print from >> >> https://gi

Re: DAG logging

2017-10-31 Thread Chris Riccomini
> > > Op 31 okt. 2017 20:56 schreef "Bolke de Bruin" <bdbr...@gmail.com>: > > > >> Where do you want those to end up? As they are (probably) evaluated > during > >> parsing, they will end up in the log of the parsing process. So dag > >

Re: DAG logging

2017-10-31 Thread Chris Riccomini
st "new logging" - apparently we cannot use logging any more and > have to init log handler. > > On Tue, Oct 31, 2017 at 1:54 PM, Chris Riccomini <criccom...@apache.org> > wrote: > > > Correction: > > > > import logging > > > > class DqRowCheckOp

Re: DAG logging

2017-10-31 Thread Chris Riccomini
Correction: import logging class DqRowCheckOperator(BaseOperator): ... def execute(...): logging.info('foo') ... It's an operator that we're using. The 'foo' doesn't show up in the logs in the UI or file. On Tue, Oct 31, 2017 at 10:47 AM, Chris Riccomini <criccom...@apache.org>

DAG logging

2017-10-31 Thread Chris Riccomini
Hey all, Just noticed when we upgraded to 1.9.0 that logging from our custom operators are no longer visible in the file. Assuming this is due to all the log changes that were made in 1.9.0. Our custom operators just have: import logging class DbDagBuilder(object): ... logging.info('foo')

Re: Airflow 1.9.0 status

2017-10-31 Thread Chris Riccomini
lke. > > > > On 30 Oct 2017, at 19:51, Chris Riccomini <criccom...@apache.org> wrote: > > > > I marked them as blocker. Still would like to understand how important > this > > is, though. The API is experimental, so I could see an argument for them &g

Re: Airflow 1.9.0 status

2017-10-30 Thread Chris Riccomini
t; > -ash > > > On 30 Oct 2017, at 17:58, Chris Riccomini <criccom...@apache.org> wrote: > > > > Hey all, > > > > Current blockers for 1.9.0: > > > > https://issues.apache.org/jira/browse/AIRFLOW-1711 > > https://issues.apache.org/jira/bro

Re: Airflow 1.9.0 status

2017-10-30 Thread Chris Riccomini
Hey all, Current blockers for 1.9.0: https://issues.apache.org/jira/browse/AIRFLOW-1711 https://issues.apache.org/jira/browse/AIRFLOW-1018 Cheers, Chris On Tue, Oct 24, 2017 at 11:06 AM, Chris Riccomini <criccom...@apache.org> wrote: > Hey all, > > Talked with @Bolke

Re: Moving on to Lyft

2017-10-26 Thread Chris Riccomini
Congrats, Max! On Thu, Oct 26, 2017 at 7:32 AM, Andy Hadjigeorgiou wrote: > Congrats! Best of luck in your new role! > > On Thu, Oct 26, 2017 at 10:28 AM, Laura Lorenz > wrote: > > > Congrats!! > > > > On Thu, Oct 26, 2017 at 10:21 AM, Bolke de

Re: ImportError: No module named sendgrid

2017-10-25 Thread Chris Riccomini
Let's do it as a plugin. I merged it. On Wed, Oct 25, 2017 at 9:20 AM, Feng Lu wrote: > Sorry for the glitch. > > I am the author of this commit, I agree that if sendgrid is not used, it > doesn't need to be installed. > That being said, I can submit a quick fix to

Re: Airflow 1.9.0 status

2017-10-24 Thread Chris Riccomini
nal test and updating the docs a bit. Hopefully > > somewhere in the next few days, however I'm a bit busy with the Spark > > Summit. > > > > Cheers, Fokko > > > > 2017-10-19 21:03 GMT+02:00 Bolke de Bruin <bdbr...@gmail.com>: > > > >> Im plan

1.9.0alpha1 published

2017-10-13 Thread Chris Riccomini
Hey all, I have cut a 1.9.0alpha1 release of Airflow. You can download it here: https://dist.apache.org/repos/dist/dev/incubator/airflow/1.9.0alpha1/ The bin tarball can be installed with pip: pip install apache-airflow-1.9.0alpha1+incubating-bin.tar.gz The goal is to have the community

Re: Airflow 1.9.0 status

2017-10-09 Thread Chris Riccomini
e team is readying a release of one of our core > products. So it will probably will be after this week when I get my hands > dirty again. > > Cheers > Bolke > > Verstuurd vanaf mijn iPad > > > Op 4 okt. 2017 om 23:35 heeft Chris Riccomini <criccom...@apa

Re: Airflow 1.9.0 status

2017-10-04 Thread Chris Riccomini
Hey all, Checking in. Has anyone tested this in their environments? Stable releases depend on community involvement. Cheers, Chris On Tue, Oct 3, 2017 at 10:30 PM, Chris Riccomini <criccom...@apache.org> wrote: > I've added AIRFLOW-1635 to the v1-9-test branch. It's not

Re: Apache Airflow welcome new committer/PMC member : Fokko Driespong (a.k.a. fokko)

2017-10-04 Thread Chris Riccomini
Welcome!! On Wed, Oct 4, 2017 at 12:51 PM, Sid Anand wrote: > Folks, > Please join the Apache Airflow PMC in welcoming its newest member and > co-committer, Fokko Driespong (a.k.a. fokko ). > > https://cwiki.apache.org/confluence/display/AIRFLOW/ >

Re: Airflow 1.9.0 status

2017-10-03 Thread Chris Riccomini
FLOW-1055 |Bug |airflow/jobs.py:create_dag_run() exception > > for > > @on > > AIRFLOW-1018 |Bug |Scheduler DAG processes can not log to > stdout > > AIRFLOW-1013 |Bug |airflow/jobs.py:manage_slas() exception for > > @once > > AIRFLOW-976

Re: Airflow 1.9.0 status

2017-10-02 Thread Chris Riccomini
eers, Chris On Fri, Sep 29, 2017 at 3:54 PM, Chris Riccomini <criccom...@apache.org> wrote: > Welp. Work got in the way, so I'll cut the beta on Monday. :) > > On Thu, Sep 28, 2017 at 1:32 PM, Chris Riccomini <criccom...@apache.org> > wrote: > >> Works for me. Will

Re: Airflow 1.9.0 status

2017-09-29 Thread Chris Riccomini
Welp. Work got in the way, so I'll cut the beta on Monday. :) On Thu, Sep 28, 2017 at 1:32 PM, Chris Riccomini <criccom...@apache.org> wrote: > Works for me. Will try and cut a beta before end of week. > > Blockers for 1.9.0 are: > > AIRFLOW-1611 |Bug |Customi

Re: Airflow 1.9.0 Bug Fix Inclusion Requests

2017-09-29 Thread Chris Riccomini
They are both already in 1.9.0. On Thu, Sep 28, 2017 at 6:15 PM, David Klosowski wrote: > Hi Airflow Devs: > > Is there any chance that we can include AIRFLOW-1397 and AIRFLOW-1483 as > part of the 1.9.0 release? > > We've noticed some UI changes after 1.8.0 and it affects

Re: Airflow 1.9.0 status

2017-09-28 Thread Chris Riccomini
s a point of reference. > > In addition the list below are mostly longer standing issues that are also > part of the 1.8.x branch. Maybe only consider 1611, 1525, 1258, and 976 as > blocker? > > Cheers > Bolke > > Verstuurd vanaf mijn iPad > > > Op 28 sep. 2017 om 19:49 he

Re: Airflow 1.9.0 status

2017-09-28 Thread Chris Riccomini
ata Engineer > cjo...@simpli.fi | M: 972.821.7631 > __ > > > Programmatic Performance.* Localized.* > __ > > 1407 Texas Street | Suite 202 | Fort Worth, TX 76102 > 800.840.0768 | www.simpli.fi > > > On Wed,

Re: Possible Bug (?) in BigQueryOperator - Missing data when writing to a partitioned table

2017-09-27 Thread Chris Riccomini
omparing apples with oranges (same query, same data, different > libraries). Although it really puzzles me how a different library can yield > different (read as: some is missing) data – when it’s job is just to > execute a query and not pulling and transforming it. > > > On 27

Re: Possible Bug (?) in BigQueryOperator - Missing data when writing to a partitioned table

2017-09-27 Thread Chris Riccomini
ues will be closed. Any issue may be reopened if it is causing ongoing problems. Looks like we might want to migrate at some point. It'll be a big change. <https://github.com/google/google-api-python-client#about> On Wed, Sep 27, 2017 at 10:41 AM, Chris Riccomini <criccom...@apach

Re: Possible Bug (?) in BigQueryOperator - Missing data when writing to a partitioned table

2017-09-27 Thread Chris Riccomini
see if the library could be the problem. > > On 27.09.2017, 19:15, "Chris Riccomini" <criccom...@apache.org> wrote: > > Is it possible that you were getting a cache hit with the BQ operator? > > https://cloud.google.com/bigquery/docs/cached-results# > big

Re: Airflow 1.9.0 status

2017-09-27 Thread Chris Riccomini
low/pull/2631 > > > > Currently importing a custom logging configuration is not work (as far > as I > > know). Any feedback on the PR would also be appreciated. > > > > Cheers, Fokko > > > > > > 2017-09-25 23:27 GMT+02:00 Chris Riccomini &

Re: Some random fun

2017-09-25 Thread Chris Riccomini
Lol. :thumbsup: On Mon, Sep 25, 2017 at 12:41 PM, Boris Tyukin wrote: > this is nerdy!! :) thanks for sharing! > > On Mon, Sep 25, 2017 at 3:00 PM, Dan Davydov invalid > > wrote: > > > Haha, this is great. > > > > On Mon, Sep 25, 2017 at 11:37

React MIT license

2017-09-25 Thread Chris Riccomini
Hey all, Wanted to call out that it looks like React is going to switch over to the MIT license: https://code.facebook.com/posts/300798627056246 I know that there have been discussions both within the Airflow community and Apache at large about whether React was acceptable to use due to its old

Re: Airflow 1.9.0 status

2017-09-21 Thread Chris Riccomini
1.9. > > On Wed, Sep 20, 2017 at 11:14 AM, Chris Riccomini <criccom...@apache.org> > wrote: > > > Sounds good. I'll plan on stable+beta next week, then. Initial warning > > stands, that I will start locking down what can get into 1.9.0 at that > > point.

Re: Airflow 1.9.0 status

2017-09-20 Thread Chris Riccomini
xed point in > time. It also gives a bit more trust to a tarball than to a git pull. > > Bolke > > > On 20 Sep 2017, at 20:09, Chris Riccomini <criccom...@apache.org> wrote: > > > > I can do a beta. Is the process significantly different? IIRC, it's > > basi

Re: Airflow 1.9.0 status

2017-09-20 Thread Chris Riccomini
> > - Bolke > > > On 20 Sep 2017, at 19:41, Chris Riccomini <criccom...@apache.org> wrote: > > > > Hey all, > > > > I want to send out a warning that I'm planning to cut the stable branch > > next week, and begin the RC1 release vote. Once the stable bran

Re: Airflow 1.9.0 status

2017-09-20 Thread Chris Riccomini
. Cheers, Chris On Mon, Sep 18, 2017 at 11:19 AM, Chris Riccomini <criccom...@apache.org> wrote: > Hey all, > > An update on the 1.9.0 release. Here are the outstanding PRs that are > slated to be included into 1.9.0: > > ISSUE ID |STATUS|DESCRIPTION > A

Airflow 1.9.0 status

2017-09-18 Thread Chris Riccomini
Hey all, An update on the 1.9.0 release. Here are the outstanding PRs that are slated to be included into 1.9.0: ISSUE ID |STATUS|DESCRIPTION AIRFLOW-1617 |Open |XSS Vulnerability in Variable endpoint AIRFLOW-1611 |Open |Customize logging in Airflow AIRFLOW-1605 |Reopened |Fix

Re: 1.9.0 test branch has been cut

2017-09-13 Thread Chris Riccomini
Google Pubsub hook and operator |1 Cheers, Chris On Wed, Sep 13, 2017 at 2:01 PM, Chris Riccomini <criccom...@apache.org> wrote: > Note: Everything on master up to the point I cut the branch is included in > 1.9.0. I'm not sure what's up with the `dev/airflow-jira compare 1.9.0` >

  1   2   3   >