Re: Airflow HA @ING

Cool, Thanks Johan for creating it ! I am sure it will save lot of time for lot of people.. Thanks for Sharing Bolke On Tue, May 30, 2017 at 12:32 PM, Bolke de Bruin wrote: > Hi, > > Just wanted to let you know that one of my team members, Johan Witman, has > been writing

Re: How to query a database and then send an email with the query results?

Hi Anne, I would go with first approach unless the result can be very big, since all xcom results gets stored in airflow db. another approach would be to use MySQLOperator + EmailOperator within pythonOperator as a wrapper. you ll have to execute the operator manually in the code though. if

Re: Airflow Meetup @ Paypal (San Jose)

uot;Advanced Data Engineering patterns with Apache Airflow". > > I thought you had many presenters already, no? > > I can fill in if needed. > > Max > > On Thu, Mar 9, 2017 at 7:51 PM, Jayesh Senjaliya <jhsonl...@gmail.com> > wrote: > > > Hi Max, > >

Re: Airflow Meetup @ Paypal (San Jose)

. > > org/msg01777.html > > > > If you feel you didn't get enough results, it might be a good idea for us > > to kick that off again. > > -s > > > > On Fri, Feb 3, 2017 at 10:34 AM, Jayesh Senjaliya <jhsonl...@gmail.com> > > wrote: > > > >

Re: Help needed: Travis builds failing - psycopg2 and ldap3

Hi Bolke, I can help with ldap issue. I have faced same issue while trying to integrate with ldap earlier, and I tracked down to the same get_connection function you mention in the ticket. I think its really due to there difference between implementation of python 2 vs python3 lib. anyway, I

Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc4

+1 ( non-binding )works fine for me. On Fri, Feb 17, 2017 at 3:37 PM, Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > +1 (binding) > > On Fri, Feb 17, 2017 at 11:33 AM, Dan Davydov < > dan.davy...@airbnb.com.invalid> wrote: > > > +1 (binding). Mark success works great now, thanks

Re: Airflow Meetup @ Paypal (San Jose)

t the SJ Convention Center this year. It's > still > > > very close.. 5 miles (12 minutes - reverse commute). > > > > > > https://goo.gl/maps/nwxmkYsNFKQ2 > > > > > > BTW, I heard Bolke may be attending ;-) Bolke, would you like to speak > at > &

Re: Airflow 1.8.0 Release Candidate 1

Thank You Bolke for all the efforts you are putting in !! I have deployed this RC now. On Thu, Feb 2, 2017 at 3:02 PM, Jeremiah Lowin wrote: > Fantastic work on this Bolke, thank you! > > We've deployed the RC and will report if there are any issues... > > On Thu, Feb 2,

Re: Airflow Meetup @ Paypal (San Jose)

n help promote it. > I suggest picking a start time after the Strata talks end but not super > late either. > > -s > > On Tue, Jan 31, 2017 at 9:19 AM, Jayesh Senjaliya <jhsonl...@gmail.com> > wrote: > > > HI All, > > > > √ I have approval from Paypal t

Re: Airflow Meetup @ Paypal (San Jose)

com> wrote: > Is someone from Paypal likely to speak? Should we start a new thread with a > call for another speaker? There was mention of three being needed. > > On Wed, Jan 25, 2017 at 5:33 PM Jayesh Senjaliya <jhsonl...@gmail.com> > wrote: > > > Yes I am waitin

Re: possible bug: scheduler dont schedule manual triggered dag on 1.8.b2

lke de Bruin <bdbr...@gmail.com bdbr...@gmail.com>> wrote: > >> > >> Thanks for tracking this down Jayesh. I suspect not wanting ‘backfills’ > is the reason for it (ie. backfills are not scheduled yet), but I need to > look through the code. Maybe Dan or Paul can

possible bug: scheduler dont schedule manual triggered dag on 1.8.b2

I am running airflow 1.8.b2 a week before I had perfectly running dag which i put it on 1.8.b2 and I find that scheduler is not scheduling the task even though the dag run was created. I debugged the issue and came down to this piece of code where scheduler finds task to schedule TI =

Re: Airflow 1.7.1.3 on Mac OS X Sierra - scheduler not triggering DAGs

Hi Kerr and Bolke, I am hitting "scheduler not scheduling task" but I am on 1.8.b2, and i have debugged to find the root cause. let me send separate email though describing the issue i am facing. Thanks Jayesh On Tue, Jan 24, 2017 at 2:11 PM, Kerr Shireman wrote: >

Re: testing the latest version of Airflow 1.8

Hi Ali, I would take it from Bolke's upload here and install it from src : https://dist.apache.org/repos/dist/dev/incubator/airflow/ -Jayesh On Tue, Jan 24, 2017 at 12:28 PM, Ali Naqvi wrote: > Hi folks, > I would like to test out the beta version of Airflow 1.8. > >

Re: Airflow Meetup @ Paypal (San Jose)

il.com> > wrote: > > > I think if we hold it in the evening, there is no requirement to buy a > > ticket to come to the meetup. Let me verify. > > > > On Fri, Jan 20, 2017 at 12:45 PM, Jayesh Senjaliya <jhsonl...@gmail.com> > > wrote: >

Re: Airflow Meetup @ Paypal (San Jose)

Hi Russell, Sure, Strata will have its own flavor of visitors, but the tickets are kind of expensive too for everybody to join. I agree on turnouts though, so we can try for Strata first and fallback to regular meetup in March end or even April if we dont get space in Strata. or we can just do

Re: New book covers Airflow with PySpark: Agile Data Science 2.0 (O'Reilly, 2017) AND Airflow Meetup?

Let me email about this with its own email subject. On Thu, Jan 19, 2017 at 10:54 PM Jayesh Senjaliya <jhsonl...@gmail.com> wrote: > Hi Siddharth, > > I am Jayesh from Paypal, and at last meetup we briefly talked about > hosting next one and I offered to host next Airflo

Re: New book covers Airflow with PySpark: Agile Data Science 2.0 (O'Reilly, 2017) AND Airflow Meetup?

Hi Siddharth, I am Jayesh from Paypal, and at last meetup we briefly talked about hosting next one and I offered to host next Airflow meetup at Paypal office. If we can come up with some dates, I can talk to facilities and reserve space accordingly. Please let me know. Thanks Jayesh On Thu,

Re: LDAP backend support for direct bind

+1, useful feature for enterprises. On Fri, Jan 6, 2017 at 9:01 PM, Erich Hochmuth wrote: > Would there be interest in updating the LDAP backend to support two new > features: > >- direct bind searching using the credentials from those of the >user authenticating

getting more fixes in Airflow-1.8 release?

Hi Guys, should we get all the patches in Airflow-1.8 release, at least those reviewed ones? I have some PR and can help on test and review others. any thoughts ? Thanks Jayesh