v2.0 Tentative Release Plan

2018-06-04 Thread vardanguptacse
Hi Team, Could you please provide details on when are we targeting to release v2.0, we’re anxiously waiting for few features like Delete workflow etc in the upcoming release? Regard, Vardan Gupta

Dealing with data latency

2018-06-04 Thread Pedro Machado
Hi. What is the recommended way to deal with data latency? For example, I have a feed that is not considered final until 72 hours have passed after the end of the daily period. For example, Monday's data would be ready by Thursday at 23:59. Should I pull data based on the execution date minus a

Adding Kaxil & Arthur as co-mods on our email lists

2018-06-04 Thread Sid Anand
https://issues.apache.org/jira/browse/INFRA-16621 If others are interested, let me know. -s

Re: Airflow London Meetup Group

2018-06-04 Thread Sid Anand
Good to hear.. QCon committees start working about 6 months before the event. So, in October, make sure you reach out to me to let me know. FYI, I've just started work on the QCon SF committee (Nov 2017)... -s On Sat, Jun 2, 2018 at 5:17 AM, Naik Kaxil wrote: > I would surely try to give a

Re: Dealing with data latency

2018-06-04 Thread Maxime Beauchemin
The common standard is to have the execution_date aligned with the partition date in the database (say 2018-08-08) and contain data from 2018-08-08T00:00:000 to 2018-08-09T23:59:999. The partition date and execution_date match and correspond to the left bound of the time interval processed. Then