Re: Cold-case PRs

2018-09-10 Thread Maxime Beauchemin
It doesn't deal with Jiras, just PRs and GH issues (which we don't use...) Max On Mon, Sep 10, 2018 at 6:58 PM Sid Anand wrote: > Max, > How do these manage the JIRAs? > > -s > > On Mon, Sep 10, 2018 at 6:14 PM Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > > > I've used

Re: [VOTE][RESULT] Replace with Gitter with Slack?

2018-09-10 Thread Sid Anand
Thanks for the context. So, I created the slack workspace and it has ~ 100 people on it now and it pretty active. What's pretty cool is that some folks created regional channels (e.g. singapore, boston, etc...), which is a good way to seed future meetups. At some point, we may want to retire

Re: Cold-case PRs

2018-09-10 Thread Maxime Beauchemin
I've used https://github.com/bstriner/github-bot-close-inactive-issues in the past to auto-close issues / PRs based on a policy around inactivity. It worked alright. There's also https://github.com/probot/stale which seems to be one of the leading solutions, but it may require an Apache INFRA

Cold-case PRs

2018-09-10 Thread Sid Anand
Folks! It's around that time (over 200 PRs with some 2 years old) when we need to consider closing some abandoned PRs. We have a lot more active maintainers now today than we had 1 or 2 years ago, we are better able to keep up with new PR demand, but it's still not perfect. I've created a GitHub

Salesforce Hook/Operator

2018-09-10 Thread Jon Nalley
Hi, I noticed that there is a sales force hook included in the Airflow repo: https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/hooks/salesforce_hook.py There is also a salesforce hook and operator in the salesforce_plugin repo:

Inaugural London Apache Airflow Meetup on Sept 20!

2018-09-10 Thread Sid Anand
Folks! Ben Marengo from Just Eat and Ash Berlin-Taylor (Fellow PPMC/Committer) will be speaking at the upcoming inaugural London Apache Airflow Meetup! Links: - https://twitter.com/ApacheAirflow/status/1039250401080487939 -

Re: TriggerDagRunOperator sub tasks are scheduled to run after few hours

2018-09-10 Thread Bolke de Bruin
Dates are converted to UTC before submitting to the DB and are converted to UTC on retrieval. Is this on 1.10.0 release? Postgres will return datetimes with timezone information so conversion should go properly (MySQL won’t but we set it to UTC explicit) B. > On 10 Sep 2018, at 20:15, John

Re: TriggerDagRunOperator sub tasks are scheduled to run after few hours

2018-09-10 Thread John Culver
Not sure if this will help, but we had same issue 3 hours apart. Problem was the DBMS (Postgres) was different timezone than webserver/scheduler/workers. When a new DAGRun was inserted, it was allowing DB to use current time for execution date. On Mon, Sep 10, 2018 at 2:10 PM, Bolke de Bruin

Re: TriggerDagRunOperator sub tasks are scheduled to run after few hours

2018-09-10 Thread Bolke de Bruin
I cannot connect this to timezone issues yet. The trigger dagrun operator seems to work fine (checked the examples). B. > On 10 Sep 2018, at 08:19, Deng Xiaodong wrote: > > Hi Goutam, > > Seems it’s due to the timezone setting? > > If that’s the case, you may try either adjusting your

Re: Use xcom in task retry

2018-09-10 Thread Emmanuel Brard
Hello Mishika, I think we had the same use case in which we wanted an operator to use the xcom values pushed from a previous try (after a retry) and also found out that it is cleared before an execution. What we did is to extend Airflow data model to hold the data we wanted to persist between

Re: airflow 1.9 tasks randomly failing | k8 - hive

2018-09-10 Thread Emmanuel Brard
Hello, You can see the zombie tasks killing in the scheduler logs (text file), I think you need the 'INFO' log level though. The detection is based on the time difference with the last heartbeat in the job table. Best, E On Fri, Aug 24, 2018 at 4:36 PM Dubey, Somesh wrote: > Thanks so much

Re: Use xcom in task retry

2018-09-10 Thread Mishika Singh
Hi Taylor, I am providing the code to show how I am using xcom pull and push, yes the push is succeeding to the database. But, the behaviour that I observed is when execute() is called for retry, xcom values are deleted from the table for combination of Then I searched for this in the Airflow

Re: TriggerDagRunOperator sub tasks are scheduled to run after few hours

2018-09-10 Thread Deng Xiaodong
Hi Goutam, Seems it’s due to the timezone setting? If that’s the case, you may try either adjusting your timezone setting or “manually” adjusting using execution_date argument in TriggerDagRunOperator. XD On Mon, Sep 10, 2018 at 14:00 Goutam Kumar Sahoo < goutamkumar.sa...@infosys.com> wrote:

RE: TriggerDagRunOperator sub tasks are scheduled to run after few hours

2018-09-10 Thread Goutam Kumar Sahoo
HI All, Could anyone please address this issue ? We are really stuck here and not able to proceed further Thanks & Regards Goutam Sahoo From: Goutam Kumar Sahoo Sent: Friday, September 07, 2018 4:51 PM To: 'dev@airflow.incubator.apache.org' ; 'd...@airflow.apache.org' Subject: