Re: Airflow Scheduler Crashing

Sigint means someone is pressing ctrl-c or sends a sigint to the scheduler. This is outside airflow. B. Sent from my iPhone > On 10 May 2018, at 06:30, ramandu...@gmail.com wrote: > > Hi All, > > We are running airflow version 1.9 in LocalExecutor mode. We are

Airflow Scheduler Crashing

Hi All, We are running airflow version 1.9 in LocalExecutor mode. We are observing that scheduler is crashed after few hours with below stack logs(Seems to be an issue with Mysql Connection. Is there any fix or workaround for this) Traceback (most recent call last): File

Re: Apache Airflow welcome new committer/PMC member : Naik Kaxil (a.k.a. kaxil)

Wow, congrats! Kevin Y Taylor Edmiston 于2018年5月9日 周三上午10:58写道： > Congrats and welcome! > > *Taylor Edmiston* > Blog | Stack Overflow CV > | LinkedIn > | AngelList >

Re: Apache Airflow welcome new committer/PMC member : Naik Kaxil (a.k.a. kaxil)

Congrats and welcome! *Taylor Edmiston* Blog | Stack Overflow CV | LinkedIn | AngelList On Wed, May 9, 2018 at 12:38 PM, Daniel Imberman

Re: How to use Kuberenetes executor with Airflow

Hi Goutham, Once we fix a few final bugs with the k8s executor (which should be in the next week). The next step will be to write out documentation for the kubernetes executor (we have one for the Kubernetes Operator but are waiting to hear back from CNCF to cross-post with them). If you look in

Re: Apache Airflow welcome new committer/PMC member : Naik Kaxil (a.k.a. kaxil)

Congratulations! On Wed, May 9, 2018 at 2:33 AM Naik Kaxil wrote: > Hi Everyone! > > Thanks for inviting me to join the Airflow PPMC! > > Brief introduction: I live and work in London, I'm currently working for a > Big Data & Analytics consultancy specialising in Google Cloud

Re: Subdag tasks unable to start

Glad it worked out! Best, Arthur On Wed, May 9, 2018 at 3:26 AM Victor Noagbodji < vnoagbo...@amplify-nation.com> wrote: > Thanks Arthur. > > I went with having two DAGs as you suggested. For anyone reading, that's > what worked for me. > > > On May 8, 2018, at 4:41 PM, Arthur Wiedmer

Re: About the difference between xcom and variable

So if without task_id(s), is it equal with variable ? On 09/05/2018, 6:26 PM, "Beau Barker" wrote: You can still specify task_ids, it's optional. > On 9 May 2018, at 20:15, 刘松(Brain++组) wrote: > > Since that it might

Re: Subdag tasks unable to start

Thanks Arthur. I went with having two DAGs as you suggested. For anyone reading, that's what worked for me. > On May 8, 2018, at 4:41 PM, Arthur Wiedmer wrote: > > Hi Victor, > > Usually, we expect the Schedules to match with Subdags, I am not sure > Airflow

Re: About the difference between xcom and variable

You can still specify task_ids, it's optional. > On 9 May 2018, at 20:15, 刘松(Brain++组) wrote: > > Since that it might have multiple upstream tasks, so which task to specify? > Or you change the xcom syntax to be global? > > > > > On Wed, May 9, 2018 at 6:13 PM +0800,

Re: About the difference between xcom and variable

Since that it might have multiple upstream tasks, so which task to specify? Or you change the xcom syntax to be global? On Wed, May 9, 2018 at 6:13 PM +0800, "刘松(Brain++组)" > wrote: What is the magic to remove the task id ? On Wed, May

Re: About the difference between xcom and variable

What is the magic to remove the task id ? On Wed, May 9, 2018 at 3:38 PM +0800, "Beau Barker" > wrote: Use xcom for communication between tasks. In the upcoming version 1.10 there's no need to specify task_ids for xcom. You can

Re: About the difference between xcom and variable

That's awesome. > On May 9, 2018, at 3:38 AM, Beau Barker wrote: > > Use xcom for communication between tasks. > > In the upcoming version 1.10 there's no need to specify task_ids for xcom. > > You can simply say: > self.xcom_pull(context, key='name') > > >> On 9

Re: Apache Airflow welcome new committer/PMC member : Naik Kaxil (a.k.a. kaxil)

Hi Everyone! Thanks for inviting me to join the Airflow PPMC! Brief introduction: I live and work in London, I'm currently working for a Big Data & Analytics consultancy specialising in Google Cloud Platform. I have some questions but I am first waiting for my account to be created by the

Re: About the difference between xcom and variable

Use xcom for communication between tasks. In the upcoming version 1.10 there's no need to specify task_ids for xcom. You can simply say: self.xcom_pull(context, key='name') > On 9 May 2018, at 17:22, Song Liu wrote: > > Hi, > > For the cross-task communication, there

About the difference between xcom and variable

Hi, For the cross-task communication, there are two options: - xcom - variable if I have make sure the key of variable is global unique (such uuid), is there any other limitations or cons for using variable instead of xcom ? since that xcom should specify the task_id which is used not very