Re: Use xcom in task retry

2018-09-10 Thread Emmanuel Brard
Hello Mishika, I think we had the same use case in which we wanted an operator to use the xcom values pushed from a previous try (after a retry) and also found out that it is cleared before an execution. What we did is to extend Airflow data model to hold the data we wanted to persist between

Re: Use xcom in task retry

2018-09-10 Thread Mishika Singh
Hi Taylor, I am providing the code to show how I am using xcom pull and push, yes the push is succeeding to the database. But, the behaviour that I observed is when execute() is called for retry, xcom values are deleted from the table for combination of Then I searched for this in the Airflow

Re: Use xcom in task retry

2018-09-08 Thread Taylor Edmiston
XComs push and pull should work as expected when a task is retried. It shouldn't make a difference, but are you using XComs with explicit keys or the implicit return-based style? Is the push succeeding to the database? Also can you show a simplified example of your code for this DAG? Taylor On

Re: Use xcom in task retry

2018-09-06 Thread Ben Gregory
Hi Mishika -- Posting this question on StackOverflow with some code examples you're using will likely be the fastest way to have this addressed. Just make sure to tag it with "airflow" so people can find it. - Ben On Thu, Sep 6, 2018 at 12:12 AM Mishika Singh wrote: > I am pushing some

Use xcom in task retry

2018-09-06 Thread Mishika Singh
I am pushing some [key,value] using xcom in an operator, which I want to use when that task fails and comes for retry, for which I am using xcom_pull in that operator. But it is returning None instead of that value. Any pointer around this will be helpful. -- Regards Mishika Singh