Failover in apache 1.8.0

Hi, I would like to know what happens if a Celery worker running one of the tasks crashes. Will the job be rescheduled? Also, if the scheduler is not able to schedule a task on time due to heavy load on all workers, what will happen to the task? Regards Shubham Gupta

Re: Failover in apache 1.8.0

Hi Ruiqin Yang, Can you please elaborate on what is meant by "and the scheduler lost track of it" in your second paragraph? When can this happen? Also, what is the default state when the scheduler restarts? Is it not* None*? Thanks for your quick reply. Regards Shubham Gupta On F

Re: Failover in apache 1.8.0

milar in airflow? Regards Shubham Gupta On Fri, Jul 20, 2018 at 1:42 PM Shubham Gupta wrote: > Hi Ruiqin Yang, > > Can you please elaborate on what is meant by "and the scheduler lost > track of it" in your second paragraph? When can this happen? Also, what > is the de

Fwd: Catchup By default = False vs LatestOnlyOperator

-- Forwarded message - From: Shubham Gupta Date: Fri, Jul 20, 2018 at 2:38 PM Subject: Catchup By default = False vs LatestOnlyOperator To: Hi! Can someone please explain the difference b/w catchup by default = False and LatestOnlyOperator? Regarding Shubham Gupta

Re: Catchup By default = False vs LatestOnlyOperator

Thanks a lot for the useful info. Regards Shubham Gupta On Wed, Jul 25, 2018 at 7:48 PM Sid Anand wrote: > I will +1 James comment and add to it. At Agari, one of our DAGs had as a > final step the sending of an alert. The alerts only made sense when the DAG > was current. But, some

Schedule Interval in both DAG and task

Hi, In airflow, both DAG and operator can have schedule intervals. In case both are specified, what is the default behaviour? Regards Shubham Gupta

Start date and end date

eone explain me what it means by an example? Regards Shubham Gupta

Airflow variables and data profiling hidden

not readable by the server.' The people who set airflow cluster initially are no longer in the same company. Regards Shubham Gupta

Fusing operators together

rators together? - Is it a good idea to combine operators at all? Here's <https://stackoverflow.com/questions/53308306> my complete (more elaborate) question on StackOverflow Thanks *Shubham Gupta* Software Engineer zomato

Create Connections in Airflow operator at runtime

k <https://stackoverflow.com/questions/53740885/create-and-use-connections-in-airflow-operator-at-runtime> to my (elaborated) question on SO Thank You *Shubham Gupta* Software Engineer zomato

Wiring top-level DAGs together

//medium.com/@hafizbadrie/airflow-when-your-dag-is-far-behind-the-schedule-ea11bf02e44c> *Shubham Gupta* Software Engineer zomato