Re: Issue with Airflow - Jobs are not triggered to run after a random period of time

> > On 1 sep. 2016, at 20:45, hila...@gmail.com wrote: > > > > > > > > We face exactly the same issue... > > I tried to describe it here this week, > > But no one had a solution. > > > > ‫ב-1 בספט׳ 2016, בשעה 17:54, ‏‏Sergei Iakhnin ‏<lle...@

Re: Issue with Airflow - Jobs are not triggered to run after a random period of time

u confirm that this happens on celery? > > It awfully sounds like this: > http://stackoverflow.com/questions/27737990/django-celery-queue-getting-stuck > > > > Sent from my iPhone > > > On 1 sep. 2016, at 21:59, Sergei Iakhnin <lle...@gmail.com> wrote: > > > >

Re: Issue with Airflow - Jobs are not triggered to run after a random period of time

As far as I know even Airbnb themselves restart their schedulers every 30 minutes because of this issue. I ended up doing it as well with a cron job after giving up hope that it would be fixed in the short term. On Thu, 1 Sep 2016, 16:03 Charalampos Paravalos, wrote: > Hi, > > I

Re: Airflow and Celery - co-ordination issue

This happened to me too. I ended up putting them in queued status via db update and then they got scheduled. On Thu, 2 Mar 2017, 15:27 twinkle, wrote: > Hi, > > We plan to use Airflow along with Celery as the backend. > Today within a DAG run, despite showing some

Re: Airflow Webserver behind a reverse proxy

might help you. > > Thanks, > Sumit Maheshwari > cell. 9632202950 > > > On Thu, Sep 14, 2017 at 2:34 AM, Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > > > It works well if you use a subdomain instead of a folder. > > > > Max >

Re: Airflow Webserver behind a reverse proxy

-applications-with-uwsgi-and-nginx-on-ubuntu-14-04 > > > > I'm not super familiar with the routing in Nginx. A subdomain like > > airflow.domain.tld will work like Maxime suggested, but I'm not really > sure > > if extending the path will work, e.g. domain.tld/airflow.

Re: Airflow behind reverse proxy

Take a look at this PR - https://github.com/apache/incubator-airflow/pull/2723#issuecomment-350356492 On Tue, Dec 19, 2017 at 9:49 AM Andreas Koeltringer < andreas.koeltrin...@n-fuse.co> wrote: > Hello everybody, > > I am wondering whether Airflow uses absolute links to get its static > content?

Re: Airflow Deployment tools

Saltstack. On Tue, 14 Nov 2017, 18:14 Scott Halgrim, wrote: > We have a bash script. Main features > > > • Uses AWS metadata URL to determine if it’s a prod box or dev box and > pulls appropriate branches > • Gets difference between commit on box and commit on

Re: Task Instance Anomaly Detection

I use the TICK stack - https://github.com/influxdata/. You can read more in our paper - https://www.biorxiv.org/content/early/2017/09/08/185736 Basically Telegraf collects metrics (including statsd metrics from Airflow; Airflow would benefit from more of these), sends them to Influxdb, Kapacitor

Re: PR Review request for url_prefix

This feature sounds very useful. I hope you get your PR in soon. On Sat, Dec 9, 2017 at 7:59 PM Gaetan Semet wrote: > Hello > > Sorry to insist on my PR review, but it seems it can help some people: >

Re: Moving on to Lyft

Congrats! On Wed, 25 Oct 2017, 23:24 Maxime Beauchemin, wrote: > Hi all, > > Quick note to say that I recently left Airbnb and will be joining Lyft on > the 30th of this month. > > First I want to thank the amazing folks at Airbnb for enabling and > sponsoring the

Re: How to trigger the dag to run immediately

You can trigger runs programmatically like this: def set_up_dag_run(context, dag_run_obj): dag_run_obj.payload = {"config": context["config"]} dag_run_obj.run_id = str(uuid4()) print context return dag_run_obj my_dag_run = TriggerDagRunOperator(

Re: We've migrated to Github to repo!

I find the dev list has gotten extremely noisy with the move to Github. Getting an email about every PR comment seems a bit excessive. Might it be a good idea to not have this list subscribed to all the github updates? People who are interested in such granular updates can still "watch" the github

Re: We've migrated to Github to repo!

e > this as nothing I read about the process mentioned this. > > -ash > > > Fokko: the new Apache-hosted URL is > https://gitbox.apache.org/repos/asf?p=incubator-airflow.git > > > > We should ask if they can remove/hide the old non-gitbox one. > > > > -ash &