Hi all,
This thread got split somehow. To bring discussion of Daniel's proposal to
this thread:
Daniel Gruno
Wed, Aug 14, 2:33 AM (1 day ago)
to dev
Hi James,
I don't mean to butt in much here, but as the general "guy who fixes
various logos around here" (I re-did the current airflow logo as th
What is your dynamic DAG doing? How long does it take to execute it just as
a python script (`time python mydag.py`)?
As an Airflow admin, people may want to lower the DAG parsing timeout
configuration key to force people to not do crazy thing in DAG module
scope. At some point at Airbnb we had so
Thank you for your reply, Max
Dynamic DAGs query the database for tables and generates DAGs and tasks based
on the output.
For Python does not take much to execute:
Dynamic - 500 tasks:
time python PPAD_OIS_MASTER_IDI.py
[2019-08-15 12:57:48,522] {settings.py:174} INFO - setting.configure_orm(
It's not just webserver and scheduler that will parse your dag file.
During the execution of a dag run, dag file will be re-parsed at the start
of every task instance. If you have 1000 tasks running in short period of
time, that's 1000 queries. It's possible these queries are piling up in a
queue
Hey all, I'm the designer of the new logo that kicked off this thread. I will
now and forever consider it the "shuriken" logo (I did not even know that word
before this thread, so thank you). Based on the feedback I went back to the
drawing board and came up with another version that is closer t
Thanks Chris for the new Airflow logo.
Personally I like the new pinwheel design better. Its fins are more natural
as they seem be made by folding four Isosceles right triangle papers, which
is natural and symmetric. On the other hand, the fins of the shuriken one
is too narrow and sharp, which se
Hi All,
Do we have airflow meetup(s) in India. We are based out of india and are using
Apache Airflow as an orchestration engine to author and manage 1000(s) of
multi-step workflows.
Would be interested in Airflow meetup in India.
Thanks,
Raman Gupta
