talking about documentation :) I made the changes re: comments by Boris:
https://gtoonstra.github.io/etl-with-airflow/ First level of review: - does the organization of tasks in DAGs make sense, - would you split it up tasks like this or would you put everything together? - task dependencies : "depends_on_past" recommended, overall DAG configuration - better ways to organize a DAG to reuse it for "daily" import vs. a more historical "monthly" or "weekly" import? (what's best way to do historical reloads? ). All source is currently at: https://github.com/gtoonstra/etl-with-airflow Happy ot move it elsewhere to make it more of a community project, but if there's lack of time, I'd like to ask you to add the site (top link) to the wiki as well, then I'll keep charge of it until we maybe decide otherwise in the future. Rgds, Gerard On Wed, Nov 2, 2016 at 7:43 PM, Maxime Beauchemin < [email protected]> wrote: > Here: > https://github.com/apache/incubator-airflow/pull/1863 > > > On Wed, Nov 2, 2016 at 10:51 AM, siddharth anand <[email protected]> > wrote: > > > Removing laurel's email address as it appears to be wrong. > > > > -s > > > > On Wed, Nov 2, 2016 at 10:49 AM, siddharth anand <[email protected]> > > wrote: > > > > > +1 for that idea. We should place all links on the wiki and just have > the > > > project page point to the wiki! > > > (https://airflow.incubator.apache.org/project.html). > > > > > > Gerard, would you like to file a quick PR for that change and I can > > > approve/merge it? > > > -s > > > > > > On Wed, Nov 2, 2016 at 10:37 AM, Gerard Toonstra <[email protected]> > > > wrote: > > > > > >> They are both on the project page of the airflow documentation in > > >> resources > > >> & links and on the wiki, the wiki is a bit > > >> richer in that regard. Maybe link to the wiki from the doc pages > > instead, > > >> so it's all in one place? > > >> > > >> https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Links > > >> > > >> https://airflow.incubator.apache.org/project.html > > >> > > >> G> > > >> > > >> > > >> > > >> On Wed, Nov 2, 2016 at 5:13 PM, Maxime Beauchemin < > > >> [email protected]> wrote: > > >> > > >> > Hi, > > >> > > > >> > Laurel Brunk reached out to share this great blog post he wrote > about > > >> using > > >> > Airflow at Astronomer: > > >> > http://www.astronomer.io/blog/airflow-at-astronomer > > >> > > > >> > Where should we link out to this type of material? README.md? The > > >> > Confluence wiki? > > >> > > > >> > Thanks, > > >> > > > >> > Max > > >> > > > >> > > > > > > > > >
