Thank you for your ideas Mike. In the spirit of local evaluations perhaps it would also be useful to outline how Airflow approaches task and dependency management compared to other tools. Specifically tools like Celery, Luigi, and Nifi. I know a broad explanation of what tasks Airflow excels at performing would be helpful to me and people I work with so we can compare it to existing solutions.
I’ll check out your tutorials and the Airflow ones and see if I can identify any potential problem areas. On Thu, May 9, 2019 at 1:25 PM Michael Ghen <[email protected]> wrote: > Hello Nathan, I hear you on setting up a local environment, that was one > reason I put this tutorial together back in 2017, unfortunately it maybe a > little out of date now: https://github.com/mikeghen/airflow-tutorial > > I recently integrated a similar tutorial into the Apache Unomi website also > along the lines of helping people setup local environments for evaluation > purposes: https://unomi.apache.org/tutorial.html > > I think something similar might be good, some documentation focused on > making it easy for anyone to evaluate Airflow. I am sure a lot of people > are capable of getting running and hopefully with some enhancements, even > beginners will be able to get setup with Airflow and perform meaningful > evaluations. > > Anyway, I think both projects sound good and the documentation one is > something I am interested in as well, if you'd like any assistance, get in > touch with me :) > > > On Wed, May 8, 2019 at 10:57 PM Nathan Maynes <[email protected]> > wrote: > > > Hey all, > > > > I wanted to introduce myself. I am Nathan Maynes and would like to get > > involved in an open source project. I currently work as a Data Engineer > and > > recently participated at an Apache RoadShow event in Washington DC. At > the > > event I was introduced to Airflow and became interested in helping the > > project grow. I asked a few Apache project contributors and committers > for > > advice on getting involved. They all mentioned I should start by looking > > for ways to improve documentation. Following their advice, I stumbled > > upon the list of possible > > <https://cwiki.apache.org/confluence/display/AIRFLOW/Season+of+Docs+2019 > > > > Airflow projects for Google Season of Docs (GSoD) 2019. I would love to > get > > involved even if it is not via the GSoD program but don't quite know > where > > to begin. Please forgive my ignorance if this message is off topic or > more > > appropriate for another forum. > > > > When I look at the possible project list on the project Confluence site, > > two of the projects jump out at me. The system diagram project is one I > > would be interested in working on. It seems much shorter than others on > the > > list. Is that by design? Are these project suggestions meant to be > combined > > to form a project of appropriate scale? The second project I find > > interesting and personally relevant is around setting up a local > > environment. This is something that I recently attempted and found > > frustrating. I think building out the documentation on getting started > > would benefit the project. As the Confluence entry points out, the > > documentation seems to be available, it just needs to be organized and > > polished. > > > > I am looking for feedback on whether a solid proposal for one of these > > ideas is enough. Should multiple ideas be combined into a larger project? > > Would instructional videos be desirable in conjunction with one of these > > projects? Again, I apologize if this is not the right forum for these > kinds > > of questions. I will happily take feedback and ideas via the mailing list > > or on the Slack channel. Thanks everyone! > > > > -- > > Nathan Maynes <http://bit.ly/115hXAt> > > @nathanmaynes > > > -- Nathan Maynes <http://bit.ly/115hXAt> @nathanmaynes
