As someone who is interested in contributing, but has not been able to yet, I agree massively with Tomasz. There is still so much I need to understand about how Airflow works and pretty much everything I have learned is from trial and error. While I do agree that a certain amount of testing is helpful in understanding a technology, there is so much lacking in the documentation that I often resort to code diving, or worse guesswork, when trying to understand a new concept. This not only raises the barrier for entry for using Airflow and developing for Airflow, but I also feel it reduces my confidence in being able to commit as there is so much of the codebase which just seems mysterious and unknown to me.
Take for example the config options. Only recently has someone made a documentation page which explains every option, but this itself is very lacking; it has no more information than the comments on the default airflow config file. Improving and expanding the documentation can be best done by those who understand Airflow the best; committers. It's a great investment in development as it accelerates the learning for new and intermediate users, who can sooner become contributing members of the community. Just a few thoughts that I hope will be considered. Jacob On Sun, 29 Dec 2019 at 18:50, Tomasz Urbaszek <tomasz.urbas...@polidea.com> wrote: > Hi all, > > In my opinion we area great community but we are not "Airflow". If someone > finds us, then his or her troubles are probably solved. At least I hope so. > But last survey result includes a few really important "non-technical" > points: > > When onboarding new members to Airflow, what is the biggest problem? > - No guide on best practises on developing DAGs 51.95% > - Small number of tutorials on different aspects of using Airflow 18.51% > In your opinion, what could be improved in Airflow? > - Examples, how-to, onboarding documentation 46.43% > - Technical documentation 44.48% > > These are "users opinions" but I would also give them a +1. I think that to > encourage > people to contribute to Airflow we frist should encourage to use the tool. > Then, > when people know how to and how not to use Airflow they will be able to > spot real > bugs and groundbreaking features. A user who contributes is a great value > and > it's something we should encourage. > > Airflow is a huge project and the easier it will be to understand how it > works, the easier > it will be to contribute. After 6 months there are still things that I do > not fully understand. > I think our new website is a tremendous opportunity. > > On the other hand, I fully support the idea of meetups. I really like the > quite new > Apache Local Community[1] idea. It's something that should bring ASF and > OSS > closer to students, devs, business people and anyone who's interested. > Moreover, > it's something that will bring value to our communities because those > meetings > will be a great opportunity to exchange experiences between projects. > > T. > > [1] > > https://cwiki.apache.org/confluence/display/COMDEV/Apache+Local+Community+-+ALC > > On Sun, Dec 29, 2019 at 7:12 PM Felix Uellendall <felue...@pm.me.invalid> > wrote: > > > Hey Jarek, > > > > I really like the points you bring up. While reading your mail I thought > > about the same things. For me at the beginning it was really hard to get > > into this community and how everything works mostly because of the > > language. I am not sure but maybe it would also be a good idea to > organize > > meetings in native languages first for people who do not feel comfortable > > speaking/reading/writing in English - just to help them to feel > > comfortable. And after they got a basic understanding of the whole > process > > of contribution and how the project works they can start learning English > > if they want to join the community. > > > > Of course English is a requirement but I think people would be more > > willing to improve their English if they have no other barriers left that > > are keeping them from joining this community. > > > > Felix > > > > Sent from ProtonMail Mobile > > > > On Sun, Dec 29, 2019 at 18:37, Jarek Potiuk <jarek.pot...@polidea.com> > > wrote: > > > > > Hello everyone, > > > > > > TL; DR; I wanted to start a non-technical discussion about being (even > > > more) welcoming community. > > > > > > It's a long read - following some deep discussions I had recently and > you > > > might not be interested in it, so feel free to skip the entirety of it. > > > > > > I also believe this might become quickly a controversial topic and > > > mis-communication over email can easily happen - so I would like to ask > > > everyone to be considerate and open-minded when responding. > > > > > > *Some context - how welcoming are we now ?* > > > > > > First of all I think we are doing a lot as community to be really > > welcoming > > > and friendly. A lot that we do is really opening up in various ways to > > new > > > community members, users, existing contributors etc. We are responsive, > > > helpful, we try to actively reach-out to get users opinions (the > survey). > > > We are open to invite non-code-committers to get "committer status" > > (that's > > > highly encouraged by the Apache Software Foundation!) or even PMC > members > > > (yeah!). We organise events (Meetups and upcoming Airflow Summits), > > > workshops for users and new contributors. We are making it easier for > new > > > contributors to start contributing - by environment and documentation > > > improvements. > > > > > > At the same time we have certain expectations/barrier of entry. It's > not > > > super easy to join the community and you must really earn your status > to > > > become a committer/PMC member. I think we are fairly good as a > community > > in > > > enforcing that in deliberate and firm ways - and all this without being > > > rude or aggressive. I remember one of the first emails when I joined > the > > > community where I was firmly but friendly reminded that in this > community > > > decisions are made by the community and not a bunch of people talking > at > > > slack and agreeing to something between them. That was a very important > > > lesson to me - and first trigger to learn what ApacheWay is. And it was > > > super cool even though I felt I have to apologize for my lack of > > > understanding how this all works (which I did). > > > > > > We have certain expectations for PRs/code - some enforced > automatically, > > > some by comments/discussions/review process. And we have expectations > for > > > engagement of people submitting the code. They are supposed to > follow-up > > > their PRs - being responsible to get the PRs to submission and engage > > > committers when they need it. We also encourage people not only to > > > finger-point things to fix but also engage and help with fixing things > > they > > > find or even improve the processes. > > > > > > I think it's rather good mixture of openness/barrier of entry. When > > someone > > > new joins any community - has to first adapt and show how they can be > > > valuable for the community before he or she can influence the way > > community > > > works. So it's great that there are firm boundaries and expectations > and > > > that we clearly explain them to anyone that tries to join and we expect > > > those people to follow the expectations before we invite them further > > after > > > they "earned" the status. This is best described in the "meritocracy" > > rule > > > defined here: > > > https://www.apache.org/foundation/how-it-works.html#meritocracy . We > are > > > following it really well I think. > > > > > > I believe in many ways we are much better than a number of other > > > open-source communities and we are following ApacheWay fairly well. And > > > I've heard personally a member of the board of the Apache Software > > > Foundation praising how welcoming Apache Airflow community is. > > > > > > *So why the discussion at all if we are in such good shape ?* > > > > > > I just wanted to see if we can do better than that - and whether we > need > > to > > > do better currently at all. > > > > > > I think it's fairly easy to overlook the moment when we should do > > something > > > more. Maybe we can change something to be even more welcoming. Maybe we > > can > > > get people engaged who currently do not engage because it is too > > difficult? > > > Maybe we miss another point of view because of that? Maybe some of the > > > rules we have should be updated? Maybe people who feel excluded do not > > > speak here because they feel the barrier of entry is too big and they > are > > > afraid they will not be heard or will be ignored or will be shouted > at. I > > > think it's better to discuss such things when everything looks great > and > > > when there is a good "vibe" in the community rather than being > triggered > > by > > > people complaining after it becomes a problem and when the "vibe" > > > deteriorates. > > > > > > The trigger for my thoughts was a looong discussion I had with one of > the > > > attendees of PyDataWarsaw conference a few weeks ago at the > after-party. > > We > > > talked for several hours I think, and we were the last ones to leave > the > > > party grounds (yes it was 3 am or so :D ). The person I spoke to > raised a > > > few important topics - like "not everyone has enough courage to openly > > > speak at the discussion list first" or "unconsciously people are > valuing > > > less contributions by women" (there is a study confirming that > > > > > > https://www.theguardian.com/technology/2016/feb/12/women-considered-better-coders-hide-gender-github > > ) > > > and "some people need a kind of mentorship when they enter new > community > > > and after the introduction they become great contributors" - and he had > > > some really good examples for all those statements from his own > > experience. > > > After the discussion he read about Apache Way (as I advised him), > looked > > at > > > our discussions and he wrote to me a few days ago that he sees how > > > welcoming we are and that we are addressing a lot of the concerns he > has > > in > > > really good way - but nevertheless it stuck with me a bit and I > thought - > > > maybe he is right that we should discuss it. > > > > > > For example - while we have two women on the PMC member list, almost > all > > > the people committing the code are male (I believe). This - of course - > > > reflects the state of our industry and is nothing new, but maybe we are > > > (unconsciously) doing something in our discussions in devlistt or slack > > or > > > reviews that puts off people who otherwise would be valuable to our > > > community? The friend of mine who triggered my thinking had a great > point > > > that not everyone new has the courage to speak openly at the devlist or > > > slack initially. Maybe we should reach out in a different way to those > > > people? Or maybe we should think about some kind of mentorship for new > > > people so that we can guide people through the first stages of becoming > > > contributors and navigate the way our community works? > > > > > > It looks like we already have people from all over the world - US, > > Europe, > > > India, Japan, Australia, China. We have meetups in almost all of those > > > places. But maybe we could do more to get more people > contributing/users > > > invited from some places (for example we have no meetups in China yet > and > > > not a lot of people from South America I think). Again - maybe we can > do > > > something about it ?. I know there was an event in Mexico where we had > > > Airflow workshop - maybe we can reach out to people there somehow :) ? > > > There was also a great presentation about Chinese user community at the > > > ApacheCon Europe few months ago > > > > > > https://aceu19.apachecon.com/session/inviting-apache-flinks-chinese-user-community > > > on > > > how difficult it is to get people in China contributing because of the > > > language barrier. Maybe we should get more workshops for new > contributors > > > in Chinese/Mandarin in China initially and get some contributors from > > there > > > (writing description of a PR might be easier even for someone who has > > > difficulties speaking english or you can have someone who will be your > > > local mentor for that). > > > > > > I do not have concrete proposals yet, or I do not ask you to have them > > > immediately. I don't even know yet if we should do something or not. > But > > I > > > wanted to open up discussion to hear what others think about it - both > > > active members of our community and those who are just listening and > > rarely > > > discuss. > > > > > > Maybe we are really in a good state and we should just continue? Or > maybe > > > there are some easy things we can do as a community to get better at > > being > > > more welcoming ? Also maybe we should forward the discussion elsewhere > > > (users@?/Slack?/Meetups?) so that others who are not reading the > devlist > > > can chime in ? > > > > > > I'd really love to hear what others think about it! > > > > > > Again - please be considerate and open-minded - this might quickly > > become a > > > controversial subject and miscommunication is almost certain, so let's > > all > > > be careful with words and statements. > > > > > > J. > > > > > > -- > > > > > > Jarek Potiuk > > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > > > M: +48 660 796 129 <+48660796129> > > > [image: Polidea] <https://www.polidea.com/> > > > > -- > > Tomasz Urbaszek > Polidea <https://www.polidea.com/> | Software Engineer > > M: +48 505 628 493 <+48505628493> > E: tomasz.urbas...@polidea.com <tomasz.urbasz...@polidea.com> > > Unique Tech > Check out our projects! <https://www.polidea.com/our-work> >
