Hi Sage, Thank you so much for this reply.
On 2019-07-05 17:49, Sage Sharp wrote: > [Moving this to a separate thread, and again, please do not discuss > funding sources or board notifications on this thread.] > > Joan had specific questions as to how to engage Outreachy mentors. I > would suggest sending them links to Outreachy's documentation of mentor > duties and the mentor FAQ: > > https://www.outreachy.org/mentor/#mentor > https://www.outreachy.org/mentor/mentor-faq/ > > Outreachy mentors need to submit their project description through the > Outreachy website. I've attached the project information we ask for in a > text file. The community sign up doesn't start until August 1, so > there's no way for mentors to sign up for the next round until then. If > the ASF D&I committee wants to review mentor's project before August, > they can use the format to start gathering proposals. > > More information on Joan's specific questions below: > > On Thu, Jun 27, 2019 at 11:59 AM Joan Touzet <[email protected] > <mailto:[email protected]>> wrote: > > On 2019-06-27 13:22, Naomi S wrote: > > On Thu, 27 Jun 2019 at 18:52, Ross Gardler > > <[email protected] > <mailto:[email protected]>.invalid> wrote: > > > >> I have seen NO discussion on the topic of who the mentors are > and minimal > >> discussion on how we evaluate proposals. > >> > > > > I will note that this very thread ("[DISCUSS] Outreachy framework > > proposal") was supposed to be the start of that conversation. > > And I would like to continue that topic, but until the noise abates and > the S/N is restored here, I refuse to do so. I simply haven't the energy > to wade through the piles of vitriol. > > I'll leave a few points here, and hopefully someone can pick up on them: > > * We need a double blind evaluation system that masks project name, > applicant name, specific details, etc. > > > (FYI - using the term "anonymized evaluation system" is a better term > "double blind evaluation system". I've seen vision-impaired people > complain about how this term is an unpleasant one for them.) Thank you so much! I should have realized this myself. I'll start using "double-anonymized" or "anonymized" instead. > I assume you mean an anonymized evaluation system for deciding which > Outreachy internship projects to accept? The Outreachy website asks for > a lot of project-specific details (like communication channel links, > documentation for how to make a first contribution, etc.). I don't think > it will be possible to make an informed decision about whether a project > is going to be successful in Outreachy without looking at those details. > You would need someone to do a lot of redaction on the project > descriptions to achieve anonymity. No, you're right, this wouldn't be easy and doesn't make a lot of sense. > Or are you talking about an anonymous review system to help coordinators > needing to make a decision about which intern selections? That would be > needed if there are more interns than you have funding for. Yes, this is what I was driving at. > Coordinators will already have access to information about applicants > and intern selects through the Outreachy website, so implementing an > anonymized intern selection system might not be feasible for that. The > information that coordinators and mentors can see on the Outreachy > website has some details about the applicants. Each project will have a > page that lists applicants who have recorded a contribution or filled > out a final application. That page lists the applicant's name, email > address, location and timezone, pronouns, and a summary of how many days > free the applicant has during the internship. The final application > lists answers to questions like "What is your experience in other free > software communities?" and "What other experience do you have that is > relevant to this project?" > > Mentors have tried to implement anonymous reviews of Outreachy > applicants' contributions. I know some projects in Mozilla have turned > on a GitHub feature to hide the name of the person submitting a > contribution. The coordinator gathers final applications and recorded > contributions, redacts personal information, and shares it with the > mentors. Then the mentors make a decision on which applicant to accept, > all without knowing details about the applicant. > > If the ASF wants to anonymize intern selections for review, it would > need a neutral third party to redact personal information and > identifying information from the intern's final application and the project. I am concerned about both conscious and non-conscious bias entering into the selection process, but perhaps I am worried over nothing. How frequently do projects receive more than one intern application? > * We need a way to ensure that mentors actually are capable of > committing the time necessary for this programme, to check in with > the admins on regular intervals to ensure this is happening, and to > blacklist them from future involvement if they are unable to meet > those expectations and they do not have an acceptable excuse, since > real money will have been wasted. (Doctor's note, death in the > family, etc.) > > > Outreachy requires mentors to check a box during project sign up to > specify that they understand this is a 5 hour time commitment during the > internship. There's another box for acknowledging the 10 hour time > commitment during the contribution period. The Outreachy organizers > don't do anything as elaborate as what you're proposing, but there's > nothing stopping coordinators from doing so. Yeah, this was really an internal point for us, and the coordinator can help ensure this point is made. I don't want the ASF to have a black eye because people signed up to mentor, then didn't follow through. Sure, things happen unexpectedly, but we should at least do a double-check that the mentorship applicant knows what they're signing up for, vs. thinking this is a "land grab for warm bodies" (which it really is not.) > Outreachy already has feedback points three times during the internship > (2 weeks in, 8 weeks in, and 13 weeks in). Mentors give feedback on > their interns, including their response time, whether the intern should > be paid their stipend for that term, and whether the internship should > be extended. Interns give private feedback on their mentors, including > the mentor's response time to their questions and how long it takes the > mentors to give feedback on their contributions. > > The mentor and intern feedback is private, only read by Outreachy > organizers. That's because interns sometimes talk about how their > personal life has been impacting the amount of time they could put into > their internship. These are often very personal matters, e.g. "I'm going > through a divorce" or "I'm moving to a new country" or "I have this > specific health issue". That's not something we want to automatically > share with coordinators. > > However, if you're concerned about mentor response times, I can let you > know if I see concerning trends in mentor response time or intern > comments on the ASF mentors. This would be very valuable feedback to the Coordinator. Yes, please! My main goal is to ensure the interns get a good experience out of this, and as a volunteer-run-and-led organisation, unexpected absences are more common than most of us would like. > * We need a double-check that the projects involved agree to follow > not just the ASF CoC, but the Outreachy CoC, and any other imposed > requirements Glad to hear this isn't an issue for you :) > * We need a ruler by which we can measure the quality of the project > in terms of its suitability for an intern - what will the intern > *gain* by working on the project? Or is it just self-interest of > the requesting PMC/org? Quantified and qualified, preferably. > > > One of the questions we ask mentors during project sign up is "What > benefit will the intern have for working on this project?" I've also > talked about other project criteria you might want to use here: > > https://lists.apache.org/thread.html/f790fa5bffd7c2aac3fb495593456d55bed940815891511c6d3fd039@%3Cdev.diversity.apache.org%3E This is all really good, and I hope the selected Coordinator on our side picks up on all of this. > * We need to review the tagged areas Outreachy provides to ensure > that the applications we vet cover a broad range of opportunities > within our Foundation, not 100% documentation or internal-facing > tools. This is really internal direction for our Coordinator, and I think it will be simple enough to handle. > * We should find a volunteer to reach out to our compatriots within > Fedora and Debian to see how they coordinate and vet their > similar opportunity programs. > > > I also would recommend talking with the Mozilla coordinators, as they > have more projects that they vet (10-15 per round). I can connect you > with Dustin (the main Mozilla coordinator). I would need a specific set > of people to connect him with, rather than dragging him to a public > mailing list thread that is at risk of meandering off topic. Fantastic! This, plus the other Mozilla name you mentioned on the call with Gris et. al., should get us very far. Thanks again for all your support! -Joan "now we're cookin'" Touzet
