Re: [Wikimania-l] WMF Scholarships to attend Wikimania

[Lodewijk]

(responding inline)

2017-04-22 7:41 GMT+02:00 Pine W :

>
> On Fri, Apr 21, 2017 at 5:08 AM, Lodewijk 
> wrote:
>
>> (not responding to a person in particular)
>> I'm a little bit at a loss here. The proposal is to share a lot of
>> information from the application process (whether attempted to anonimize or
>> not) beyond statistics. Given the high number of countries and other rather
>> specific characteristics, anything vaguely useful will likely contain at
>> least some personally identifiable information.
>>
>
> PII disclosures can be limited to what users have already disclosed in
> public (which, admittedly, may not be entirely current and truthful.)
> Aggregated information can be provided as well.
>

Sure - like I said, probably rather useless depending on the goal you want
to actually *do* with the information. And still tricky, aggregated
information could be provided to some extent, but probably not to the level
of detail you'd want.

>
>
>> More likely even, anything you can share without being personally
>> identifiable will probably not be very relevant for the application
>> consideration. Sure, you could do some gender statistics, but how does that
>> tell you why people have been rejected?
>>
>
> I anticipate that the level of transparency would be insufficient to
> evaluate the Scholarship Committee and WMF decisions about individual
> applicants. However, the information that is published may still be useful
> and of interest when considering trends and groups.
>

OK, so you want to discover 'trends and groups'. Goal 1 identified.


>
>
>>
>> So I'd like to take a step back: what exactly is the problem you're
>> trying to solve? Is publishing a lot of data really the best approach to
>> that solution? If you define the problem well, I can imagine a few
>> alternative approaches, like asking the scholarship committee to report
>> back with an analysis of the problem and how they went about it - or asking
>> an independent person/persons to sign an NDA, and go into the data,
>> investigate and report back. They could actually go in depth - but it
>> requires a good definition of the problem.
>>
>
> My impression is that there are disappointments and complaints almost
> every year about scholarship awards. I hope that increasing transparency
> will result in a decreased number and intensity of complaints about
> individual cases, and will also increase the amount of information that is
> made public which can be used by anyone and everyone to analyze policies
> and practices and to make recommendations for refinements or changes as may
> seem best.
>
> Also, as a broader theme, I would like to see more transparency about how
> WMF funds are used. A change of practice like we're discussing here would
> be one step in that direction.
>
>

Of course there are going to be disappointments and complaints every year.
Unless we increase the acceptance rate to 100%, that is bound to happen in
a process that always results in some personally disappointing outcomes.
Even with perfect transparency and process, people will be disappointed.
And a process will never be perfect. I sincerely doubt transparency will
decrease the intensity or number of complaints about individual cases - I
rather suspect it will increase them. As transparency often does. Which is
fine if the transparency brings other benefits - but don't expect it to go
down.

Also, identified the second goal: propose recommendations for refinements
and changes.

Good! Two valuable goals. Now, just disclosing stuff the best approach to
tackling it?

Lodewijk


>> Best,
>> Lodewijk
>>
>> 2017-04-21 13:32 GMT+02:00 Jonathan Cardy :
>>
>>> Hi Pine, I agree with you that partial transparency can be a positive
>>> and at least assure people that their region/language/project is getting a
>>> fair share even if they were declined. But I'd suggest that can be done
>>> with anonymised stats rather than applications with some details redacted
>>> or withheld.
>>>
>>
> Let me ask: why shouldn't the usernames of applicants, and whether they
> were offered scholarships, be made public in future years if scholarship
> applicants are told in advance that this information will be published?
>
>
>>
>>> Trend analysis can be self defeating,
>>>
>>
> How so?
>
>
>> I've discussed this off wiki with some of the people who have had
>>> scholarships in the past, including a couple of people who didn't apply
>>> this year because they assumed they would be declined for Montreal after
>>> having had scholarships recently.
>>>
>>> What might save a lot of time on everyone's part would be if there was a
>>> simple rule such as we don't give the same person a scholarship for two
>>> consecutive Wikimanias. Emphasis on give rather than award as there will be
>>> people who were awarded a scholarship but could not get a visa. That would
>>> reduce the workload  of the scholarship team, and 

Re: [Wikimania-l] WMF Scholarships to attend Wikimania

[Pine W]

On Fri, Apr 21, 2017 at 5:08 AM, Lodewijk 
wrote:

> (not responding to a person in particular)
> I'm a little bit at a loss here. The proposal is to share a lot of
> information from the application process (whether attempted to anonimize or
> not) beyond statistics. Given the high number of countries and other rather
> specific characteristics, anything vaguely useful will likely contain at
> least some personally identifiable information.
>

PII disclosures can be limited to what users have already disclosed in
public (which, admittedly, may not be entirely current and truthful.)
Aggregated information can be provided as well.


> More likely even, anything you can share without being personally
> identifiable will probably not be very relevant for the application
> consideration. Sure, you could do some gender statistics, but how does that
> tell you why people have been rejected?
>

I anticipate that the level of transparency would be insufficient to
evaluate the Scholarship Committee and WMF decisions about individual
applicants. However, the information that is published may still be useful
and of interest when considering trends and groups.


>
> So I'd like to take a step back: what exactly is the problem you're trying
> to solve? Is publishing a lot of data really the best approach to that
> solution? If you define the problem well, I can imagine a few alternative
> approaches, like asking the scholarship committee to report back with an
> analysis of the problem and how they went about it - or asking an
> independent person/persons to sign an NDA, and go into the data,
> investigate and report back. They could actually go in depth - but it
> requires a good definition of the problem.
>

My impression is that there are disappointments and complaints almost every
year about scholarship awards. I hope that increasing transparency will
result in a decreased number and intensity of complaints about individual
cases, and will also increase the amount of information that is made public
which can be used by anyone and everyone to analyze policies and practices
and to make recommendations for refinements or changes as may seem best.

Also, as a broader theme, I would like to see more transparency about how
WMF funds are used. A change of practice like we're discussing here would
be one step in that direction.


>
> Best,
> Lodewijk
>
> 2017-04-21 13:32 GMT+02:00 Jonathan Cardy :
>
>> Hi Pine, I agree with you that partial transparency can be a positive and
>> at least assure people that their region/language/project is getting a fair
>> share even if they were declined. But I'd suggest that can be done with
>> anonymised stats rather than applications with some details redacted or
>> withheld.
>>
>
Let me ask: why shouldn't the usernames of applicants, and whether they
were offered scholarships, be made public in future years if scholarship
applicants are told in advance that this information will be published?


>
>> Trend analysis can be self defeating,
>>
>
How so?


> I've discussed this off wiki with some of the people who have had
>> scholarships in the past, including a couple of people who didn't apply
>> this year because they assumed they would be declined for Montreal after
>> having had scholarships recently.
>>
>> What might save a lot of time on everyone's part would be if there was a
>> simple rule such as we don't give the same person a scholarship for two
>> consecutive Wikimanias. Emphasis on give rather than award as there will be
>> people who were awarded a scholarship but could not get a visa. That would
>> reduce the workload  of the scholarship team, and also of the applicants.
>> You could of course balance that by other factors, I'm hoping thatFrench
>> speakers are being given preference for Montreal.
>>
>
I agree with the general sentiment that giving scholarships to the same
person for multiple consecutive Wikimanias should be avoided. If what I'm
told is true that there are thousands of applicants for only a few hundred
scholarship spots, perhaps the bar should be even higher and scholarships
should be awarded to the same person at most once out of every three years.
It would help to have the information that we're discussing in this thread
be made public so that we can have a better-informed conversation about the
policies for scholarship awards. (:

Pine
___
Wikimania-l mailing list
Wikimania-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimania-l

Re: [Wikimania-l] WMF Scholarships to attend Wikimania

[Lodewijk]

(not responding to a person in particular)
I'm a little bit at a loss here. The proposal is to share a lot of
information from the application process (whether attempted to anonimize or
not) beyond statistics. Given the high number of countries and other rather
specific characteristics, anything vaguely useful will likely contain at
least some personally identifiable information. More likely even, anything
you can share without being personally identifiable will probably not be
very relevant for the application consideration. Sure, you could do some
gender statistics, but how does that tell you why people have been
rejected?

So I'd like to take a step back: what exactly is the problem you're trying
to solve? Is publishing a lot of data really the best approach to that
solution? If you define the problem well, I can imagine a few alternative
approaches, like asking the scholarship committee to report back with an
analysis of the problem and how they went about it - or asking an
independent person/persons to sign an NDA, and go into the data,
investigate and report back. They could actually go in depth - but it
requires a good definition of the problem.

Best,
Lodewijk

2017-04-21 13:32 GMT+02:00 Jonathan Cardy :

> Hi Pine, I agree with you that partial transparency can be a positive and
> at least assure people that their region/language/project is getting a fair
> share even if they were declined. But I'd suggest that can be done with
> anonymised stats rather than applications with some details redacted or
> withheld.
>
> Trend analysis can be self defeating, I've discussed this off wiki with
> some of the people who have had scholarships in the past, including a
> couple of people who didn't apply this year because they assumed they would
> be declined for Montreal after having had scholarships recently.
>
> What might save a lot of time on everyone's part would be if there was a
> simple rule such as we don't give the same person a scholarship for two
> consecutive Wikimanias. Emphasis on give rather than award as there will be
> people who were awarded a scholarship but could not get a visa. That would
> reduce the workload  of the scholarship team, and also of the applicants.
> You could of course balance that by other factors, I'm hoping thatFrench
> speakers are being given preference for Montreal.
>
>
>
>
>
> On 21 April 2017 at 06:14, Pine W  wrote:
>
>> Hi WSC,
>>
>> > Otherwise you have list of applicants and when you query why a decision
>> was made to give a scholarship to one person and not another all that
>> people can say is that "judging by the applications we think we made the
>> right
>> > choice". OK you could redact some data they hopefully ignore such as
>> real name and exact contact details. But simply publishing part of the
>> information used to make a  decision does not enable you to understand how
>> people
>> > came to the decisions they did.
>>
>> My view is that partial transparency is better than none. I don't
>> anticipate that redacted applications will be sufficient for people to make
>> appeals of individual decisions, but what could be of public interest and
>> analyzable from partial transparency are patterns of selections, for
>> example if all 10 applicants from Wikimedia Alaska were awarded
>> scholarships while all 20 applicants from Wikimedia User Group Microsoft
>> were denied scholarships. Also, seeing year-to-year trends would be of
>> interest, such as people who are awarded or denied scholarships for
>> multiple consecutive years.
>>
>> > the community seems to be greying and stabilising. Not many editors
>> under 18 attend Wikimania, and several of the roles that Risker talks of
>> are limited to legal adults; so the decline in our number of minors at a
>> time
>> > of general growth should mean we have many more people available for
>> such roles or who are likely to attend things like Wikimania.
>>
>> Perhaps WMF will want to research whether it's true that the quality of
>> participants and/or number of applicants to online committee roles is
>> declining. On English Wikipedia, the *Signpost *is currently having a
>> near-death experience
>> ,
>> which I consider worrisome and disappointing. I share Risker's concern
>> about the "community health" of online organized groups such as grants
>> committees (as well as WikiProjects, arbitration committees, etc), and
>> would be interested in seeing a holistic analysis of the situation of
>> organized Wikimedia community groups that do most of their work via
>> Internet. The scope of this is a bit different from the scope of Wikimania,
>> so perhaps we can continue discussing this topic on-wiki or on a different
>> mailing list.
>>
>> Pine
>>
>>
>> On Thu, Apr 20, 2017 at 6:43 AM, WereSpielChequers <
>> werespielchequ...@gmail.com> wrote:
>>
>>> 

Re: [Wikimania-l] WMF Scholarships to attend Wikimania

[Jonathan Cardy]

Hi Pine, I agree with you that partial transparency can be a positive and at 
least assure people that their region/language/project is getting a fair share 
even if they were declined. But I'd suggest that can be done with anonymised 
stats rather than applications with some details redacted or withheld.

Trend analysis can be self defeating, I've discussed this off wiki with some of 
the people who have had scholarships in the past, including a couple of people 
who didn't apply this year because they assumed they would be declined for 
Montreal after having had scholarships recently. 

What might save a lot of time on everyone's part would be if there was a simple 
rule such as we don't give the same person a scholarship for two consecutive 
Wikimanias. Emphasis on give rather than award as there will be people who were 
awarded a scholarship but could not get a visa. That would reduce the workload  
of the scholarship team, and also of the applicants. You could of course 
balance that by other factors, I'm hoping thatFrench speakers are being given 
preference for Montreal.





> On 21 April 2017 at 06:14, Pine W  wrote:
> Hi WSC,
> 
> > Otherwise you have list of applicants and when you query why a decision was 
> > made to give a scholarship to one person and not another all that people 
> > can say is that "judging by the applications we think we made the right 
> > choice". OK you could redact some data they hopefully ignore such as real 
> > name and exact contact details. But simply publishing part of the 
> > information used to make a  decision does not enable you to understand how 
> > people 
> > came to the decisions they did.
> 
> My view is that partial transparency is better than none. I don't anticipate 
> that redacted applications will be sufficient for people to make appeals of 
> individual decisions, but what could be of public interest and analyzable 
> from partial transparency are patterns of selections, for example if all 10 
> applicants from Wikimedia Alaska were awarded scholarships while all 20 
> applicants from Wikimedia User Group Microsoft were denied scholarships. 
> Also, seeing year-to-year trends would be of interest, such as people who are 
> awarded or denied scholarships for multiple consecutive years.
> 
> > the community seems to be greying and stabilising. Not many editors under 
> > 18 attend Wikimania, and several of the roles that Risker talks of are 
> > limited to legal adults; so the decline in our number of minors at a time 
> > of general growth should mean we have many more people available for such 
> > roles or who are likely to attend things like Wikimania.
> 
> Perhaps WMF will want to research whether it's true that the quality of 
> participants and/or number of applicants to online committee roles is 
> declining. On English Wikipedia, the Signpost is currently having a 
> near-death experience, which I consider worrisome and disappointing. I share 
> Risker's concern about the "community health" of online organized groups such 
> as grants committees (as well as WikiProjects, arbitration committees, etc), 
> and would be interested in seeing a holistic analysis of the situation of 
> organized Wikimedia community groups that do most of their work via Internet. 
> The scope of this is a bit different from the scope of Wikimania, so perhaps 
> we can continue discussing this topic on-wiki or on a different mailing list.
> 
> Pine
> 
> 
>> On Thu, Apr 20, 2017 at 6:43 AM, WereSpielChequers 
>>  wrote:
>> Dear Pine,
>> 
>> You wouldn't get transparency simply by publishing a list of applicants. You 
>> would only get transparency by publishing a list of applications, including 
>> any other info being used by the scholarship committee.  For example if they 
>> want to give priority to people who they have previously declined, they 
>> could only do that transparently by including previous applications. 
>> Otherwise you have list of applicants and when you query why a decision was 
>> made to give a scholarship to one person and not another all that people can 
>> say is that "judging by the applications we think we made the right choice". 
>> OK you could redact some data they hopefully ignore such as real name and 
>> exact contact details. But simply publishing part of the information used to 
>> make a decision does not enable you to understand how people came to the 
>> decisions they did.
>> 
>> As for whether the community is plateauing or growing, from the stats I 
>> monitor or help maintain, the English Wikipedia community at least has 
>> rebounded significantly since the 2014 low. More importantly from the 
>> perspective of things like Wikimania, the community seems to be greying and 
>> stabilising. Not many editors under 18 attend Wikimania, and several of the 
>> roles that Risker talks of are limited to legal adults; so the decline in 
>> our number of minors at a time of general