Re: [Wikimedia-l] Funds Dissemination Committee first progress reports
I am a little disappointed at the focus by WMF staff on quantitative metrics over everything else, which I think may have the unfortunate side-effect of encouraging entities to go after easily measurable activities rather than the most effective and worthwhile activities. Hopefully this will be taken into account on future assessments. +1 It is quick to target the easily measurable, but will it actually bring us forward? Activities and outputs are only a means to an end. So instead of setting the focus on easily measurable means, I would personally prefer a focus on building up the movement's knowledge about sustainable outcomes and on how to get there. Cheers, Manuel -- Manuel Merz Stabsstelle Evaluation / Evaluation Unit Wikimedia Deutschland e.V. Obentrautstr. 72, 10963 Berlin, Germany www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985. 2013/6/12 Craig Franklin cfrank...@halonetwork.net Given that this is an assessment that is being performed by paid staff, I think it's unreasonable to think that the staff would issue more than very mild criticism (Your report is so great it makes everyone else look terrible!), even if the report was so poor as to deserve criticism. I'm not saying that it *is*, but I don't think anyone that values their job would carpet their employer in a public forum, even if the employer invited them to do so. There should certainly be a note in this report to declare the massive COI involved in having WMF staff 'critically' assessing a WMF report. That said, I do find the assessment for everyone else useful in terms of seeing what the WMF staff will think, and I'm sure that chapters considering an FDC application will take that on board. I am a little disappointed at the focus by WMF staff on quantitative metrics over everything else, which I think may have the unfortunate side-effect of encouraging entities to go after easily measurable activities rather than the most effective and worthwhile activities. Hopefully this will be taken into account on future assessments. Cheers, Craig Franklin On 12 June 2013 20:52, Federico Leva (Nemo) nemow...@gmail.com wrote: Katy Love, 11/06/2013 22:52: [2] http://meta.wikimedia.org/**wiki/FDC_portal/Proposals/** 2012-2013_round1/Staff_**summary/Progress_report_form/**Q1 http://meta.wikimedia.org/wiki/FDC_portal/Proposals/2012-2013_round1/Staff_summary/Progress_report_form/Q1 Funny: «WMF notes [stats]», «WMFR claims [stats]». Nemo __**_ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.**org Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/**mailman/listinfo/wikimedia-l https://lists.wikimedia.org/mailman/listinfo/wikimedia-l ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Funds Dissemination Committee first progress reports
hi, a few thoughts: On Fri, Jun 14, 2013 at 1:37 PM, Manuel Merz manuel.m...@wikimedia.dewrote: I am a little disappointed at the focus by WMF staff on quantitative metrics over everything else, which I think may have the unfortunate side-effect of encouraging entities to go after easily measurable activities rather than the most effective and worthwhile activities. Hopefully this will be taken into account on future assessments. +1 It is quick to target the easily measurable, but will it actually bring us forward? Activities and outputs are only a means to an end. So instead of setting the focus on easily measurable means, I would personally prefer a focus on building up the movement's knowledge about sustainable outcomes and on how to get there. I personally believe that there may be some confusion about the goals here. What is important is to seek quantitative metrics WHEN APPLICABLE. In all goals and projects where quantitative metrics are impossible to offer, it is perfectly reasonable to state so and justify, why. The thing is that many organizations have the problem of not thinking about measuring results as well as about making impact at all - it is just very easy to assume that what we do (and enjoy doing) makes sense by default and avoid reflecting upon it. The process of thinking about measuring outcomes is important in itself, even if it leads to the conclusion that in some cases the results will not be easily quantifiable. As long as entities realize that and reflect on the reasons of the impossibility to measure results, I myself would not see any problems with accepting such an approach. Typically, all good ideas have some outcomes that can be quantified, as well as some that can't (or shouldn't). best, Dariusz Jemielniak pundit (expressing my own view, and not in the capacity of the FDC chair). ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Funds Dissemination Committee first progress reports
Thanks SJ for these thoughts, it's gratifying and encouraging that we have a WMF trustee on the case :-) While getting chapter staff to likewise review reports is a good idea, there are two potential problems that I can see with it: 1. Chapter staff may be unwilling to criticise the reports of other chapters that they're hoping to embark on joint projects with, and; 2. The various funding programs available through the WMF (FDC, GAC) make no secret of the fact that they want staff to be doing programme work, *not* administrative or overhead work. It would be difficult for most chapters to spare the resources to do this properly. Perhaps the movement could look at getting an external firm in to do the assessment? It would probably be costly, but if the firm is properly chosen it should at least minimise any COI concerns. Of course, their reporting can and should be supported by vigourous assessment by the community. Cheers, Craig On 15 June 2013 12:02, Samuel Klein meta...@gmail.com wrote: On Fri, Jun 14, 2013 at 1:37 PM, Manuel Merz manuel.m...@wikimedia.de wrote: I am a little disappointed at the focus by WMF staff on quantitative I find it distracting, though currently accurate, that this is framed as a WMF staff focus. The report makes a point of taling about FDC staff instead. How can we set up FDC support, from across the movement, so that we stop talking about WMF staff and start talking about staff supporting the FDC? In my view, this should be a mix of [staff] from across the movement. This does not get away from the COI problem of having movement entities reviewing how well they are doing, but it adds some of the natural checks and balances of peer review. (I put [staff] in brackets because this could also include FDC support that are not staff. Indeed some aspects of COI suggest that any evaluation group should include non-staff as well.) On Fri, Jun 14, 2013 at 8:24 AM, Dariusz Jemielniak dar...@alk.edu.pl wrote: On Fri, Jun 14, 2013 at 1:37 PM, Manuel Merz manuel.m...@wikimedia.de wrote: [focusing on] quantitative metrics over everything else... may have the unfortunate side-effect of encouraging entities to go after easily measurable activities rather than the most effective and worthwhile activities. instead of setting the focus on easily measurable means, I would personally prefer a focus on building up the movement's knowledge about sustainable outcomes and on how to get there. I agree with Manuel here: we should focus on how to build the movement's knowledge about the most helpful, generative, and sustainable outcomes. And how to expand this knowledge: experiments that will help us learn more about what is possible. (This is important exploration, even if the result of an experiment is not immediately impactful) Dariusz is also right to note that most ideas have some outcomes that can be quantified, and some that cannot: and it is useful to identify each group of outcome. I personally believe that there may be some confusion about the goals here. What is important is to seek quantitative metrics WHEN APPLICABLE... Typically, all good ideas have some outcomes that can be quantified, as well as some that can't (or shouldn't). Regards, Sam ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Funds Dissemination Committee first progress reports
On Fri, Jun 14, 2013 at 11:43 PM, Craig Franklin cfrank...@halonetwork.net wrote: Thanks SJ for these thoughts, it's gratifying and encouraging that we have a WMF trustee on the case :-) The Board FDC liaisons are putting more thought into this than I am; I'm just brainstorming quickly on the mailing list and encouraging more of the same :-) While getting chapter staff to likewise review reports is a good idea, there are two potential problems that I can see with it: 1. Chapter staff may be unwilling to criticise the reports of other chapters that they're hoping to embark on joint projects with Yes. WMF staff may have the same potential concerns (and joint projects). Other options such as outside review (as you suggest) are also available; but these brief reviews should be much less difficult and controversial than the FDC decisions. 2. The various funding programs available through the WMF (FDC, GAC) make no secret of the fact that they want staff to be doing programme work, *not* administrative or overhead work. It would be difficult for most chapters to spare the resources to do this properly. Perhaps. If we're organizing an increasing number of things into programs with plans, timelines, and metrics: then every community needs to develop some facility for refactoring, reviewing, measuring, and tracking projects. That sort of self-reflection is essential to daily work, and should happen regularly at the lowest possible level; so I'm not comfortable framing it as costly overhead. That's like saying that organizing an RfC is costly overhead. Moreover some of the program work of local groups involves overseeing microgrants. Which requires specific facility in this sort of review. Perhaps the movement could look at getting an external firm in to do the assessment? It would probably be costly, but if the firm is properly chosen it should at least minimise any COI concerns. Of course, their reporting can and should be supported by vigourous assessment by the community. This is certainly an option if it proves essential and worth the expense. I'd like to see how simply and inexpensively we can accomplish the same thing, however. I'd rather see this become less of a big deal - a rolling process that many people can contribute to, in steps - than more of one. SJ ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Funds Dissemination Committee first progress reports
Dariusz Jemielniak pundit (expressing my own view, and not in the capacity of the FDC chair). To be extra clear, as Dariusz was: anything I say in this thread is my own view, not a statement in the capacity of WMF Trustee. Freeform discussion of what is possible is important, and I hope people will share their thoughts even if they are not one of the parties involved. SJ On Sat, Jun 15, 2013 at 12:46 AM, Samuel Klein meta...@gmail.com wrote: On Fri, Jun 14, 2013 at 11:43 PM, Craig Franklin cfrank...@halonetwork.net wrote: Thanks SJ for these thoughts, it's gratifying and encouraging that we have a WMF trustee on the case :-) The Board FDC liaisons are putting more thought into this than I am; I'm just brainstorming quickly on the mailing list and encouraging more of the same :-) While getting chapter staff to likewise review reports is a good idea, there are two potential problems that I can see with it: 1. Chapter staff may be unwilling to criticise the reports of other chapters that they're hoping to embark on joint projects with Yes. WMF staff may have the same potential concerns (and joint projects). Other options such as outside review (as you suggest) are also available; but these brief reviews should be much less difficult and controversial than the FDC decisions. 2. The various funding programs available through the WMF (FDC, GAC) make no secret of the fact that they want staff to be doing programme work, *not* administrative or overhead work. It would be difficult for most chapters to spare the resources to do this properly. Perhaps. If we're organizing an increasing number of things into programs with plans, timelines, and metrics: then every community needs to develop some facility for refactoring, reviewing, measuring, and tracking projects. That sort of self-reflection is essential to daily work, and should happen regularly at the lowest possible level; so I'm not comfortable framing it as costly overhead. That's like saying that organizing an RfC is costly overhead. Moreover some of the program work of local groups involves overseeing microgrants. Which requires specific facility in this sort of review. Perhaps the movement could look at getting an external firm in to do the assessment? It would probably be costly, but if the firm is properly chosen it should at least minimise any COI concerns. Of course, their reporting can and should be supported by vigourous assessment by the community. This is certainly an option if it proves essential and worth the expense. I'd like to see how simply and inexpensively we can accomplish the same thing, however. I'd rather see this become less of a big deal - a rolling process that many people can contribute to, in steps - than more of one. SJ -- Samuel Klein @metasj w:user:sj +1 617 529 4266 ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Funds Dissemination Committee first progress reports
Katy Love, 11/06/2013 22:52: [2] http://meta.wikimedia.org/wiki/FDC_portal/Proposals/2012-2013_round1/Staff_summary/Progress_report_form/Q1 Funny: «WMF notes [stats]», «WMFR claims [stats]». Nemo ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Funds Dissemination Committee first progress reports
Given that this is an assessment that is being performed by paid staff, I think it's unreasonable to think that the staff would issue more than very mild criticism (Your report is so great it makes everyone else look terrible!), even if the report was so poor as to deserve criticism. I'm not saying that it *is*, but I don't think anyone that values their job would carpet their employer in a public forum, even if the employer invited them to do so. There should certainly be a note in this report to declare the massive COI involved in having WMF staff 'critically' assessing a WMF report. That said, I do find the assessment for everyone else useful in terms of seeing what the WMF staff will think, and I'm sure that chapters considering an FDC application will take that on board. I am a little disappointed at the focus by WMF staff on quantitative metrics over everything else, which I think may have the unfortunate side-effect of encouraging entities to go after easily measurable activities rather than the most effective and worthwhile activities. Hopefully this will be taken into account on future assessments. Cheers, Craig Franklin On 12 June 2013 20:52, Federico Leva (Nemo) nemow...@gmail.com wrote: Katy Love, 11/06/2013 22:52: [2] http://meta.wikimedia.org/**wiki/FDC_portal/Proposals/** 2012-2013_round1/Staff_**summary/Progress_report_form/**Q1http://meta.wikimedia.org/wiki/FDC_portal/Proposals/2012-2013_round1/Staff_summary/Progress_report_form/Q1 Funny: «WMF notes [stats]», «WMFR claims [stats]». Nemo __**_ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.**org Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/**mailman/listinfo/wikimedia-lhttps://lists.wikimedia.org/mailman/listinfo/wikimedia-l ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Funds Dissemination Committee first progress reports
Hi Katy, thanks for sharing. As a minor suggestion, to me it would be helpful if you could be a little more specific than 'we' in your responses. Are you in those instances speaking for your own (with your specific expertise), for a group of WMF staffers, for the FDC, for the whole Foundation etc? Knowing that would make it a bit more insightful. Lodewijk 2013/6/12 Craig Franklin cfrank...@halonetwork.net Given that this is an assessment that is being performed by paid staff, I think it's unreasonable to think that the staff would issue more than very mild criticism (Your report is so great it makes everyone else look terrible!), even if the report was so poor as to deserve criticism. I'm not saying that it *is*, but I don't think anyone that values their job would carpet their employer in a public forum, even if the employer invited them to do so. There should certainly be a note in this report to declare the massive COI involved in having WMF staff 'critically' assessing a WMF report. That said, I do find the assessment for everyone else useful in terms of seeing what the WMF staff will think, and I'm sure that chapters considering an FDC application will take that on board. I am a little disappointed at the focus by WMF staff on quantitative metrics over everything else, which I think may have the unfortunate side-effect of encouraging entities to go after easily measurable activities rather than the most effective and worthwhile activities. Hopefully this will be taken into account on future assessments. Cheers, Craig Franklin On 12 June 2013 20:52, Federico Leva (Nemo) nemow...@gmail.com wrote: Katy Love, 11/06/2013 22:52: [2] http://meta.wikimedia.org/**wiki/FDC_portal/Proposals/** 2012-2013_round1/Staff_**summary/Progress_report_form/**Q1 http://meta.wikimedia.org/wiki/FDC_portal/Proposals/2012-2013_round1/Staff_summary/Progress_report_form/Q1 Funny: «WMF notes [stats]», «WMFR claims [stats]». Nemo __**_ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.**org Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/**mailman/listinfo/wikimedia-l https://lists.wikimedia.org/mailman/listinfo/wikimedia-l ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
[Wikimedia-l] Funds Dissemination Committee first progress reports
Greetings, everyone! Are you curious about what the Funds Dissemination Committee (FDC) Round 1 grantees have been working on these last few months? If you haven't already seen the first progress reports submitted by the FDC Round 1 grantees, come on over and check them out! To find these first quarter progress reports, go to the Round 1 hub on the FDC portal and click on the progress report form Q1 for any of the Round 1 entities [1]. I want to thank all the entities for sharing their progress and learning with us; we have really enjoyed reading the updates and look forward to continuing to learn from them. Second, the FDC staff published a summary of the first progress reports for the FDC. [2] This summary shares some emerging themes and an overview of each of the entity's work to date on programmatic, organizational and financial progress. We have also posted more detailed feedback and questions on the discussion page of all of the individual reports. As ever, contact me with questions or comments! Warm regards, Katy [1] http://meta.wikimedia.org/wiki/FDC_portal/Proposals/2012-2013_round1 [2] http://meta.wikimedia.org/wiki/FDC_portal/Proposals/2012-2013_round1/Staff_summary/Progress_report_form/Q1 ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
