Thanks Lodewijk for answering my questions. I don't find your feedback moot and it's actually quite helpful. From what you're saying, it sounds like opening up feedback to those who reported data would help to solidify the content of the report before pushing the announcement publicly. We have 8 more program reports to publish and I'm starting to think of ways we might include a window for this kind of feedback, but I would need to check with the rest of the team and our timelines to know what is feasible. We have been extra busy these last few months.
Also, to make a clarification, we don't assume that all Wiki Loves Monuments are the same. The metrics we collect are fairly broad and not exhaustive so that we can first know the collective impact of the program and then we can dig deeper and learn about the contests in greater detail afterward. In the coming months, we will be doing one-on-one interviews with organizers to surface the processes and goals of several photo contests, and learn what works and what doesn't in different contexts. This would be the opportunity to explore these assumptions and questions. Thanks again for your helpful suggestions, Edward On Thu, May 7, 2015 at 9:46 PM, Lodewijk <lodew...@effeietsanders.org> wrote: > Hi Edward, > > Thanks for the questions. The Wiki Loves Monuments mailing list would have > made a very logical starting place to ask for initial feedback. But also > sending an email to the people who shared their data with you to work with > in the first place, or people who worked on internal evaluations in these > projects before. > > The feeling has been created that right now, the 'damage is done': the > report is published, you have done all you could to make sure that all > community members are as much aware as possible of what you consider the > conclusions. That means that any feedback now, becomes somewhat moot. We > have seen this before with Foundation publications (i.e. statistics on the > chapters), once it is announced to the community at large, feedback often > doesn't get incorporated any more (I hope this time it does!) and even if > it is, the "facts" already found their place into other publications like > the signpost. Asking feedback is most valuable *before* you announce it, > and proactively. You could (even better) consider involving those > stakeholders even earlier in the process, which makes it less of a black > box. > > I strongly believe that it would improve the quality of the work you do. > Still some of the basic flaws will remain due to the basic setup of the > evaluation framework (assumptions that all WLM are comparable etc.) but > others could be managed better. > > Best, > Lodewijk > > On Fri, May 8, 2015 at 3:47 AM, Edward Galvez <egal...@wikimedia.org> > wrote: > > > > > > > > > > > > Hi Lodewijk, > > > > Thanks for your feedback about the process. It's been very valuable. > > > > I have a few follow up questions below: > > > > > > > Sure, the team did reach out in the collection phase - after all, > without > > > the data such evaluation would be impossible. But after that, the > > > conclusions were drafted and shared with the wide community, rather > than > > > with the stakeholders involved to discuss interpretation. > > > > > > > Can you say more about which stakeholders? Do you have ideas how we might > > include them in the future, for example, through the Wiki Loves Monuments > > mailing list, or were you thinking in some other way? > > > > > > Either way, all communication seemed to be aimed to announce the > > > evaluation, rather than to ask active input on whether the analysis > made > > > sense, whether there were misunderstandings, etc. But maybe you have > had > > a > > > lot of follow-up discussions with the people you collected data from > on a > > > 1-to-1 level, which would be admirable. > > > > > > > We tried to encourage input and questions through the next steps and in > the > > talk page, but it sounds like this might not have been enough. How do you > > think we can do this better next time? Anything specific that stands out > to > > you, beyond sharing with stakeholders beforehand? > > > > Thanks so much, > > Edward > > > > > > > > > > > Again, I do appreciate the effort, I don't agree with the approach and > > > process. > > > > > > Best, > > > Lodewijk > > > _______________________________________________ > > > Wikimedia-l mailing list, guidelines at: > > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > > Wikimediafirstname.lastname@example.org > > > < > > > https://meta.wikimedia.org/wiki/Mailing_lists/guidelineswikimedi...@lists.wikimedia.org > > > > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> > > > > > > > > > > > -- > > Edward Galvez > > Program Evaluation Associate > > Wikimedia Foundation > > _______________________________________________ > > Wikimedia-l mailing list, guidelines at: > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > Wikimediaemail@example.com > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> > > > _______________________________________________ > Wikimedia-l mailing list, guidelines at: > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > Wikimediafirstname.lastname@example.org > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> > -- Edward Galvez Program Evaluation Associate Wikimedia Foundation _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimediaemail@example.com Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>