[Wikimedia-l] Effectiveness of meetings (Was: Affcom ...)
I think this is very true, and we could perhaps improve our procedures and documentation in many ways. However, I do think it is important to realize that you're comparing apples with oranges here. The meeting in Brussels for example was on the topic of influencing legislative processes - a topic by definition long term and hard to measure. The goals there would be better (copyright) legislation than without these initiatives - and rather strategic. The bootcamp in DC was intended as a boost for volunteers, where they would get a lot of information and experience in a short time: capacity building. This is also long term, but much better to measure. And finally there's the hackathon, which is much more short-term focused (at least partially), and will have very concrete results (but probably zero measured in new articles created). What I think is most important, is that for every meeting we have, we identify desired outcomes. These outcomes can be very tangible, or intangible - but they should always be defined, and the agenda should be built around it. Based on these desired outcomes, you can estimate up front if the meeting is likely to be 'worth it'. If not, you can reduce the costs, increase the outcomes (different setup) or cancel all together. Finally, after the meeting a report (private or public - there should always be a report or documentation) should be produced and if possible published. In that report you should always return to these desired outcomes, and see if they were met - and how/why not. But even then on the strategy side of things meetings are always hard to estimate. Of course you're totally right that there are expectations towards participants of meetings. Not only because of the money invested, but also because of the attention of the other participants. I definitely have been in meetings where people literally fell asleep or played games during the meeting - and that is simply insulting to the other people in the room. So yeah, if that is your mindset, perhaps it is better not to go at all. But then I am assuming good faith, and think that everyone will be going to meetings with the best of intentions, and not simply to play tourist. Lodewijk 2013/5/14 Balázs Viczián balazs.vicz...@wikimedia.hu 40k for a single meetup can be justified if the results worth that much money. (I've already argued about this regarding Milan chapters meetup btw). Providing a more than basic travel and accomodation can be a way of appreciation as well for their work, what they do as volunteers, that should be calculated into the costs. Ain't these intrenational events' (not just this meetup's, but all events') success ratio being measured by some way already? The cost/benefit ratio [1] is a pretty basic (and extreme important) thing we like to calculate with here in Hungary about all of our activities. For example the number of articles created divided with the total costs of the article writing contest they've been created within gives a number we can work a lot with to improve cost-effectiveness. Seemingly very few chapters doing anything similar (or not in a visible way) AffCom already measures itself in some ways in their reports, but regarding other meetups, I've barely seen at least a basic followup or aftercare and especially not a detailed overall (measurement) report what is usually /at least in those commercial events I was involved with/ being published after about six month of the last day of the event, and gives a detailed summary of its pros and cons, dos and don'ts, successes and fails, overall impact (upon proactively collected feedbacks), etc. There were plenty of international events already this year (Brussels meeting about EU policies, Milan, London glam, Amsterdam hackathon that are coming in my mind right now from 2013, and this is just the first 4 month (only 1/3rd of the year) and not the full list!) most of them with no or very low visible results yet (ok they need some time to evolve, but regarding events in 2012, I barely read anything to remember nor any followups or summares, reports). Compared to the GLAM camp in the US recently, it seems definitely true. The latter looks like a very good example of a beneficial meetup, having a lot of potential and it seems there is a chance that it will be followed up by WMDC and other participating parties well after the event. Why to have a long term followup? To give a definite answer wheter those potentials actually resulted in anything at all (was there any real benefit) or it was just a very good mooded, fun and positive, but totally fruitless event (wasted money from the movement's POV) I see compared to 2012 costs going up without any visible rise in effectiveness or more worse, a decline in it. This is solely based upon what I can (or can not) read about them on meta and other places, like the comments here. Wikimania 2014 was the first event ever where this thing was taken
Re: [Wikimedia-l] Effectiveness of meetings (Was: Affcom ...)
I see a few misunderstanding here: I do not wish to compare them to each other, but to the previous one(s). At least I hope this is what I've written because this is what I was ment. So compare GLAM 2013 to GLAM 2012, and GLAM 2011, ..., compare Brussels meetup with previous similar thematic meetups and so forth. Create a breakdown about the costs and the benefits. Aimed at GLAM? then the main thing to measue how did it boost GLAM cooperations. Is there any initiative that had been picked up or created upon the inspiration they got there? If yes, how much? What types? What are their outcomes? Any other side goals (like international cooperations - like a new WLM participant who decided to join after discussions there)? Any other unexpected benefits? (like an out of the blue content donation from a participating G,L,A or M representative) etc. Easy to identify a dozen questions, plus if you start thinking it over, you'll find a dozen more that are as well pretty important, just not that much visible or in front. Count them and divide the total number with the total costs, and you'll get how much was invested in a single initiative there. If an idea was brought home by a chapter from that event (or the local ideas have been influenced by it), note even if locally it would cost almost nothing, on an absolute level, there is this cost/benefit ratio, both the chapter's and the event's, what you have to further divide it locally amongst the number of projects. If an example helps you, lets say you brought home 2 article writing contest ideas, both of them conducted and did cost nothing, except a few merchandise (total of 200USD) your travel and stay there should be added to the total costs, since without it, these would likely never been conducted at you. Hence the absolute numbers would include your flight and stay in London, divided equally in between the projects that meeting inspired. Got me? Balázs 2013/5/14 Lodewijk lodew...@effeietsanders.org I think this is very true, and we could perhaps improve our procedures and documentation in many ways. However, I do think it is important to realize that you're comparing apples with oranges here. The meeting in Brussels for example was on the topic of influencing legislative processes - a topic by definition long term and hard to measure. The goals there would be better (copyright) legislation than without these initiatives - and rather strategic. The bootcamp in DC was intended as a boost for volunteers, where they would get a lot of information and experience in a short time: capacity building. This is also long term, but much better to measure. And finally there's the hackathon, which is much more short-term focused (at least partially), and will have very concrete results (but probably zero measured in new articles created). What I think is most important, is that for every meeting we have, we identify desired outcomes. These outcomes can be very tangible, or intangible - but they should always be defined, and the agenda should be built around it. Based on these desired outcomes, you can estimate up front if the meeting is likely to be 'worth it'. If not, you can reduce the costs, increase the outcomes (different setup) or cancel all together. Finally, after the meeting a report (private or public - there should always be a report or documentation) should be produced and if possible published. In that report you should always return to these desired outcomes, and see if they were met - and how/why not. But even then on the strategy side of things meetings are always hard to estimate. Of course you're totally right that there are expectations towards participants of meetings. Not only because of the money invested, but also because of the attention of the other participants. I definitely have been in meetings where people literally fell asleep or played games during the meeting - and that is simply insulting to the other people in the room. So yeah, if that is your mindset, perhaps it is better not to go at all. But then I am assuming good faith, and think that everyone will be going to meetings with the best of intentions, and not simply to play tourist. Lodewijk 2013/5/14 Balázs Viczián balazs.vicz...@wikimedia.hu 40k for a single meetup can be justified if the results worth that much money. (I've already argued about this regarding Milan chapters meetup btw). Providing a more than basic travel and accomodation can be a way of appreciation as well for their work, what they do as volunteers, that should be calculated into the costs. Ain't these intrenational events' (not just this meetup's, but all events') success ratio being measured by some way already? The cost/benefit ratio [1] is a pretty basic (and extreme important) thing we like to calculate with here in Hungary about all of our activities. For example the number of articles created divided with the total costs of the article
Re: [Wikimedia-l] Effectiveness of meetings (Was: Affcom ...)
I totally understand (or rather, I think I do). However, just like with any project, also such calculations would be subject to cost/benefit ratios. What information do you expect to win, and how much overhead does it create? It sounds great to be able to put a number on things, but up to a certain size, things will be mostly intangible anyway (enthusiasm, inspiration etc.) so you will have little valuable output of such measurements, while the costs are relatively high (they need to learn speak your measure language, they need to understand concepts, document thoroughly, measure etc.) and for small events that seems unreasonable to me to expect it from them - such as with the article writing contests in 'new' countries. However, this is of course different for large international events with many participants. Anyway, I am glad to hear you're not trying to compare different categories of events with each other. Lodewijk 2013/5/14 Balázs Viczián balazs.vicz...@wikimedia.hu I see a few misunderstanding here: I do not wish to compare them to each other, but to the previous one(s). At least I hope this is what I've written because this is what I was ment. So compare GLAM 2013 to GLAM 2012, and GLAM 2011, ..., compare Brussels meetup with previous similar thematic meetups and so forth. Create a breakdown about the costs and the benefits. Aimed at GLAM? then the main thing to measue how did it boost GLAM cooperations. Is there any initiative that had been picked up or created upon the inspiration they got there? If yes, how much? What types? What are their outcomes? Any other side goals (like international cooperations - like a new WLM participant who decided to join after discussions there)? Any other unexpected benefits? (like an out of the blue content donation from a participating G,L,A or M representative) etc. Easy to identify a dozen questions, plus if you start thinking it over, you'll find a dozen more that are as well pretty important, just not that much visible or in front. Count them and divide the total number with the total costs, and you'll get how much was invested in a single initiative there. If an idea was brought home by a chapter from that event (or the local ideas have been influenced by it), note even if locally it would cost almost nothing, on an absolute level, there is this cost/benefit ratio, both the chapter's and the event's, what you have to further divide it locally amongst the number of projects. If an example helps you, lets say you brought home 2 article writing contest ideas, both of them conducted and did cost nothing, except a few merchandise (total of 200USD) your travel and stay there should be added to the total costs, since without it, these would likely never been conducted at you. Hence the absolute numbers would include your flight and stay in London, divided equally in between the projects that meeting inspired. Got me? Balázs 2013/5/14 Lodewijk lodew...@effeietsanders.org I think this is very true, and we could perhaps improve our procedures and documentation in many ways. However, I do think it is important to realize that you're comparing apples with oranges here. The meeting in Brussels for example was on the topic of influencing legislative processes - a topic by definition long term and hard to measure. The goals there would be better (copyright) legislation than without these initiatives - and rather strategic. The bootcamp in DC was intended as a boost for volunteers, where they would get a lot of information and experience in a short time: capacity building. This is also long term, but much better to measure. And finally there's the hackathon, which is much more short-term focused (at least partially), and will have very concrete results (but probably zero measured in new articles created). What I think is most important, is that for every meeting we have, we identify desired outcomes. These outcomes can be very tangible, or intangible - but they should always be defined, and the agenda should be built around it. Based on these desired outcomes, you can estimate up front if the meeting is likely to be 'worth it'. If not, you can reduce the costs, increase the outcomes (different setup) or cancel all together. Finally, after the meeting a report (private or public - there should always be a report or documentation) should be produced and if possible published. In that report you should always return to these desired outcomes, and see if they were met - and how/why not. But even then on the strategy side of things meetings are always hard to estimate. Of course you're totally right that there are expectations towards participants of meetings. Not only because of the money invested, but also because of the attention of the other participants. I definitely have been in meetings where people literally fell asleep or played games during the meeting -
Re: [Wikimedia-l] Effectiveness of meetings (Was: Affcom ...)
Theoratically every penny spent should be measured wheter it was spent well, or was wasted, including salaries. Ideally you could find out all the direct and indirect investment in a given program. Since the latter is extremely hard (or impossible) to measure, it is always an estimated extra cost. The emphasis here is not on the programs a meetup may trigger, but the meetup itself; its effectiveness. (it is a program itself) Regarding evalulation of programs through a meetup's perspective (if I understood your words correctly), there is no point going any deeper than the name of it, the type of it, and a note that it wasdone, in progress , failed or postponed as of the date of publishing. Details should be carefully adjusted for all questions you wish to answer. 90% of evaluating an event is actually done through surveys collected from participants on site (!) and after a given period (somewhere between three to six month), by evaluating the answers and feedbacks you collect _proactively_ (a.k.a. by asking) The rest 10% is an evaluation summary of the catering, the venue and the executing staff. Six month later a summary was published and an evaluation meetup was held before we started organizing the next similar event (what's content was heavily influenced by that that report). Cheers, Balázs 2013/5/14 Lodewijk lodew...@effeietsanders.org I totally understand (or rather, I think I do). However, just like with any project, also such calculations would be subject to cost/benefit ratios. What information do you expect to win, and how much overhead does it create? It sounds great to be able to put a number on things, but up to a certain size, things will be mostly intangible anyway (enthusiasm, inspiration etc.) so you will have little valuable output of such measurements, while the costs are relatively high (they need to learn speak your measure language, they need to understand concepts, document thoroughly, measure etc.) and for small events that seems unreasonable to me to expect it from them - such as with the article writing contests in 'new' countries. However, this is of course different for large international events with many participants. Anyway, I am glad to hear you're not trying to compare different categories of events with each other. Lodewijk 2013/5/14 Balázs Viczián balazs.vicz...@wikimedia.hu I see a few misunderstanding here: I do not wish to compare them to each other, but to the previous one(s). At least I hope this is what I've written because this is what I was ment. So compare GLAM 2013 to GLAM 2012, and GLAM 2011, ..., compare Brussels meetup with previous similar thematic meetups and so forth. Create a breakdown about the costs and the benefits. Aimed at GLAM? then the main thing to measue how did it boost GLAM cooperations. Is there any initiative that had been picked up or created upon the inspiration they got there? If yes, how much? What types? What are their outcomes? Any other side goals (like international cooperations - like a new WLM participant who decided to join after discussions there)? Any other unexpected benefits? (like an out of the blue content donation from a participating G,L,A or M representative) etc. Easy to identify a dozen questions, plus if you start thinking it over, you'll find a dozen more that are as well pretty important, just not that much visible or in front. Count them and divide the total number with the total costs, and you'll get how much was invested in a single initiative there. If an idea was brought home by a chapter from that event (or the local ideas have been influenced by it), note even if locally it would cost almost nothing, on an absolute level, there is this cost/benefit ratio, both the chapter's and the event's, what you have to further divide it locally amongst the number of projects. If an example helps you, lets say you brought home 2 article writing contest ideas, both of them conducted and did cost nothing, except a few merchandise (total of 200USD) your travel and stay there should be added to the total costs, since without it, these would likely never been conducted at you. Hence the absolute numbers would include your flight and stay in London, divided equally in between the projects that meeting inspired. Got me? Balázs 2013/5/14 Lodewijk lodew...@effeietsanders.org I think this is very true, and we could perhaps improve our procedures and documentation in many ways. However, I do think it is important to realize that you're comparing apples with oranges here. The meeting in Brussels for example was on the topic of influencing legislative processes - a topic by definition long term and hard to measure. The goals there would be better (copyright) legislation than without these initiatives - and rather strategic. The bootcamp in DC was intended as a boost for volunteers, where they would
Re: [Wikimedia-l] Effectiveness of meetings (Was: Affcom ...)
Le 14/05/2013 13:39, Lodewijk a écrit : (...) Based on these desired outcomes, you can estimate up front if the meeting is likely to be 'worth it'. If not, you can reduce the costs, increase the outcomes (different setup) or cancel all together. 2013/5/14 Lodewijk lodew...@effeietsanders.org The second relevant question is, in my opinion, whether the WMF travel policy is good, proportional etc. This policy has mostly received criticism from two sides - some think it is too elaborate, and WMF should get 'less luxury' (again fair discussion, but we should then focus on the whole question) - another criticism is that it should be equalized for all Wikimedia-sponsored trips, including individual engagement grants, trips to the chapters meeting etc. When this question was brought up last time, I believe it was Sue who mentioned that this would simply result in much less travel and participation. Again, a fair question. this would simply result in much less travel !!! Well, less travel, not only cost control, could be a justifiable objective itself. I mean that to convert the donations to Wikimedia into burning vast amounts of under-taxed kerosene is definitely not an environmentally friendly and responsible way to run the Wikimedia movement. I personally went to the chapter meeting in Milan rather than insist on attending the next Wikimania in good part because I could get there by train. -- Mathias Damour 49 rue Carnot F-74000 Annecy 00 33 (0)4 57 09 10 56 00 33 (0)6 27 13 65 51 mathias.dam...@laposte.net http://fr.vikidia.org/wiki/Utilisateur:Astirmays ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Effectiveness of meetings (Was: Affcom ...)
I have sat through so many meetings that wasted time in various degrees that I thought of a football referee system where people could hold us yellow or red cards if they thought the meeting should be 'warned' or 'sent off'. More than two cards and the meeting gets abandoned or sorts itself out. On 14 May 2013 18:16, Mathias Damour mathias.dam...@laposte.net wrote: Le 14/05/2013 13:39, Lodewijk a écrit : (...) Based on these desired outcomes, you can estimate up front if the meeting is likely to be 'worth it'. If not, you can reduce the costs, increase the outcomes (different setup) or cancel all together. 2013/5/14 Lodewijk lodew...@effeietsanders.org The second relevant question is, in my opinion, whether the WMF travel policy is good, proportional etc. This policy has mostly received criticism from two sides - some think it is too elaborate, and WMF should get 'less luxury' (again fair discussion, but we should then focus on the whole question) - another criticism is that it should be equalized for all Wikimedia-sponsored trips, including individual engagement grants, trips to the chapters meeting etc. When this question was brought up last time, I believe it was Sue who mentioned that this would simply result in much less travel and participation. Again, a fair question. this would simply result in much less travel !!! Well, less travel, not only cost control, could be a justifiable objective itself. I mean that to convert the donations to Wikimedia into burning vast amounts of under-taxed kerosene is definitely not an environmentally friendly and responsible way to run the Wikimedia movement. I personally went to the chapter meeting in Milan rather than insist on attending the next Wikimania in good part because I could get there by train. -- Mathias Damour 49 rue Carnot F-74000 Annecy 00 33 (0)4 57 09 10 56 00 33 (0)6 27 13 65 51 mathias.dam...@laposte.net http://fr.vikidia.org/wiki/**Utilisateur:Astirmayshttp://fr.vikidia.org/wiki/Utilisateur:Astirmays __**_ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.**org Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/**mailman/listinfo/wikimedia-lhttps://lists.wikimedia.org/mailman/listinfo/wikimedia-l -- *Jon Davies - Chief Executive Wikimedia UK*. Mobile (0044) 7803 505 169 tweet @jonatreesdavies Wikimedia UK is a Company Limited by Guarantee registered in England and Wales, Registered No. 6741827. Registered Charity No.1144513. Registered Office 4th Floor, Development House, 56-64 Leonard Street, London EC2A 4LT. United Kingdom. Wikimedia UK is the UK chapter of a global Wikimedia movement. The Wikimedia projects are run by the Wikimedia Foundation (who operate Wikipedia, amongst other projects). Telephone (0044) 207 065 0990. Visit http://www.wikimedia.org.uk/ and @wikimediauk ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
