Re: [Wikitech-l] Community vs. centralized development
On 9/11/10 2:48 PM, Jamie Morken wrote: Doing the same on my log of the secret channel gives 100903 00:03:40, meaning it has roughly the same traffic level as #wikimedia-tech over that period. Anyone who hangs out there can tell you that almost nothing there is secret. I can't speak for private-l, because I'm not on it. Which channel are you talking about? Regarding private-l, my understanding is that that list was originally set up to deal with real-life wikistalking issues, which obviously requires privacy to discuss. Please correct me if that is incorrect. Ryan Kaldari ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 13 September 2010 21:14, emijrp emi...@gmail.com wrote: I think that Jamie has started an important topic. I don't think that WMF is going to usurp Wikipedia and the sister projects now or in the future, but it is statistically possible. If we want to protect us, the human knowledge and our work of this hypothetical scenario, we need complete full dumps frequently. But this scenario is a malicious one, and I think that there are many more dangerous posibilities, and unfortunately, they are common. For example, small or massive lost of data due to natural disasters, crackers attacks, stolen passwords, hardware and software bugs, sudden crazy sysops, and _human errors_. Is WMF ready for that? Shit happening is by far more likely than malice. Denise considers job #1 the elimination of existing single points of failure, so that's something. And she knows her stuff. And has the proper sysadminly horror at the notion of systems susceptible to such. - d. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Also, I think that we need to start mirroring Wiki[mp]edia dumps to other servers around the globe, as the common GNU/Linux ISOs mirrors do. Also, Library of Congress said some time ago that they are going to save a copy of all the tweets sent to Twitter.[6] When are they going to save a copy of Wiki[mp]edia? I hope we have learnt a bit since Library of Alexandria was destroyed. They've actually just reached out to us to discuss archiving all of the Wikimedia projects :) Discussions are in their early stages but I'll happily update as I know more. --tomasz ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On Mon, Sep 13, 2010 at 1:44 PM, Ryan Kaldari rkald...@wikimedia.org wrote: Regarding private-l, my understanding is that that list was originally set up to deal with real-life wikistalking issues, which obviously requires privacy to discuss. Please correct me if that is incorrect. My understanding of private-l is that it's the Wikimedia shell/root sysadmin discussion list, it just has a very bad name. I could be wrong too though. :-) -- Casey Brown Cbrown1023 ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On Mon, Sep 13, 2010 at 6:24 PM, Casey Brown li...@caseybrown.org wrote: On Mon, Sep 13, 2010 at 1:44 PM, Ryan Kaldari rkald...@wikimedia.org wrote: Regarding private-l, my understanding is that that list was originally set up to deal with real-life wikistalking issues, which obviously requires privacy to discuss. Please correct me if that is incorrect. My understanding of private-l is that it's the Wikimedia shell/root sysadmin discussion list, it just has a very bad name. I could be wrong too though. :-) Yep. It's a sysadmin list. Name kinda sucks :) -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Hi Tomasz, That is great news, congrats! I am happy they are spending time to archive wikis, I also am praying that my post ends up in the correct thread. cheers, Jamie - Original Message - From: Tomasz Finc tf...@wikimedia.org Date: Monday, September 13, 2010 3:09 pm Subject: Re: [Wikitech-l] Community vs. centralized development To: Wikimedia developers wikitech-l@lists.wikimedia.org Cc: Wikimedia Foundation Mailing List foundatio...@lists.wikimedia.org Also, I think that we need to start mirroring Wiki[mp]edia dumps to other servers around the globe, as the common GNU/Linux ISOs mirrors do. Also, Library of Congress said some time ago that they are going to save a copy of all the tweets sent to Twitter.[6] When are they going to save a copy of Wiki[mp]edia? I hope we have learnt a bit since Library of Alexandria was destroyed. They've actually just reached out to us to discuss archiving all of the Wikimedia projects :) Discussions are in their early stages but I'll happily update as I know more. --tomasz ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 12.09.2010, 1:48 Jamie wrote: On 9/8/2010 10:18 AM, Aryeh Gregor wrote: Well, this is probably my last post on this subject for now. I think I've made my points. Those who don't get them yet probably will continue not to get them, and those who get them but disagree probably [...] This is unbearable. Jamie, could you post to this list with a mail client that does not break threads and reformat quoted text horribly? Thanks. -- Best regards, Max Semenik ([[User:MaxSem]]) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 9/8/2010 10:18 AM, Aryeh Gregor wrote: Well, this is probably my last post on this subject for now. I think I've made my points. Those who don't get them yet probably will continue not to get them, and those who get them but disagree probably will continue to disagree. It looks like nothing big is going to change right now, but I hope that when Danese gets up to this, we'll see real improvements and not just attempts to paper over the problem without properly understanding it. I'll just make a few further brief points to reiterate some things I said that seem to still be misunderstood: On Sun, Sep 5, 2010 at 10:27 PM, Tim Starlingtstarl...@wikimedia.org wrote: I don't think you really know that. It's hard to see how much work goes on behind closed doors when you only have a cursory involvement with the project. It's pretty easy to figure out that there aren't daily (or weekly or monthly) face-to-face meetings among developers who live scattered across the world. None of the open source projects I've been involved with fit the model you describe. For instance, Squid makes heavy use of face-to-face meetings, despite their geographically distributed development team. Just to be clear: face-to-face meetings are great, in moderation. I'm totally in favor of them. But having lots of conferences is not the same as working in an office together. I think that's a false dichotomy. It is. There's a spectrum of middle ground in between, but the endpoints are perfectly tenable as well. I think that, given Wikimedia's mission as well as practical concerns, moving MediaWiki development significantly further toward openness would be a good thing. I can say that despite being a nobody at Mozilla and having gotten only one (rather trivial) patch accepted, I feel like I'm taken more seriously by most of their paid developers than by most of ours. I'm sorry to hear that, and I'd like to know (off list) which paid developers are making you feel that way. It would be unfair to name anyone, in public or in private. If I've had negative experiences with some paid developers, that should really count in their favor, because it means I have had *some* experience interacting with them, period. If we exclude paid developers who were preexisting community members: * I can think of two who I see with any regularity in #mediawiki. * I can think of maybe three who I've had more than one conversation with on IRC ever. * I don't think I've ever seen a wikitech-l post from the majority of them. I can't think why most of them should even know who I am, except now maybe some disgruntled volunteer who's making trouble for them. Why would I *expect* them to respect me? On Tue, Sep 7, 2010 at 8:29 PM, Ryan Kaldarirkald...@wikimedia.org wrote: First of all, all this talk of secret listservs and IRC channels is malarkey. Yes, there are private listservs and IRC channels. All of them are private for very specific and well-established reasons. Most of them are only used in very specific circumstances (for example if there was a security breach that needed to be discussed privately) and tend to be very low traffic. They are not the places where important decisions are made. 1) Either paid developers are coordinating someplace where volunteers don't see it, or they're not coordinating at all. The latter is implausible, so it's the former. It makes no difference if it's face-to-face meetings, teleconferences, IRC, or mailing lists, or even a technically public place that volunteers don't know about -- it's hidden. 2) The secret IRC channel is not low-traffic. The 1000th line before now in #wikimedia-tech (excluding parts/joins/etc., also excluding /me for simplicity) was about five days ago: $ grep -v '[^ ]* [^ ]* \*' FreeNode-#wikimedia-tech.log | tail -n 1000 | head -n 1 100903 16:08:55jps and if you are only doing those in groups of 10, you need to multiply by at least 3 Doing the same on my log of the secret channel gives 100903 00:03:40, meaning it has roughly the same traffic level as #wikimedia-tech over that period. Anyone who hangs out there can tell you that almost nothing there is secret. I can't speak for private-l, because I'm not on it. Secondly, the idea that developers here in the office don't interact with the community is absurd. The developers here interact with the community constantly. If the goal is to attract volunteers and make them feel part of the community, it doesn't matter whether the paid people think they're doing a good enough job. It matters whether the volunteers think it. I'm pretty sure it's clear by now that practically none of us do. As I said, anyone interested in fixing the problem would do well to start by surveying volunteers rather than looking at the issue from their own perspective, and Danese told me she does plan to do that -- so
Re: [Wikitech-l] Community vs. centralized development
On Thu, Sep 9, 2010 at 1:58 PM, Roan Kattouw roan.katt...@gmail.com wrote: 2010/9/9 Neil Kandalgaonkar ne...@wikimedia.org: I need an open source irony detector. Hint: it starts with this code: if ( $name === 'domas' ) return true; fixme: needs curly braces. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
2010/9/8 David Gerard dger...@gmail.com: This is something that's been a problem for years now. I do not think there is any sort of deliberate intent. However, keeping the data close is a way to proprietise a wiki even if it's free content, so making it easy to fork is an important attitude to maintain. I realise this is difficult when the devs have to work as hard as possible just to keep everything from falling over ... That's right, there is no deliberate intent and it's really a lack of people on the ops side (dumps are an ops thing, not a dev thing, and devs generally can't do much to help here). WMF is also not ignoring requests to provide image dumps, it just hasn't gotten around to setting them up yet; presumably, this is because text dumps aren't running smoothly yet (I'd appreciate a reply from Ariel Glenn to get the facts here, but since Ariel is out of the country I may or may not get my wish). It's true that the dumps situation is still a problem, but you (OP) should assume some good faith here rather than accusing the WMF of ignoring you, not earning the community's trust or even trying to usurp Wikipedia. You're right, you are being paranoid. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
2010/9/9 Jamie Morken jmor...@shaw.ca: Hi, I created a yahoo group for people interested in continuing the discussion on Community vs. centralized development as well as up to date wiki backups. Please join if you want to help to keep the Wikimedia foundation part of the community or just like chatting about it! Here is the group link: I see absolutely no reason to move this discussion to a separate place. Fragmentation like this is one of the things that's being criticized in this thread, and I believe this issue is important enough to involve the entire community rather than requiring people to subscribe to Yet Another Mailing List. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Στις 09-09-2010, ημέρα Πεμ, και ώρα 20:08 +0200, ο/η Roan Kattouw έγραψε: 2010/9/8 David Gerard dger...@gmail.com: This is something that's been a problem for years now. I do not think there is any sort of deliberate intent. However, keeping the data close is a way to proprietise a wiki even if it's free content, so making it easy to fork is an important attitude to maintain. I realise this is difficult when the devs have to work as hard as possible just to keep everything from falling over ... That's right, there is no deliberate intent and it's really a lack of people on the ops side (dumps are an ops thing, not a dev thing, and devs generally can't do much to help here). WMF is also not ignoring requests to provide image dumps, it just hasn't gotten around to setting them up yet; presumably, this is because text dumps aren't running smoothly yet (I'd appreciate a reply from Ariel Glenn to get the facts here, but since Ariel is out of the country I may or may not get my wish). It's true that the dumps situation is still a problem, but you (OP) should assume some good faith here rather than accusing the WMF of ignoring you, not earning the community's trust or even trying to usurp Wikipedia. You're right, you are being paranoid. I am not thinking about image dumps at all. I am concentrating on the regular XML dumps which have been in sorry shape for various reasons ever since I started as a volunteer in the community adding content. (note that I am not laying blame about the sorry state, that's not the point). For the rest of September I'll be fooling with these parallel runs until I get something that seems to perform well. For the next 5-6 days I'm out of action on them but after that it's back to the grind on them. Today, though I should hae been working on something else, I spent crunching some numbers and trying to figure out what more optimal chunk sizes ought to be. Since earlier articles by far have the bulk of the revisions it turns out I need to write some code to implement that. Anyways, either I'm (mostly) hard at work on this problem or I'm secretly plotting to run off with all the old copies of wikipedia to Bermuda and retire :-P Off of the dumps page on wikitech http://wikitech.wikimedia.org/view/Dumps there's a link to a page where I'm starting to keep updates, now that there is an actual run going. I may shoot this run and restart this piece in a few days, but what the heck, at least there's some information there. Also there's a link to a wish list for the XML dumps; if the image dumps aren't listed there please add them. I'm not going to try to think about how feasible or not they might be right now though, brain too full. Happy trails, Ariel ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
2010/9/9 Neil Kandalgaonkar ne...@wikimedia.org: I need an open source irony detector. Hint: it starts with this code: if ( $name === 'domas' ) return true; Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 2010-09-07, Ryan Kaldari wrote: I am both a long-time community member and a new WMF paid developer (in the SF office) so I think I'm in a unique position to clear up some misconceptions. First of all, all this talk of secret listservs and IRC channels is malarkey. Yes, there are private listservs and IRC channels. All of them are private for very specific and well-established reasons. Most of them are only used in very specific circumstances (for example if there was a security breach that needed to be discussed privately) and tend to be very low traffic. They are not the places where important decisions are made. This has been repeated several times in this thread. But from a volunteers perspective it is difficult to imagine where discussion between staff members is going on then. Until after this topic started I have seen very little discussion of staff projects on this mailing list, no discussion on the IRC channel, and very little on the MediaWiki wiki. These are three of the main enshrined development channels, and up until this discussion of staff activity on them was minimal. I find it difficult to believe that this is all the discussion that is going on, or is everything else in face to face meetings - if so where are the minutes and notes for these, because MediaWiki.org seems the obvious place to put them? And furthermore where is all the project documentation that you say has been produced? Sorry, but from the point of view of a volunteer the only logical reason there is/was no activity in any of these development channels is that there must be various private ones. Secondly, the idea that developers here in the office don't interact with the community is absurd. The developers here interact with the community constantly. We discuss community opinion and ideas, we talk with community members all day long on IRC, listservs, and on-wiki. I'm amazed that the developers here ever get anything done considering how much time they spend documenting what they are working on and interacting with the community about it. The problem is they can't interact with everyone everywhere: Code Review, IRC, listservs, the Tech Blog, meta, Signpost, etc. So no matter what, someone is going to feel like they are out of the loop. This may well be true for the community in general, but this discussion is specifically about the volunteer developer community, which is clearly being left out of the loop in a large respect - otherwise this discussion would not exist. They are arguably the most important members of the community from a technology perspective as volunteer developers have the time and initiative to actively improve MediaWiki, and development cannot survive on ideas and feedback alone. By readily involving the dev. community the WMF creates an environment where there is the potential (not everything will recieve a community response, but the potential for it then exists) for more ideas are implemented cost-free, and paid development time can be focused more on what it needs to be to maximise gain for all stakeholders. The WMF is extremely interested in new developers interacting with the community, indeed they try to hire developers from within the community when possible. The notion that the foundation is hiring corporate drones who are only going to listen to their task masters is completely unfounded. Yes, there have been situations where the foundation has been given grant money for very specifically defined projects and those projects have been implemented without adequate community involvement. Everyone (including the foundation) knows that that's not how we want to do development in the future. As has been discussed throughout the past year, the foundation wants to move away from accepting any money with strings attached and away from relying on grants in general. Hopefully, if this year's fundraiser goes well, we won't have to worry about these issues in the future. There is nothing wrong with recieiving technology grants for specific projects, and with the correct transparency and integration I think they can be a good thing as they are likely to improve areas that were previously ignored and bring new developers, and their insight, on to the paid staff. Transparency and integration were the main things the usability initiative lacked, but on their whole the contribution to MediaWiki and the WMF projects is undoubtedly very large. Robert ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
2010/9/7 Robert Leverington rob...@rhl.me.uk: if so where are the minutes and notes for these, because MediaWiki.org seems the obvious place to put them? Indeed, there are lots of face-to-face meetings / teleconferences where minutes are currently captured on EtherPad, but I don't think these are routinely visibly shared. I think it would be great, wherever possible, to post these to MediaWiki.org in standardized places so that folks can follow what's happening (and express interest in participating). And while we've stopped using usabil...@wm, there's still quite a bit of e-mail traffic that could be usefully directed toward wikitech-l or to lists that are, at minimum, publicly archived and have clear processes for joining and leaving. You and Ryan both right -- there's still too much of an in-group/out-group communication pattern, and too little explicit invitation of and collaboration with volunteers. As I hope recent communications demonstrate, that's slowly shifting, but it will take some time. And yes, it takes threads like this one. But, Ryan is correct that there's no pattern of deliberate secrecy here either, and if you developed an open source transparency/collaboration scorecard for WMF, you'd probably get a mixed result today. We're doing well in some areas like general corporate transparency [1], or making sure that all code goes into a public VCS, or granting commit access liberally, or being routinely and openly communicative in countless public spaces and at all levels. But there's no reason we shouldn't be best in every category. ;-) And that ultimately means seeing _everything_ as a shared responsibility. Everyone who works for Wikimedia, as a volunteer or staff member, is passionate about what we're doing, or they won't be here for long. We're here to build wonderfully useful information resources and the open source technologies to sustain and grow them -- and to be excellent at it. If we maintain awareness of the forcing functions that influence how people communicate, then I think that's eminently achievable. This includes, for example: - felt deadline pressure in the original usability project - problems with signal/noise ratio on existing lists / channels - in-group bias created by high proximity / peer relationships among WMF staff This isn't a game of assigning responsibilities or blame. Rather, it's about us collectively breaking out of the in-group/out-group pattern, creating constructive and healthy public spaces of communication and collaboration, remaining flexible about deadlines and targets where possible, reminding ourselves to be inclusive and open in how we work, etc. I'm confident that we can do it. :-) Erik [1] It's a fun exercise to research other organizations, non-profits and for-profits. I do so routinely as part of my work, and there are very, very few similarly funded organizations that even come close to the level of disclosure that WMF is currently at. -- Erik Möller Deputy Director, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Well, this is probably my last post on this subject for now. I think I've made my points. Those who don't get them yet probably will continue not to get them, and those who get them but disagree probably will continue to disagree. It looks like nothing big is going to change right now, but I hope that when Danese gets up to this, we'll see real improvements and not just attempts to paper over the problem without properly understanding it. I'll just make a few further brief points to reiterate some things I said that seem to still be misunderstood: On Sun, Sep 5, 2010 at 10:27 PM, Tim Starling tstarl...@wikimedia.org wrote: I don't think you really know that. It's hard to see how much work goes on behind closed doors when you only have a cursory involvement with the project. It's pretty easy to figure out that there aren't daily (or weekly or monthly) face-to-face meetings among developers who live scattered across the world. None of the open source projects I've been involved with fit the model you describe. For instance, Squid makes heavy use of face-to-face meetings, despite their geographically distributed development team. Just to be clear: face-to-face meetings are great, in moderation. I'm totally in favor of them. But having lots of conferences is not the same as working in an office together. I think that's a false dichotomy. It is. There's a spectrum of middle ground in between, but the endpoints are perfectly tenable as well. I think that, given Wikimedia's mission as well as practical concerns, moving MediaWiki development significantly further toward openness would be a good thing. I can say that despite being a nobody at Mozilla and having gotten only one (rather trivial) patch accepted, I feel like I'm taken more seriously by most of their paid developers than by most of ours. I'm sorry to hear that, and I'd like to know (off list) which paid developers are making you feel that way. It would be unfair to name anyone, in public or in private. If I've had negative experiences with some paid developers, that should really count in their favor, because it means I have had *some* experience interacting with them, period. If we exclude paid developers who were preexisting community members: * I can think of two who I see with any regularity in #mediawiki. * I can think of maybe three who I've had more than one conversation with on IRC ever. * I don't think I've ever seen a wikitech-l post from the majority of them. I can't think why most of them should even know who I am, except now maybe some disgruntled volunteer who's making trouble for them. Why would I *expect* them to respect me? On Tue, Sep 7, 2010 at 8:29 PM, Ryan Kaldari rkald...@wikimedia.org wrote: First of all, all this talk of secret listservs and IRC channels is malarkey. Yes, there are private listservs and IRC channels. All of them are private for very specific and well-established reasons. Most of them are only used in very specific circumstances (for example if there was a security breach that needed to be discussed privately) and tend to be very low traffic. They are not the places where important decisions are made. 1) Either paid developers are coordinating someplace where volunteers don't see it, or they're not coordinating at all. The latter is implausible, so it's the former. It makes no difference if it's face-to-face meetings, teleconferences, IRC, or mailing lists, or even a technically public place that volunteers don't know about -- it's hidden. 2) The secret IRC channel is not low-traffic. The 1000th line before now in #wikimedia-tech (excluding parts/joins/etc., also excluding /me for simplicity) was about five days ago: $ grep -v '[^ ]* [^ ]* \*' FreeNode-#wikimedia-tech.log | tail -n 1000 | head -n 1 100903 16:08:55 jps and if you are only doing those in groups of 10, you need to multiply by at least 3 Doing the same on my log of the secret channel gives 100903 00:03:40, meaning it has roughly the same traffic level as #wikimedia-tech over that period. Anyone who hangs out there can tell you that almost nothing there is secret. I can't speak for private-l, because I'm not on it. Secondly, the idea that developers here in the office don't interact with the community is absurd. The developers here interact with the community constantly. If the goal is to attract volunteers and make them feel part of the community, it doesn't matter whether the paid people think they're doing a good enough job. It matters whether the volunteers think it. I'm pretty sure it's clear by now that practically none of us do. As I said, anyone interested in fixing the problem would do well to start by surveying volunteers rather than looking at the issue from their own perspective, and Danese told me she does plan to do that -- so I'll wait. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org
[Wikitech-l] Community vs. centralized development
Hi, I was involved in an open source project that was usurped by one of the main developers for the sole reason of making money, and that project continues now to take advantage of the community to increase the profit of that developer. I never would have thought such a thing was possible until I saw that happen. If that developer wasn't acting greedy, there would now be open source hardware for radio transceivers of all types, but instead there is only open source software for radio of all types. I find it a shame, and when I was working on that project I could *feel* it being usurped! I unfortunately may be paranoid as I feel the same thing here with the wikimedia foundation usurping wikipedia. If you don't believe me, just consider that it is a very gradual process, like getting people used to not being able to download image dumps anymore, and ignoring ALL requests to restore this functionality. Also failing to provide full history backups of the flagship wiki. These two facts allow the wikimedia foundation to maintain the control of intellectual property that wasn't created by the people. If you want the wikimedia foundation to respect you as volunteers, you will have to DEMAND respect by making sure that they never usurp the project. I think the best way to do this is to make sure we can all download up to date full history with images wikipedia's so a fork at any time is possible. Sure it may be paranoid, but trust me it is worth it to be paranoid regarding a project as important as wikipedia. I have been in situations like this before, I wish I had acted before even if I was wrong! I wouldn't even be speaking now except for reading the heart-felt words of volunteers in this thread that are unhappy with how the wikimedia foundation is running. We need to organize to get wikimedia foundation to release images tarballs, they are only ignoring multiple requests to do so, so far. cheers, Jamie ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 8 September 2010 22:15, Jamie Morken jmor...@shaw.ca wrote: I was involved in an open source project that was usurped by one of the main developers for the sole reason of making money, and that project continues now to take advantage of the community to increase the profit of that developer. I never would have thought such a thing was possible until I saw that happen. If that developer wasn't acting greedy, there would now be open source hardware for radio transceivers of all types, but instead there is only open source software for radio of all types. I find it a shame, and when I was working on that project I could *feel* it being usurped! I unfortunately may be paranoid as I feel the same thing here with the wikimedia foundation usurping wikipedia. If you don't believe me, just consider that it is a very gradual process, like getting people used to not being able to download image dumps anymore, and ignoring ALL requests to restore this functionality. Also failing to provide full history backups of the flagship wiki. These two facts allow the wikimedia foundation to maintain the control of intellectual property that wasn't created by the people. This is something that's been a problem for years now. I do not think there is any sort of deliberate intent. However, keeping the data close is a way to proprietise a wiki even if it's free content, so making it easy to fork is an important attitude to maintain. I realise this is difficult when the devs have to work as hard as possible just to keep everything from falling over ... - d. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 8 September 2010 23:00, Ryan Kaldari rkald...@wikimedia.org wrote: I had no idea that usurping an open source project was as easy as not providing full history back-ups and image dumps. And here I was trying to replace all the board members with proxies from Wikia! What a waste of time ;) Neglecting to make it possible to get the data out is an effective way to proprietise a wiki. Making the backups work is important. - d. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 9/8/2010 1:45 PM, Ryan Kaldari wrote: On 9/7/10 11:23 PM, Robert Leverington wrote: This may well be true for the community in general, but this discussion is specifically about the volunteer developer community, which is clearly being left out of the loop in a large respect - otherwise this discussion would not exist. I agree that the volunteer dev community is not as in-the-loop as paid staff, but it's not because the Foundation is trying to keep them out of the loop. Communicating with the dev community, the broader community, fellow staff, and keeping up with an aggressive development schedule is not an easy task! In 90% of of cases, there should be little difference between communicating with staff and communicating with the developer community. Obviously there will be some non-development related staff communication, but there really shouldn't be a difference between communicating with staff and the community. An intentional or incidental separation between staff and community is really the underlying problem. It doesn't take a conspiracy to not satisfactorily fulfill all of those requirements. Obviously, we can stand some improvement, which is why the Foundation is planning on specifically hiring someone to act as a bugmeister and development coordinator in the very near future. What we need are helpful (and realistic) suggestions for how to improve communication, not misguided badgering about secret channels. Getting rid of excessive amounts of communication channels isn't a useful suggestion? There are 2 main public mailing lists and 3 main public IRC channels. Then there's about a dozen other minor public mailing lists. How many additional ones are needed? Why does the non-staff community need to be excluded from them? A few years ago, there were few staff developers and many worked remotely. There were few of the communication issues we have now. More recently, the WMF hired more staff developers and began concentrating them in the main office and some private channels/lists sprung up to support them. Now we have communication problems. Its possible its just a coincidence, but it certainly doesn't look like one. At the very least, decreasing excessive bureaucracy (for lack of a better term) is something that should be considered as a matter of course. -- Alex (wikipedia:en:User:Mr.Z-man) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Community vs. centralized development
Hi, I created a yahoo group for people interested in continuing the discussion on Community vs. centralized development as well as up to date wiki backups. Please join if you want to help to keep the Wikimedia foundation part of the community or just like chatting about it! Here is the group link: http://tech.groups.yahoo.com/group/wikishare/ In this group topics are discussed related to the Wikimedia Foundation's relationship to the community of volunteer developers and users, as well as distribution of wiki backups and image backups. Two main goals of the group are to ensure that Wikimedia foundation development is community centered and also to have up to date full history Wikimedia Foundation wiki's and wiki images freely available for download. cheers, Jamie ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Hi! I created a yahoo group Why not Facebook Page?!!? Domas ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On Wed, Sep 8, 2010 at 8:11 PM, Domas Mituzas midom.li...@gmail.com wrote: Hi! I created a yahoo group Why not Facebook Page?!!? Domas Or a new mailing list! Just like wikitext-l, a specialized list :) -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 9 September 2010 01:25, Chad innocentkil...@gmail.com wrote: Just like wikitext-l, a specialized list :) Careful there, wikitext-l may have come up with something slightly useful. You never know what might breed in such a swamp. - d. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 9/8/10 2:26 PM, Domas Mituzas wrote: Hi! ... there would now be open source hardware Do you need open source Enter key? Open source hardware isn't an inherent absurdity... it usually means that the hardware designs or other precursors (such as code that can generate circuit designs) are freely available. http://en.wikipedia.org/wiki/Open-source_hardware -- Neil Kandalgaonkar ne...@wikimedia.org ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On Wed, Sep 8, 2010 at 8:28 PM, David Gerard dger...@gmail.com wrote: On 9 September 2010 01:25, Chad innocentkil...@gmail.com wrote: Just like wikitext-l, a specialized list :) Careful there, wikitext-l may have come up with something slightly useful. You never know what might breed in such a swamp. When it's even remotely usable in MediaWiki, then maybe I'll take notice. I've come to learn that parser rewrites are largely vaporware, so pardon me if I'm a little skeptical. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On Wed, Sep 8, 2010 at 8:32 PM, Neil Kandalgaonkar ne...@wikimedia.org wrote: On 9/8/10 2:26 PM, Domas Mituzas wrote: Hi! ... there would now be open source hardware Do you need open source Enter key? Open source hardware isn't an inherent absurdity... it usually means that the hardware designs or other precursors (such as code that can generate circuit designs) are freely available. http://en.wikipedia.org/wiki/Open-source_hardware -- Neil Kandalgaonkar ne...@wikimedia.org I think the point was not about hardware, but the OPs inability to include a single linebreak in the e-mail. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 9/8/10 5:35 PM, Chad wrote: I think the point was not about hardware, but the OPs inability to include a single linebreak in the e-mail. I need an open source irony detector. -- Neil Kandalgaonkar ne...@wikimedia.org ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Why decentralized discussions even more? And is there a reason you always seem to spilt your replies to the thread into new treads/topics instead of just replying to the original one? ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
I made seven suggestions. Only one was about actually dissolving the office, and I acknowledged that it might be extreme. What about the others? Why does the private IRC chat need to exist, for example? Why can't we have clear official statements that everything should be as public as possible and that volunteer developers should be treated as peers? Why can't teleconferences be replaced by public-logged IRC chats? Are these also too extreme? Aryeh, I think many volunteer and more casual developers share your concern. I in principle agree with your proposals, although of course no-one can be forced to abandon private communication, and private means of communication are always going to exist. I have raised similar concerns about volunteers not knowing what is going on by not being on secret channels of communication, in person, to a couple of members of staff during last two wikimanias and developer meetings, and I had the feeling they agreed with me. The community needs to be nurtured, and I think all new employees of the WMF need to be aware of it, and at first interview informed that they will *need* to interact with the community and with volunteer developers. I think many programmers who have worked in programming companies are too used to just talking and listening to their team leaders and no-one else. It should be made clear that this is not how things (should) work in WMF, and this should be an official position from however is hiring. Or maybe it is utopia and we do need to have a more stereotypical corporate setup, but I really hope not because wikipedia is fueled by enthusiasm. Finally, speaking as a volunteer who is not on any secret IRC channels, mailing lists or payrolls I want to share my experience in WMF software development. Back in 2006 I wanted to make search better, and if then it wasn't for Tim Starling to give me shell access and a couple of test servers to play with, I think we would not have the new search, or at least not developed by me. An act of kindness, but also a sign that a core developer cares about what a relatively unknown volunteer is trying to do and achieve. As for code review, I know the foundation knows how important this task is, and that it is no 9-5 job, but one that requires an extremely dedicated person with a great knowledge of the mediawiki codebase and ability to comment on virtually every programming issue. The foundation better pay this person well and not just hope for someone to fill in this place in their spare time. Cheers, Robert ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 9/7/10 4:15 PM, Robert Stojnic wrote: The community needs to be nurtured, and I think all new employees of the WMF need to be aware of it, and at first interview informed that they will *need* to interact with the community and with volunteer developers. Just FYI, this was the most-stressed topic during my interviews at the WMF. So I don't think it's a matter of the WMF not being aware of the topic. I think many programmers who have worked in programming companies are too used to just talking and listening to their team leaders and no-one else. This is true, but we are trying not to hire this sort of person. Off the top of my head I would guess at least 75% of the developers here had significant open source experience before being hired, many with Wikimedia projects. -- Neil Kandalgaonkar ne...@wikimedia.org ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
I am both a long-time community member and a new WMF paid developer (in the SF office) so I think I'm in a unique position to clear up some misconceptions. First of all, all this talk of secret listservs and IRC channels is malarkey. Yes, there are private listservs and IRC channels. All of them are private for very specific and well-established reasons. Most of them are only used in very specific circumstances (for example if there was a security breach that needed to be discussed privately) and tend to be very low traffic. They are not the places where important decisions are made. Secondly, the idea that developers here in the office don't interact with the community is absurd. The developers here interact with the community constantly. We discuss community opinion and ideas, we talk with community members all day long on IRC, listservs, and on-wiki. I'm amazed that the developers here ever get anything done considering how much time they spend documenting what they are working on and interacting with the community about it. The problem is they can't interact with everyone everywhere: Code Review, IRC, listservs, the Tech Blog, meta, Signpost, etc. So no matter what, someone is going to feel like they are out of the loop. The WMF is extremely interested in new developers interacting with the community, indeed they try to hire developers from within the community when possible. The notion that the foundation is hiring corporate drones who are only going to listen to their task masters is completely unfounded. Yes, there have been situations where the foundation has been given grant money for very specifically defined projects and those projects have been implemented without adequate community involvement. Everyone (including the foundation) knows that that's not how we want to do development in the future. As has been discussed throughout the past year, the foundation wants to move away from accepting any money with strings attached and away from relying on grants in general. Hopefully, if this year's fundraiser goes well, we won't have to worry about these issues in the future. Ryan Kaldari On 9/7/10 4:15 PM, Robert Stojnic wrote: I made seven suggestions. Only one was about actually dissolving the office, and I acknowledged that it might be extreme. What about the others? Why does the private IRC chat need to exist, for example? Why can't we have clear official statements that everything should be as public as possible and that volunteer developers should be treated as peers? Why can't teleconferences be replaced by public-logged IRC chats? Are these also too extreme? Aryeh, I think many volunteer and more casual developers share your concern. I in principle agree with your proposals, although of course no-one can be forced to abandon private communication, and private means of communication are always going to exist. I have raised similar concerns about volunteers not knowing what is going on by not being on secret channels of communication, in person, to a couple of members of staff during last two wikimanias and developer meetings, and I had the feeling they agreed with me. The community needs to be nurtured, and I think all new employees of the WMF need to be aware of it, and at first interview informed that they will *need* to interact with the community and with volunteer developers. I think many programmers who have worked in programming companies are too used to just talking and listening to their team leaders and no-one else. It should be made clear that this is not how things (should) work in WMF, and this should be an official position from however is hiring. Or maybe it is utopia and we do need to have a more stereotypical corporate setup, but I really hope not because wikipedia is fueled by enthusiasm. Finally, speaking as a volunteer who is not on any secret IRC channels, mailing lists or payrolls I want to share my experience in WMF software development. Back in 2006 I wanted to make search better, and if then it wasn't for Tim Starling to give me shell access and a couple of test servers to play with, I think we would not have the new search, or at least not developed by me. An act of kindness, but also a sign that a core developer cares about what a relatively unknown volunteer is trying to do and achieve. As for code review, I know the foundation knows how important this task is, and that it is no 9-5 job, but one that requires an extremely dedicated person with a great knowledge of the mediawiki codebase and ability to comment on virtually every programming issue. The foundation better pay this person well and not just hope for someone to fill in this place in their spare time. Cheers, Robert ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
2010/9/6 Ryan Lane rlan...@gmail.com: The chat in those channels isn't anything crucial or even related to development. If the channels go away, the talk that occurs there will simply move to private chats on IM. This being on IRC just makes it slightly easier to broadcast things to staff members. Do you really care to see chatter in the mediawiki channel like: Free brownies in the break room., Water machine is getting fixed., Phone system is getting upgraded.? Should non-tech related employees be forced to deal with all the bot traffic, dev talk, and support talk of #mediawiki? Maybe it doesn't need to be private, but dealing with trolls gets old pretty fast. There is also productive, work-related chat going on there, mostly someone working from home (a lot of of our staff have a habit of working from home one day a week) trying to grab someone else. Conversations where all participants work in tech frequently happen in other channels instead. 5. Mute those who troll/are unwilling to stick to the agenda (they will still be invited to listen) I'm not completely sure that this is easy to do with our phone system, but maybe Skype or WebEx allow it. 6. Post minutes as quickly as possible after the meeting is over (hopefully within a couple of hours) This should be trivial, as such meeting notes are usually (almost always for meetings I'm in) written collaboratively and on the fly in Etherpad, and publicly readable and editable to everyone who knows the URL (probably best to move stuff to the wikis anyway, though). We often posted the Etherpad link to our usability progress meeting notes in #wikipedia_usability , although to be fair that wasn't done with transparency or non-staff involvement in mind. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
2010/9/6 Jamie Morken jmor...@shaw.ca: So it sounds like respect is also centralized in the wikimedia foundation, please include me in your email to your underlings Tim, as I would also like to have respect, maybe it will mean my request for image dumps is taken seriously! It would be nice if respect was earned, but it sounds like Tim can create/enforce respect in a more simple way, sounds like a good leadership path to me.. NOT! :) Has it occurred to you that maybe the one person in charge of both dumps and media storage is a busy person who also happens to be on vacation right now? Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Community vs. centralized development
So it sounds like respect is also centralized in the wikimedia foundation, please include me in your email to your underlings Tim, as I would also like to have respect, maybe it will mean my request for image dumps is taken seriously!? It would be nice if respect was earned, but it sounds like Tim can create/enforce respect in a more simple way, sounds like a good leadership path to me.. NOT! :) Has it occurred to you that maybe the one person in charge of both dumps and media storage is a busy person who also happens to be on vacation right now? Hi, No, but you bring up a good point, having a single person in charge of both of these departments is not working very well, since the department of image dumps hasn't actually released any publicly available image dumps in 4+ years, and the department of xml dumps has only released a single successful* full history dump of enwiki in 3+ years. *only partially successful as some of the revision history is missing from the dump cheers, Jamie Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Jamie Morken wrote: Hi, What do you mean by opening? enwiki pages-meta-history is hard due to its size, not because Ariel or Tomasz being more stupid than any volunteer. I trust them to do it at least as well as a volunteer would. Of course, if you can perform better I'm all for giving you a shell to fix it, and the scripts are there for improvements as well. I wasn't aware that the dump scripts were publicly available, where can they be downloaded from or are they part of mediawiki? It is in http://svn.wikimedia.org/viewvc/mediawiki/trunk/backup/ although the files look a bit old, so perhaps there are some uncommitted changes? /me looks for offenders What do you need exactly about the images? Which image dumps do you want? Do you have enough terabytes to store them? Dumps/Access has been given by request in the past to that data. If it's not there it's because: a) Those dumps would take a lot of space. I don't think that is a valid reason, thumbnail dumps of all the images from enwiki would probably be a smaller file than the current enwiki pages-meta-history bz2 file. We have thumbs on lots of sizes. Which size do you want the thumbs? It's easy to tar all the images used on a wiki, since that's tracked in the database, but not at all knowing which exact size was each of them used. enwiki has a total of 858979 local files which sum 229 GB (and there's still commmons). 2357967 unique images (37050694 uses) are in their articles. Assuming 20Kb per image thumb (is that a good value?), that's 48 Gb, more than the 31.9 GB of the (really compressed) pages-meta-history.xml.7z but we would need to agree. They would tie at 14 Kb. Even if all thumbs were unrealistically small, 1Kb each, they would still be several GB. b) Nobody feels particulary interested in them. I disagree, there has been a lot of interest in having image dumps available for download. There was a discussion on this recently on the xmldatadumps list, that basically concluded that subsets of images (ie. enwiki thumbnails) would be useful. I am unable to find it, although a thread like that somewhere rings a bell to me. There are wiki pages dedicated to this topic of how to download images, this is because there are no image dumps available. Is the wikimedia foundation interested to host image dumps again? If they are maybe we can start a discussion on how to make the script and what image dumps to start with. cheers, Jamie ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On Tue, Sep 7, 2010 at 9:50 AM, Platonides platoni...@gmail.com wrote: enwiki has a total of 858979 local files which sum 229 GB (and there's still commmons). 2357967 unique images (37050694 uses) are in their articles. Assuming 20Kb per image thumb (is that a good value?), that's 48 Gb, more than the 31.9 GB of the (really compressed) pages-meta-history.xml.7z but we would need to agree. They would tie at 14 Kb. Even if all thumbs were unrealistically small, 1Kb each, they would still be several GB. Comparing the size to pages-meta-history isn't all that fair since with the images they wont change much, so you only need to do the base copy then on the next run you just need to update/add the appropriate ones that have changed/been added or delete the ones that are gone. Also does that figure take into the fair use images which we wouldn't be able to dump? -Peachey ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On Fri, Sep 3, 2010 at 2:06 PM, OQ overlo...@gmail.com wrote: Have to say this is the first I've heard of this channel existing. Yay for communication. It was only created very recently. On Fri, Sep 3, 2010 at 2:14 PM, Chad innocentkil...@gmail.com wrote: As I've said elsewhere to people, this isn't an excuse for fracturing the discussion. Using a single channel for development *and* support has worked for *years* until the Usability Initiative decided it needed its own channel. The channel is not actually that busy if you don't count the bots. And for those of you who *really* don't like them, you can /ignore them. I don't consider people asking for help noise either, it's part of being engaged with the community. I don't see the need for a separate development channel at all. Me neither. We don't get that many support questions. On Fri, Sep 3, 2010 at 4:14 PM, Neil Kandalgaonkar ne...@wikimedia.org wrote: Aryeh: first off I want to thank you again for the constructive criticism. And thank you for the thoughtful and constructive response. I think you are not really appreciating that the WMF employees are also human beings. We share an office. We go out to lunch together every day. This is, as I said, one of the problems -- at least from the point of view of greater volunteer involvement. (That you share an office and go to lunch together, not that you're human beings. :) ) It's not necessarily prohibitive, in that you could certainly have totally healthy community development with lots of paid developers in one place, but it definitely lends itself to locking out volunteers. I don't see anyone deliberately hiding things. It's more the case that we don't have established procedures about how to be open, in a regular, repeatable fashion. We try really hard but the efforts are always competing with just trying to get things done. I don't know what you can call a private IRC chat where staff gets to hang out and non-staff does not, except deliberately hiding things. I can't speak for anything else because, well, it's hidden. I know it's hidden somewhere, because I don't see it, but I don't know where, or whether it's deliberate or not. And in part that openness has practical limits, for exactly the same reasons that a network card has limited bandwidth. There's no practical limit here. An approach where everyone communicates almost entirely by public e-mail is feasible. Any open-source project without a single well-funded controlling organization works that way by necessity -- the Linux kernel, Debian, etc. Some with such a controlling organization also work that way. When Wikipedia started, it wasn't a perfectly distributed effort either; they always had developers and people like Jimmy Wales in the same room. This certainly wasn't true in 2006, when I got commit access. There were two paid developers at the time, both of whom lived more than a thousand miles from Jimmy Wales. At that time there was no office at all, IIRC, but if there was, it was in Tampa -- Brion lived in San Francisco and Tim in Australia. It is my impression that the reason they are able to be so open and communicate well about roadmaps and so on is because they do have enough resources (centralized) to do such coordination and make sure to publish documents and do press releases and whatnot. If everyone was dispersed I don't think they'd be very successful at this task. Everyone *is* dispersed. For instance, the layout engine is owned by Robert O'Callahan (New Zealand), and its peers are Bernd Mielke (?), Simon Montagu (Israel), L. David Baron (Los Angeles area), Boris Zbarsky (Boston area). (Locations based on Googling, might not all be accurate.) Linked from the mozilla.com website: Most positions are based at our headquarters in Mountain View, California, but we also have offices in Tokyo, Paris, Toronto, Beijing and Auckland (with more to come!). Not near one of our offices? Mozilla thrives as an organization because of our diverse and distributed workforce, so remote employment is an option. http://hire.jobvite.com/CompanyJobs/Jobs.aspx?c=qpX9Vfwa As far as I can tell, very little happens at Mozilla face-to-face, compared to what's done by newsgroup/Bugzilla/IRC. I've subscribed to lots of Mozilla bugs and watched patches get submitted and reviewed, and I've never noticed much of anything that seemed to be hidden. Maybe because the patch author and reviewer usually live in separate parts of the world. Even if Mozilla weren't a perfect example, though, the Linux kernel is a case where virtually nothing important happens except by mailing list, which proves it's feasible in principle to run a large software project that way. I'm okay with discussing whether there might be efficiency or morale problems with not having most paid developers in a central office, but it's just not correct to suggest that it's impractical to develop software that way. It's entirely
[Wikitech-l] Community vs. centralized development
Hi, I think it would be a nice gesture if the wikimedia foundation decentralized some of the internal projects that have had little success over the last few years. Two that come to mind are the enwiki pages-meta-history file creation (1 successful dump in ~3 years), and apparently very little internal concern over this, and also the images/thumbnail wiki-specific dumps that have been neglected for several years. If these two projects are opened up to the community I think that would be great! cheers, Jamie ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Hi Aryeh, Thanks for bringing this topic up. It looks like its been a pretty productive conversation so far, so I hope I don't ruin it. ;-) Here's where I think you and I are on common ground. We seem to disagree about magnitude (e.g. more vs all or less vs none), but I think we can agree on direction: a. More peer-to-peer communication needs to occur on public channels. As many of you know, prior to starting at Wikimedia Foundation, I had been a (very) peripheral member of the development community (lurking on IRC, contributing a patch or two, developing extensions, attending Wikimania 2006 dev days). I knew I had a lot to learn about the development process, but I've been surprised about how little I did know, which is in part due to the fact that many conversations that I expected to be public and discoverable by search engine weren't. It makes it much easier to involve and vet newcomers if we can watch their participation in our peer-to-peer communications. b. Volunteers need the opportunity to participate as peers. If someone demonstrates competence, a track record for useful contribution, and an ability not to make the existing core contributors all stabby, they deserve a seat at the table. We can't make everyone a full development peer in the same way that we can't make everyone a sysop in their respective editing community, but we should push the boundaries the same way our editing communities do. c. Being on the staff at Wikimedia Foundation shouldn't automatically confer any special authority on MediaWiki trunk or even an automatic seat at the table when deciding what belongs there. Of course, many of the people at WMF are there because they've earned a special place in the community already, but getting hired shouldn't be a shortcut to that. Wikimedia Foundation staff dominates MediaWiki development as the primary financial contributor to its development, and by virtue of running many of the highest traffic MediaWiki websites. But we shouldn't have norms which keep other organizations and individuals from operating as peers in a more diverse developer ecosystem. Many of the greatest open source projects have organizational diversity in their contributor base (Linux, Apache, etc), and it's great to aspire to that. d. No one should be so sure to what degree Wikimedia Foundation employees need exclusive control of the code that runs on Wikimedia's production servers. There is a certain unique responsibility that Foundation employees have to keep the servers running, and to support the strategic goals as laid out by the community, the board, and our executive staff. However, that doesn't mean the correct answer is to become control freaks or lock out the volunteers that helped us get where we are, let alone shut the door to new volunteers. It just means that, unlike with point c above, we're in much more uncharted territory. There aren't many examples of open source projects for which the community is as involved in the version and configuration of the software running on the production cluster as this community is. The difference between c and d above is very important, because it seems to be where the core of the disagreements are about direction setting. If most non-Foundation developers that have weighed in on this thread are most interested in c, this would be a straightforward problem to solve. However, I suspect many people care about d every bit as much, and as a result, I think we're talking past each other a little bit. I'm not going to stake out a firm position on d; it's a complicated matter, and I think I can argue both sides of it. I think it's a fruitful area for discussion on foundation-l, since a lot of the tension has to do with non-technical staff and community members being more involved in the direction setting now that the Foundation budget actually allows for such a thing (and some of the direction-setting comes from the budget itself; e.g. directed grants with deliverables and schedules). Points a and b are points that I plan to push harder on, partly as a result of this thread. The monthly update that the EPM team put together [1] is a step in that direction. Merely broadcasting what we are doing isn't sufficient to actually achieve a peer-to-peer relationship, but the information is an important prerequisite. We should be able to draft the October version of the update on mediawiki.org.[2] You now should have a better idea of the program managers to pester to get involved in something we're working on. There are still plenty of internal meetings, but I think I might be able convert at least one or two of the ones I'm responsible for into public IRC-only in the near future. Inviting the general public to our telecons won't scale, but I think we should keep an open mind about inviting key non-WMF contributors to more of our conversations, and come up with more collaboration tools that scale better but also have many of the positive
Re: [Wikitech-l] Community vs. centralized development
On 04/09/10 03:39, Aryeh Gregor wrote: On Fri, Sep 3, 2010 at 1:20 AM, Tim Starling tstarl...@wikimedia.org wrote: Your recommendations seem insensitive and unrealistic. What works for you does not necessarily work for everyone. It works for many, many open-source projects. I don't think you really know that. It's hard to see how much work goes on behind closed doors when you only have a cursory involvement with the project. None of the open source projects I've been involved with fit the model you describe. For instance, Squid makes heavy use of face-to-face meetings, despite their geographically distributed development team. PHP has projects developed by individuals and small groups which are then reviewed on the mailing list. Does Wikimedia want to be among them, or does it want to be among the projects that are open-source but have a stagnant community because they don't involve volunteers enough and the code is tightly controlled by a sponsoring organization? There are countless projects of both types -- both strategies work, to a degree. Completely closed-source development works too. Wikimedia is free to pick whichever model best fits its needs and ideals. I think that's a false dichotomy. For a start, control and design are two different things, as the case of the collapsed interlanguage links demostrates. Just because an individual or small group designs something doesn't mean the community will accept it. Design by individuals or small groups, with open community review and consensus decision-making, is entirely consistent with a thriving community. I'll grant that it's not be ideal, and that an open design process should be encouraged where possible, but it's not a contradiction as you seem to imply. My goal as a developer is to support the community such as it is, not to browbeat shy or otherwise sensitive developers into either posting their comments in public or keeping their thoughts to themselves. And my goal as a community leader is to seek consensus on all decisions and to defuse conflict wherever possible by finding compromises. I don't think these two things are contradictory. I can say that despite being a nobody at Mozilla and having gotten only one (rather trivial) patch accepted, I feel like I'm taken more seriously by most of their paid developers than by most of ours. I'm sorry to hear that, and I'd like to know (off list) which paid developers are making you feel that way. I've made it pretty clear to Danese that I think you're one of our best volunteer developers, and that you should be given whatever you ask for, which, I think it's understood, includes respect. Maybe I can help to get that message to filter down to the rest of the organisation. I know more about what Mozilla is planning to do in their next release than what Wikimedia employees are planning. I've had lengthy discussions on occasion with core Mozilla developers, and sometimes I've gotten something changed because of it. I can say none of these things about any Wikimedia project that's dominated by paid employees. There are two projects which Wikimedia employees are working on for the next core release: the resource loader and the new installer. Both of them have been discussed on wikitech-l, and both have invited community involvement by way of design documents published on mediawiki.org. Both of them did their initial development in a branch, which anyone was free to review and contribute to. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Community vs. centralized development
Hi, I can say that despite being a nobody at Mozilla and having gotten only one (rather trivial) patch accepted, I feel like I'm taken more seriously by most of their paid developers than by most of ours. I'm sorry to hear that, and I'd like to know (off list) which paid developers are making you feel that way. I've made it pretty clear to Danese that I think you're one of our best volunteer developers, and that you should be given whatever you ask for, which, I think it's understood, includes respect. Maybe I can help to get that message to filter down to the rest of the organisation. So it sounds like respect is also centralized in the wikimedia foundation, please include me in your email to your underlings Tim, as I would also like to have respect, maybe it will mean my request for image dumps is taken seriously! It would be nice if respect was earned, but it sounds like Tim can create/enforce respect in a more simple way, sounds like a good leadership path to me.. NOT! :) cheers, Jamie ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Στις 03-09-2010, ημέρα Παρ, και ώρα 05:34 -0700, ο/η Conrad Irwin έγραψε: On 3 September 2010 00:51, Danese Cooper dcoo...@wikimedia.org wrote: 1. Eliminate single points of failure / bottlenecks 2. Reconfigure into teams designed to encourage faster (shorter duration) and more accurate projects / deployments 3. Develop programs to encourage / grow volunteers into paid staff...recognizing that as a non-profit WMF needs to plan for more frequent turnover in the Tech department to ensure that we can grow expertise across the dev community rather than concentrating it in a few core people. 4. Restore the balance between community-led and foundation-led efforts so WMF feels again like a partnership rather than concentric circles of permission / blame I really like item 2. When I was working on MediaWiki related stuff, the main problem I had was working out who to talk to. The answer, if you ask, seems always to be Tim — which is not very scalable (brilliant though Tim is). I'd hope that along with an organisation into more formal teams would come a structure where individuals are responsible (either permanently, or on rotation like the chromium) for subsets of MediaWiki/Wikimedia. I think of the who to talk to issue (which in part reduces to a who can officially sign off on your code so it may be deployed issue) as #1, bottlenecks. Over the past few years that I have been involved in various ways in the Wikimedia projects, I have seen people give up after waiting a couple of years for their code to make to the deployable stage. This in term impacts both #3 (getting/attracting volunteers who can later turn into paid staff) and #4 (dynamics between WMF and the community, to the extent that WMF staff are not considered community members. It sort of puts a new spin on the old If you want it done, write it yourself because we are too busy line. When writing it yourself (and being willing to revise it until it's considered good) isn't available as an option any more, an open source project is rather less open. I know that is by no means the only bottleneck we have; it's just the one that looms large in my mind at the moment. This would imply that the structure used by the Usability Initiative is something we want to emulate — a few tight-knit teams that can focus on specific concerns, containing people with the power to say yes or no to particular features/ideas. Having a person responsible for saying no is essential; as Danese says, you can't accept every random patch or idea that gets thrown your way, and leaving things languishing forever is less kind than just saying no (ideally with a reason). Hiding behind team decisions is impersonal to the point of rudeness — if I'm committing into your area of interest, I am part of your team. Having teams responsible (and with authority) for different areas, able to approve features and code, sounds fabulous. I know we are moving towards broader team structure already. Ideally there might be more than one person on each team who could give the thumbs up/down. The other advantage of having teams is that it makes it more explicit which areas of development are being focussed on, and by whom. Wikimedia obviously concentrates a reasonable amount on fundraising, which is essential as a means, but it doesn't achieve anything directly. I'd hope that having some kind of explicit structure would expose any less obvious blind-spots — my personal gripe is that no time gets spent on Wiktionary (cf. bug 20246). Clearly there are downsides, cliques are likely to form. I'd certainly regard it as a failure of the system if it stopped someone from doing something merely because they were currently on the wrong team — there's not much point in keeping lists of team members, perhaps all that is needed is a team leader (responsible for accepting or refusing changes/ideas), and the team is simple those who are talking to him. In case it's not obvious, I would not make team leaders exclusively employees, but use the meritocracy previously described that would only make it likely that most of them would be. The other sentiment I very much agree with is that more communication should be public — for the simple reason that if I don't know what you're doing, I can't help. Keeping track (however loosely) of what's being worked on is much more efficient than trying to second guess. If the channels we have are too noisy, it's easy to split them by topic. I like the idea of making #mediawiki the support channel, and having #mediawiki-dev for development — this is a model used by lots of projects on freenode. We could even have #mediawiki-offtopic too, if people want to do a lot of misc chatting. Being able to talk to other developers is very useful for getting things done, whereas having some developers who can't be contacted at all by others is very troublesome indeed. I had assumed that it was a requirement for
Re: [Wikitech-l] Community vs. centralized development
Okay, so here's my take... First of all I want to thank Aryeh for taking the time to write out the original mail in the thread. You've obviously done quite a bit of thinking. I realize there has been discontent and even concern with the way things have been / are between Foundation tech staff / paid contractors and everyone else on the volunteer dev side of this Project upon which we all toil. Aryeh, your mail was a well articulated instance of those feelings. I'm not going to deconstruct your suggestions; I actually agree with several of them (although I think you'd find the reality of flagship open source projects somewhat different than the legend). Instead I'm going to make a more general statement about why and how we're trying to shape our future efforts... As I came into WMF, the UX project was still in full swing and we decided not to interrupt their momentum until the Stanton Grant was finished. I focused my efforts instead on planning for the new Fiscal Year's budget and hiring plans and on observing the work patterns of the existing Tech staff, the expectations and needs of the Programs staff and the community interaction dynamic. During most of this time I've been close to silent on public lists because my opinions were still in formation. I figured there was nothing worse than somebody rushing in with a lot of uninformed blathering. Gradually I decided these were the most serious problems (in order of importance) to tackle with a new design WMF Tech: 1. Eliminate single points of failure / bottlenecks 2. Reconfigure into teams designed to encourage faster (shorter duration) and more accurate projects / deployments 3. Develop programs to encourage / grow volunteers into paid staff...recognizing that as a non-profit WMF needs to plan for more frequent turnover in the Tech department to ensure that we can grow expertise across the dev community rather than concentrating it in a few core people. 4. Restore the balance between community-led and foundation-led efforts so WMF feels again like a partnership rather than concentric circles of permission / blame Once I recognized these objectives, I started working out how to reach them. Notice that they are in order of importance above...starting with hiring / training / documenting and otherwise creating redundancies to mitigate the SPOF problem (in Operations to keep the sites functioning for instance) is a big focus of the hiring plan for this Fiscal Year. Fully 1/3 of the Tech hires in this fiscal year are SPOF related. Item 2 is necessary to again make us very productive (as at least one respondent on the current thread believes we should be given there are more of us than ever before). Experienced developers know that more hands aren't necessarily better unless you can modularize work to form small agile teams within the larger structure to get discreet bits of work done. The Linux Kernel team does this, and Apache projects are implicitly organized this way. Most of the expense of this re-org hits in the next fiscal year (hiring engineering program managers to help teams stay undistracted and productive and to maintain an overview so discreet bits of work add up to the correct whole), so this year about 1/3 of the Tech hiring will serve this item. But the crux of Aryeh's original email is not about the first two items on my list...it really rests in items 3 and 4. It is my belief that the Foundation can not and should not plan to fund or achieve most development projects cathedral-style. To do so would not be consistent with our founding principles (and in any case wouldn't be possible given our resources). At the same time our Projects have grown in both complexity and popularity to the point where we can't we solely expect volunteerism to provide for all needs. There has to be a partnership...not a detente. To that end it is clear we need to improve transparency and afford more opportunities for participation and cooperation between paid Tech staff and the volunteer developer community. We need to undertake this shift with realistic expectations. We are never going to be interested in every scrap of volunteer engineering ever undertaken, but we are going to take steps to make it easier for volunteer engineers wishing to contribute to figure out what contributions are sought and how to gain increased responsibility in our essentially meritocratic community (which is how Mozilla finds their new paid staff as well). We're going to create more entry points and to get volunteers more involved. In this fiscal year 1/3 of the new Tech hires are in service of items 3 and 4... If you're still reading, then you must really care to know what I think. Thank you for that. Mostly I wanted to share that re-balancing the volunteer-to-paid staff dynamic is definitely a first-year priority for me, although its not more urgent than establishing a new primary Data Center
Re: [Wikitech-l] Community vs. centralized development
On 3 September 2010 00:51, Danese Cooper dcoo...@wikimedia.org wrote: 1. Eliminate single points of failure / bottlenecks 2. Reconfigure into teams designed to encourage faster (shorter duration) and more accurate projects / deployments 3. Develop programs to encourage / grow volunteers into paid staff...recognizing that as a non-profit WMF needs to plan for more frequent turnover in the Tech department to ensure that we can grow expertise across the dev community rather than concentrating it in a few core people. 4. Restore the balance between community-led and foundation-led efforts so WMF feels again like a partnership rather than concentric circles of permission / blame I really like item 2. When I was working on MediaWiki related stuff, the main problem I had was working out who to talk to. The answer, if you ask, seems always to be Tim — which is not very scalable (brilliant though Tim is). I'd hope that along with an organisation into more formal teams would come a structure where individuals are responsible (either permanently, or on rotation like the chromium) for subsets of MediaWiki/Wikimedia. This would imply that the structure used by the Usability Initiative is something we want to emulate — a few tight-knit teams that can focus on specific concerns, containing people with the power to say yes or no to particular features/ideas. Having a person responsible for saying no is essential; as Danese says, you can't accept every random patch or idea that gets thrown your way, and leaving things languishing forever is less kind than just saying no (ideally with a reason). Hiding behind team decisions is impersonal to the point of rudeness — if I'm committing into your area of interest, I am part of your team. The other advantage of having teams is that it makes it more explicit which areas of development are being focussed on, and by whom. Wikimedia obviously concentrates a reasonable amount on fundraising, which is essential as a means, but it doesn't achieve anything directly. I'd hope that having some kind of explicit structure would expose any less obvious blind-spots — my personal gripe is that no time gets spent on Wiktionary (cf. bug 20246). Clearly there are downsides, cliques are likely to form. I'd certainly regard it as a failure of the system if it stopped someone from doing something merely because they were currently on the wrong team — there's not much point in keeping lists of team members, perhaps all that is needed is a team leader (responsible for accepting or refusing changes/ideas), and the team is simple those who are talking to him. In case it's not obvious, I would not make team leaders exclusively employees, but use the meritocracy previously described that would only make it likely that most of them would be. The other sentiment I very much agree with is that more communication should be public — for the simple reason that if I don't know what you're doing, I can't help. Keeping track (however loosely) of what's being worked on is much more efficient than trying to second guess. If the channels we have are too noisy, it's easy to split them by topic. I like the idea of making #mediawiki the support channel, and having #mediawiki-dev for development — this is a model used by lots of projects on freenode. We could even have #mediawiki-offtopic too, if people want to do a lot of misc chatting. Being able to talk to other developers is very useful for getting things done, whereas having some developers who can't be contacted at all by others is very troublesome indeed. I had assumed that it was a requirement for a wikimedia staff member to be somewhere public on freenode — clearly I was mistaken. I hope that items 1, 3 and 4 will become much easier if we solve 2 correctly, and communications will always be improvable. Good luck with your plans, and please keep us updated — I'd much rather have to add you to my spam filter than to not know what's happening. Conrad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 03.09.2010, 4:40 Roan wrote: * Shut down #wikimedia-dev (formerly #wikipedia_usability, kind of). The explicit purpose of the channel is to allow development discussion with less noise, but noise here means community involvement. In community development, you do get a lot more discussion, but that's not something you should try avoiding. In general, use existing discussion fora wherever possible, and if you do fragment them, make sure you don't have too much of a staff-volunteer split in which fora people use. No, noise means bots and people trying to support people with questions like how do I disable anon reads on my wiki as opposed to developers (paid and unpaid alike) being engaged in a design discussion. Maybe #wikimedia-dev should be renamed to #mediawiki-dev to remove the suggestion of WMF-exclusivity, but I definitely see the value of a channel dedicated to communication between developers with support questions and bots kept out. It should ultimately be decided whether MediaWiki is a software developed by an open community for everyone to use, or just a software that runs Wikipedia and is sporadically released to public only for lulz and Wikia? If it's the latter, such noise is indeed unwanted. If the former, this chit-chat is vital to being a part of the community. -- Best regards, Max Semenik ([[User:MaxSem]]) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On Thu, Sep 2, 2010 at 8:40 PM, Roan Kattouw roan.katt...@gmail.com wrote: I also think that we already have a fair number of tech employees outside of San Francisco, and AFAIK we're definitely open to hiring remote people for tech jobs unless in-person interaction is essential, say for a CTO or an EPM (although we do have a half-remote EPM). I do agree that the current level of community participation and feeling like you're part of the community is lacking, but I don't agree with your solutions. What solutions do you propose? You're assuming that development discussions actually take place in these channels, which is not the case. We mostly use them for chit-chat and private or office-related things. Questions like how should I design X in my code very rarely show up in these channels, and I don't think anyone would have a problem with being more conscious about this and moving to a public place for such a discussion. I'm not assuming that -- I've been idling in the secret channel for a while now. (I keep almost saying its name, argh. Channels that aren't access-restricted and rely on secret names are annoying.) Most of it is just chitchat. But that's exactly something that the broader community should be part of, so that staff doesn't form its own group that excludes volunteers. 95% of it could easily be public, and the rest could easily be taken to private e-mail or PM. There is not enough stuff that needs to be private to justify a whole channel. Some of your points are good, some I disagree with, and some I think are based on paranoia-fed overestimations as to how secretive we're being. Let me put it this way: I am saying what I perceive as a volunteer. If the goal is to make volunteers feel like they're a full-fledged part of the development community, then the fact that I made this post and the fact that every single volunteer who's commented so far appears to basically agree with me means that ipso facto, my complaints are valid. You're looking at it from the perspective of someone who sees all this stuff. Oh, don't worry, it's nothing important. Well, thank you for reassuring me -- but I'm capable of deciding myself whether something is important, if I can see it. Maybe I'll find some of it important. But I'm given no opportunity. Nor is any other volunteer. The problem isn't necessarily the actual content of what we can't see, but the mere fact that so much is clearly hidden. It draws a line that need not exist. When you hide things from volunteers, you cannot turn around and blame them for inaccurate speculation about what you've hidden. If you don't want speculation (or paranoia, as you put it), there's an easy solution: make it public. Anything short of that is not really going to solve the problem. On Thu, Sep 2, 2010 at 8:08 PM, MZMcBride z...@mzmcbride.com wrote: In large part, the problems and solutions are obvious (you pointed out most of them, and this is hardly the first time this has come up). The issue is that those in power (those who sign the paychecks and employment contracts) are deliberately choosing to ignore these problems and their solutions. I will reply to you in this thread only to say that as usual, you are being aggressive and unconstructive, and your input is unwelcome. Personally, I didn't even read any of your posts, or the responses to them, because I knew from long experience that it would be a waste of time. Part of building a successful community is allowing anyone the chance to speak, and part of building a successful community is evicting those who took their chance and blew it. Your contributions consistently clock in only moderately above the level of unconstructiveness that would justify actually banning you, and your endorsement of the point I'm making here will do nothing but harm it. You will not accomplish anything positive by attacking against the people who are responsible for fixing the problem, even if you were actually correct about the cause of the problem (which you are not). My advice to you right now is that the most useful thing you can do is not comment in this thread again. On Thu, Sep 2, 2010 at 9:20 PM, Erik Moeller e...@wikimedia.org wrote: I don't agree with unrealistic suggestions (e.g. face-to-face meetings have very real and very serious productivity advantages that we don't want to lose) But somehow most open-source projects do very well without them, including projects much bigger and better-funded than MediaWiki. I did try to emphasize this in my initial post, but several people apparently missed it. Do you think Wikimedia could do better than to emulate any of the high-profile projects that use almost no face-to-face conversation? but we've generally been trending in this direction. My observation as a volunteer is the opposite. Finally, most of these decisions aren't made by executive fiat -- Wikimedia is a very collaborative organization, and virtually all the decisions
Re: [Wikitech-l] Community vs. centralized development
On Fri, Sep 3, 2010 at 11:54 AM, Max Semenik maxsem.w...@gmail.com wrote: On 03.09.2010, 4:40 Roan wrote: * Shut down #wikimedia-dev (formerly #wikipedia_usability, kind of). The explicit purpose of the channel is to allow development discussion with less noise, but noise here means community involvement. In community development, you do get a lot more discussion, but that's not something you should try avoiding. In general, use existing discussion fora wherever possible, and if you do fragment them, make sure you don't have too much of a staff-volunteer split in which fora people use. No, noise means bots and people trying to support people with questions like how do I disable anon reads on my wiki as opposed to developers (paid and unpaid alike) being engaged in a design discussion. Maybe #wikimedia-dev should be renamed to #mediawiki-dev to remove the suggestion of WMF-exclusivity, but I definitely see the value of a channel dedicated to communication between developers with support questions and bots kept out. Have to say this is the first I've heard of this channel existing. Yay for communication. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On Thu, Sep 2, 2010 at 8:40 PM, Roan Kattouw roan.katt...@gmail.com wrote: * Shut down #wikimedia-dev (formerly #wikipedia_usability, kind of). The explicit purpose of the channel is to allow development discussion with less noise, but noise here means community involvement. In community development, you do get a lot more discussion, but that's not something you should try avoiding. In general, use existing discussion fora wherever possible, and if you do fragment them, make sure you don't have too much of a staff-volunteer split in which fora people use. No, noise means bots and people trying to support people with questions like how do I disable anon reads on my wiki as opposed to developers (paid and unpaid alike) being engaged in a design discussion. Maybe #wikimedia-dev should be renamed to #mediawiki-dev to remove the suggestion of WMF-exclusivity, but I definitely see the value of a channel dedicated to communication between developers with support questions and bots kept out. As I've said elsewhere to people, this isn't an excuse for fracturing the discussion. Using a single channel for development *and* support has worked for *years* until the Usability Initiative decided it needed its own channel. The channel is not actually that busy if you don't count the bots. And for those of you who *really* don't like them, you can /ignore them. I don't consider people asking for help noise either, it's part of being engaged with the community. I don't see the need for a separate development channel at all. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Aryeh Gregor writes, I'm not assuming that -- I've been idling in the secret channel for a while now. (I keep almost saying its name, argh. Channels that aren't access-restricted and rely on secret names are annoying.) Most of it is just chitchat. But that's exactly something that the broader community should be part of, so that staff doesn't form its own group that excludes volunteers. As a student of human communication, I'd like to affirm what Aryeh says. Chit-chat is phatic communication -- part of group-forming and the way that people form social ties and learn how to work together. Saying we chit-chat over here and discuss development over there imagines a false dichotomy between the two kinds of conversations, and draws a line between the two groups. It ought to be we chit-chat and discuss development together. Pete ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Thanks you Aryeh for your excellent comment (Score: 5, Insightful). I fully agree that excluding volunteer coders from decision making processes is a dangerous path, which in the long run could cost WMF more than the time spent on including the community in a collaborative and open way. As an occasional volunteer coder (most of my contributions to Wikimedia are on other projects), my focus is less on lack of involvement in decision making (even though I can fully understand that being an issue for more experienced volunteer coders). A lot frustration emerges from the huge code review lag, which has been a known problem for a long time and I don't see any improvement despite the large amount of hiring WMF has conducted. I for one have mostly withdrawn from my attempt to get involved more deeply in MediaWiki's development. -- Tobias signature.asc Description: OpenPGP digital signature ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 2010-09-02, Aryeh Gregor wrote: Over the last couple of years, MediaWiki development has moved from being almost entirely volunteer-based to having a large contingent of paid developers. A lot of people have noted that this has led to a lot of work being done without much community involvement. Just for a basic statistic, in July, I estimate that about 90% of non-localization commits to extensions/UsabilityInitiative/ were by paid employees. (I use employee loosely in this post, to include all paid staff, such as contractors.) By contrast, about 25% (ballpark figure) of non-localization commits to phase3/ were by paid employees, and the number of volunteer commits to phase3/ was much higher than the total number of commits to UsabilityInitiative, so this isn't just a matter of community members not doing as much work overall. ... I'd just like to repeat what others have said, and note that I agree with Aryeh's comments and replies 100%. It's very dissapointing to see many of the suggestions discarded almost immediatley by most of the staff members replying as unrealistic. These are well thought out points and I think a decent amount of consideration should be given to all of them, regardless of anyones predispositions. Robert ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 9/3/10 4:55 PM, Robert Leverington wrote: It's very dissapointing to see many of the suggestions discarded almost immediatley by most of the staff members replying as unrealistic. I can't speak for others, but I have to say that the idea of having paid developers without a space for them to work together in person is very unrealistic. My first month working for the WMF was as a remote contractor, and let me tell you it was a miserable experience. There is a reason why virtually every job posting for a programmer in any organization says no remote contractors. It is almost impossible to work effectively in a software organization without being physically in the same room at least part of the time, and it's especially difficult to ramp up quickly as a new employee. Now, with open source projects, contributors have already self-selected, passed through many filters, and probably gone through a slow process of learning how to contribute. (But you should consider though that something like 9/10 people who want to contribute to such projects usually find the barriers too high). So, people who are already part of the Wikimedia community can probably tolerate more remote work. But even open source projects often hold hackathons or other coding get-togethers, and I think everyone will agree these are far more productive. To his credit Aryeh is aware of this but he believes that the productivity hit is worth it if it ensures the organization is 100% transparent. I just don't agree we need to go to such lengths. That said -- there are techniques which we could adopt which could help equalize things. For instance, Nat Friedman believes that when conducting conference calls, everyone should dial in (even people who could be in the same room).[1] [1] http://nat.org/blog/2010/04/everyone-dials-in/ -- Neil Kandalgaonkar ( ne...@wikimedia.org ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 2010-09-03, Neil Kandalgaonkar wrote: On 9/3/10 4:55 PM, Robert Leverington wrote: It's very dissapointing to see many of the suggestions discarded almost immediatley by most of the staff members replying as unrealistic. I can't speak for others, but I have to say that the idea of having paid developers without a space for them to work together in person is very unrealistic. In the past all paid developers worked remotely (at least, not in the same office as one another), and there still are paid developers who work remotely. Additionally, all volunteers work remotely. Based on my experience with MediaWiki I would say that development in the past was significantly more efficient and community involved than it is currently. I can understand how this suggestion in particular might not be entirely realistic, but it is only one of the suggestions that have been put forward and regardless of whether it is taken as a whole has some amount of merit in the context of this discussion. Robert ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
2010/9/4 Robert Leverington rob...@rhl.me.uk: In the past all paid developers worked remotely (at least, not in the same office as one another), and there still are paid developers who work remotely. Additionally, all volunteers work remotely. Based on my experience with MediaWiki I would say that development in the past was significantly more efficient and community involved than it is currently. http://en.wikipedia.org/wiki/Post_hoc_ergo_prompter_hoc The fact that we have scalability (in terms of code review) and transparency issues now that we have a number of devs in one office while we didn't have them back in the day when there was only one dev at the office doesn't mean the concentration of developers in the office *caused* these issues, much less that undoing said concentration will fix them. For instance, the activity level in the MW SVN repository grew significantly about 2 years ago if memory serves [1] , and our code review infrastructure shrunk by 50% with Brion's departure just under a year ago rather than being expanded. This has to be one of the main causes of the current code review situation, and I don't believe concentrating devs in the office made much if any difference here. Certain transparency issues that have been mentioned probably are related to having an office, but you'll still need to make sound arguments to support this notion (fortunately, some people have done this) rather than committing a logical fallacy. You can't just blame any arbitrary event that occurred in the past 5 years for everything that's worse now than it was 5 years ago without backing up that assertion with convincing arguments. Roan Kattouw (Catrope) [1] These numbers blow my mind every so often: when I started in July 2007 we were in the r26000s vs. the r72000s today, even though the SVN history goes back to 2001. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 9/3/2010 11:55 PM, Roan Kattouw wrote: 2010/9/4 Robert Leverington rob...@rhl.me.uk: In the past all paid developers worked remotely (at least, not in the same office as one another), and there still are paid developers who work remotely. Additionally, all volunteers work remotely. Based on my experience with MediaWiki I would say that development in the past was significantly more efficient and community involved than it is currently. http://en.wikipedia.org/wiki/Post_hoc_ergo_prompter_hoc The fact that we have scalability (in terms of code review) and transparency issues now that we have a number of devs in one office while we didn't have them back in the day when there was only one dev at the office doesn't mean the concentration of developers in the office *caused* these issues, much less that undoing said concentration will fix them. For instance, the activity level in the MW SVN repository grew significantly about 2 years ago if memory serves [1] , and our code review infrastructure shrunk by 50% with Brion's departure just under a year ago rather than being expanded. This has to be one of the main causes of the current code review situation, and I don't believe concentrating devs in the office made much if any difference here. It grew, then has mostly been dropping since. The total number of commits is down from a peak in 2008. There were 5% fewer commits overall in 2009 than 2008, and there were 20% fewer in phase3. 4 of the top 5 months for most phase3 commits are in 2008. Based on the number of 2010 commits to date, it will be a similar drop this year (3% overall, 21% phase3) I made a graph of phase3 commits per quarter - http://www.mediawiki.org/wiki/File:Commits.png The second quarter of this year had the fewest commits since Q3 2006. Certain transparency issues that have been mentioned probably are related to having an office, but you'll still need to make sound arguments to support this notion (fortunately, some people have done this) rather than committing a logical fallacy. You can't just blame any arbitrary event that occurred in the past 5 years for everything that's worse now than it was 5 years ago without backing up that assertion with convincing arguments. Roan Kattouw (Catrope) [1] These numbers blow my mind every so often: when I started in July 2007 we were in the r26000s vs. the r72000s today, even though the SVN history goes back to 2001. -- Alex (wikipedia:en:User:Mr.Z-man) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Community vs. centralized development
Over the last couple of years, MediaWiki development has moved from being almost entirely volunteer-based to having a large contingent of paid developers. A lot of people have noted that this has led to a lot of work being done without much community involvement. Just for a basic statistic, in July, I estimate that about 90% of non-localization commits to extensions/UsabilityInitiative/ were by paid employees. (I use employee loosely in this post, to include all paid staff, such as contractors.) By contrast, about 25% (ballpark figure) of non-localization commits to phase3/ were by paid employees, and the number of volunteer commits to phase3/ was much higher than the total number of commits to UsabilityInitiative, so this isn't just a matter of community members not doing as much work overall. I've commented on this a few times before, but never at length. I think there's widespread confusion about what the problem even is, never mind how to solve it, so I'm writing this to set out at least my own views on the topic. Since my shorter remarks in other places tended to be misunderstood, I'll start at the beginning and go into considerable detail, which means this post will probably end up pretty long. I should say in advance that I'm discussing institutional problems here, not anything specific to individuals or projects, and no one should feel slighted if I pick them as an example. If you aren't really interested, start skimming. ;) Let me begin with definitions. I will draw a basic distinction between community development and centralized development. I'll start with two motivating examples. Firefox is developed by a community. Everything involved in the project and its development is open. Most of the work is done by employees of Mozilla, and all important decisions are made by employees of Mozilla, but anyone on the Internet can view what's happening and get involved. Bugs you open might get ignored forever, and you might have to poke people a bunch to get patches reviewed, and you might have to tolerate a considerable amount of bluntness and follow other people's marching orders if you want to contribute anything. But in principle, any random person in the world can make largely the same contributions as a Mozilla employee. Internet Explorer is developed by a centralized team. They have blogs where they sometimes share detailed info about their development process and reasoning. They very carefully read all user feedback left in the comments. They have a bug tracker where anyone can file bugs, and they guarantee that they'll look at and attempt to reproduce every single bug filed in a timely fashion. But although they pay close attention to feedback, giving feedback is the only way you can really participate without getting hired by Microsoft. You can't write any code, or have a voice in discussions at all comparable to an IE team member. These examples illustrate some important things: * Community development does not mean democracy. Even in a totally community-oriented project, all decisions might ultimately be made by a small group of individuals. (For instance, in the case of the Linux kernel, one person.) * Community development does not mean community members do most of the work. From what I've heard, employees of Mozilla write most of Firefox's code, but it's still completely community-oriented development. * Listening to feedback is not the same as actually involving the community. Even a totally closed project can be extremely attentive to feedback. In fact, it's common for community projects to be *less* receptive to feedback, taking a we'll listen to you when you write the code attitude. Keeping these in mind, I'll characterize a perfectly community-based development process like this: your say in the project is proportional to your contributions, and nothing prevents you from contributing as much as your time and ability allow. If you happen to be paid, it doesn't give you any additional say -- you just happen to be able to spend more time contributing. The decision-making process is open and transparent, and arguments are weighed on the basis of their merits and the speaker's history of contributions. This is of course not fully attainable in practice, but one can see how close or far a project is from the ideal. Centralized and community development processes both have advantages and disadvantages. Some of the advantages of centralized development (as relevant to open-source projects) are: * Paid employees don't have to spend time reviewing code from a lot of people who will only ever contribute a few patches, so they don't duplicate effort teaching everyone their project's coding conventions, or even educating them on basic things like XSS. * Because discussion can be private and everyone is more likely to be in similar time zones, it's possible to rely heavily on face-to-face or voice communication, which a lot of people are more comfortable with
Re: [Wikitech-l] Community vs. centralized development
On Fri, Sep 3, 2010 at 9:05 AM, Aryeh Gregor simetrical+wikil...@gmail.com wrote: ... I scrolled; but agree with the bits I read... I don't know how seriously these suggestions will be taken in practice by the powers that be, but I hope I've made a detailed and cogent enough case to make at least some impact. There is one very, very simple solution. All changes should either be: a) a patch on bugzilla, reviewed by anyone, and approved _on_bugzilla_ by a tech lead b) the landing of a branch, approved by a tech lead in some public forum. From that, all else flows. -- John Vandenberg ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On Thu, Sep 2, 2010 at 7:05 PM, Aryeh Gregor simetrical+wikil...@gmail.com wrote: I don't know how seriously these suggestions will be taken in practice by the powers that be, but I hope I've made a detailed and cogent enough case to make at least some impact. I hope so as well. You managed to articulate a feeling I share quite deeply. I agree with you on pretty much every point here. A /very/ strong +1 -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
2010/9/3 Aryeh Gregor simetrical+wikil...@gmail.com: * Consider what to do about code review. This is pretty much the hardest problem on this list, which is why I don't propose a specific solution here, but there has to be a better solution than assume a bunch of employees are trusted enough to sync their own code, force everyone else to wait months for central review. The current code review situation is a problem, and I don't have a ready-to-go solution for it either. Just wanted to point out I do completely agree with one of your important points before I start disagreeing with parts of almost all your other points. * Stop concentrating tech employees in San Francisco. Either have most of them work from home, or perhaps establish other small offices so that they're split up. The point is, make them rely on telecommunication, because if you put people in the same office they'll talk a lot face-to-face, and volunteers simply cannot participate. The purpose of putting people together in an office is so that they work together as a team, and this is exactly what you do *not* want, because volunteers cannot be part of that team. This is the second-hardest problem, or maybe the hardest, and I can't give a full solution for it either. I'd suggest checking with Mozilla about how they do it, because I know they do have offices, but they're a perfect example of community-oriented development. I personally don't think it should be necessary to enforce discipline (i.e. doing stuff in public) this way. Sometimes, you just want to bounce your ideas off this one person that happens to be an expert in that specific field. To give another example, I did in-person code review at the office with Ryan Kaldari last week, which was very productive. Both are inherently one-on-one and don't need to happen in public: in the bouncing ideas case, you're gonna take it public later after one person has told you it's not totally insane, in the code review case there's barely any benefit to doing it in public because it's really this one guy reviewing revisions written by this other guy. I also think that we already have a fair number of tech employees outside of San Francisco, and AFAIK we're definitely open to hiring remote people for tech jobs unless in-person interaction is essential, say for a CTO or an EPM (although we do have a half-remote EPM). I do agree that the current level of community participation and feeling like you're part of the community is lacking, but I don't agree with your solutions. * Explicitly encourage all paid developers to do everything in public and to treat volunteer developers as they would paid ones. I'm not saying this should be enforced in any particular manner, but it should be clearly stated so that everyone knows how things are intended to be. I mostly agree here. As I said above I think there's things that don't need to happen in public (little to no added value while raising the bar: walking over to someone's cubicle is easier than broadcasting your possibly stupid idea to the world), but I agree that we're not doing enough in public at this time. * Shut down the secret staff IRC channel. Development discussion can take place in #mediawiki, ops in #wikimedia-tech, other stuff in #wikimedia or whatever. If users interfere with ops' discussions sometimes in #wikimedia-tech during outages or such, set all sysadmins +v and set the channel +m as necessary. That's worked in the past. You're assuming that development discussions actually take place in these channels, which is not the case. We mostly use them for chit-chat and private or office-related things. Questions like how should I design X in my code very rarely show up in these channels, and I don't think anyone would have a problem with being more conscious about this and moving to a public place for such a discussion. * Shut down #wikimedia-dev (formerly #wikipedia_usability, kind of). The explicit purpose of the channel is to allow development discussion with less noise, but noise here means community involvement. In community development, you do get a lot more discussion, but that's not something you should try avoiding. In general, use existing discussion fora wherever possible, and if you do fragment them, make sure you don't have too much of a staff-volunteer split in which fora people use. No, noise means bots and people trying to support people with questions like how do I disable anon reads on my wiki as opposed to developers (paid and unpaid alike) being engaged in a design discussion. Maybe #wikimedia-dev should be renamed to #mediawiki-dev to remove the suggestion of WMF-exclusivity, but I definitely see the value of a channel dedicated to communication between developers with support questions and bots kept out. * Don't conduct teleconferences about development, ever. Even if volunteers are invited (are they?), time zones and non-MediaWiki obligations make all synchronous
Re: [Wikitech-l] Community vs. centralized development
On 2 September 2010 17:40, Roan Kattouw roan.katt...@gmail.com wrote: 2010/9/3 Aryeh Gregor simetrical+wikil...@gmail.com: * Shut down the secret staff IRC channel. Development discussion can take place in #mediawiki, ops in #wikimedia-tech, other stuff in #wikimedia or whatever. If users interfere with ops' discussions sometimes in #wikimedia-tech during outages or such, set all sysadmins +v and set the channel +m as necessary. That's worked in the past. You're assuming that development discussions actually take place in these channels, which is not the case. We mostly use them for chit-chat and private or office-related things. Questions like how should I design X in my code very rarely show up in these channels, and I don't think anyone would have a problem with being more conscious about this and moving to a public place for such a discussion. I don't think he is making that assumption, lots of chit-chat happens on any channel — it makes us feel left out if you have a special one (humans are really bad at the jealousy thing). What private or office-related things are there that you can't share with developers? Conrad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Aryeh Gregor wrote: I don't know how seriously these suggestions will be taken in practice by the powers that be, but I hope I've made a detailed and cogent enough case to make at least some impact. In large part, the problems and solutions are obvious (you pointed out most of them, and this is hardly the first time this has come up). The issue is that those in power (those who sign the paychecks and employment contracts) are deliberately choosing to ignore these problems and their solutions. This isn't an assumption of bad faith and I won't hear anything of the sort. It's the reality. The problems are obvious; the solutions are obvious. What isn't obvious is why certain people in executive positions have chosen to ignore the problems. It would take a matter of minutes to shutdown the private IRC channels and private mailing lists. It would take one order from one of the members of the executive team for substantive code review and deployment to take place. But the current reality is that if it isn't part of usability or fundraising, it doesn't get any type of attention or priority. MZMcBride ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
2010/9/3 MZMcBride z...@mzmcbride.com: In large part, the problems and solutions are obvious (you pointed out most of them, and this is hardly the first time this has come up). The issue is that those in power (those who sign the paychecks and employment contracts) are deliberately choosing to ignore these problems and their solutions. This isn't an assumption of bad faith and I won't hear anything of the sort. It's the reality. The problems are obvious; the solutions are obvious. What isn't obvious is why certain people in executive positions have chosen to ignore the problems. Your allegations that these problems are deliberately being ignored is a serious one, and you may take my word for it (although I'm fairly sure that you won't) that these people definitely care. I think you're wrong in assuming that all these solutions are totally obvious to everyone: serious thought needs to be given to this, and these people have more issues on their mind that just this single one. You are right that there doesn't seem to have been any concrete action or clear statements from people in key positions (say Erik, or, better, Danese) and I very much want that to change. I just disagree with the assertion that they don't care. It would take a matter of minutes to shutdown the private IRC channels and private mailing lists. You're assuming this is one of the obvious solutions, which I contend above. It would take one order from one of the members of the executive team for substantive code review and deployment to take place. Oh really? So I guess we have dozens of people capable of and available for reviewing and deploying code? We don't. As you have said yourself and Aryeh has pointed out, review and deployment has been a problem for a long time, and if one order could have solved it that would've happened long ago. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Roan Kattouw wrote: 2010/9/3 MZMcBride z...@mzmcbride.com: In large part, the problems and solutions are obvious (you pointed out most of them, and this is hardly the first time this has come up). The issue is that those in power (those who sign the paychecks and employment contracts) are deliberately choosing to ignore these problems and their solutions. This isn't an assumption of bad faith and I won't hear anything of the sort. It's the reality. The problems are obvious; the solutions are obvious. What isn't obvious is why certain people in executive positions have chosen to ignore the problems. Your allegations that these problems are deliberately being ignored is a serious one, and you may take my word for it (although I'm fairly sure that you won't) that these people definitely care. I think you're wrong in assuming that all these solutions are totally obvious to everyone: serious thought needs to be given to this, and these people have more issues on their mind that just this single one. You are right that there doesn't seem to have been any concrete action or clear statements from people in key positions (say Erik, or, better, Danese) and I very much want that to change. I just disagree with the assertion that they don't care. I'm not sure if you intended it as such, but this reads as an appeal to emotion. It's not a matter of feelings or a matter of whether someone is committed to Wikimedia's mission or anything of that sort. That is, it isn't about whether people care in the sense in which you're using it. It's a matter of whether those in control are making it a priority. And from where I'm sitting, it seems fairly clear that general code review and deployment isn't being made a priority. At least where I work, if my boss says to do something, it gets done. And, more to the point, how many managers (or other employees) need to be hired before serious thought can be devoted to these issues? They seem fairly fundamental to me for an organization that runs websites. It would take a matter of minutes to shutdown the private IRC channels and private mailing lists. You're assuming this is one of the obvious solutions, which I contend above. It seems fairly obvious to me. Aryeh's points about #wikimedia-dev seem fairly spot-on. Do you disagree? If so, why? It would take one order from one of the members of the executive team for substantive code review and deployment to take place. Oh really? So I guess we have dozens of people capable of and available for reviewing and deploying code? We don't. As you have said yourself and Aryeh has pointed out, review and deployment has been a problem for a long time, and if one order could have solved it that would've happened long ago. Yes, really. When it's fundraising-related or Usability-related, there seems to be no issue with code deployment. The server admin log bears me out on this.[1] So yes, I will contend that there would be man-power for review and deployment of the rest of the codebase if it were made a priority. MZMcBride [1] http://wikitech.wikimedia.org/view/Server_admin_log ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
All of us at WMF care and follow discussions like this, especially clearly laid out and well thought-out analysis like Aryeh's original post. We don't always agree. :-) I know Danese is planning to weigh in, so I won't write too much at this time, but will point to this post from a couple of months ago where I laid out my view on some (not all) of these topics. http://lists.wikimedia.org/pipermail/foundation-l/2010-June/059052.html In general, I think most of us are in favor of more public communications, including public lists, participation in public IRC channels, wikis, etc. I don't agree with unrealistic suggestions (e.g. face-to-face meetings have very real and very serious productivity advantages that we don't want to lose), but we've generally been trending in this direction. Finally, most of these decisions aren't made by executive fiat -- Wikimedia is a very collaborative organization, and virtually all the decisions about how/where to communicate and work have been made by the people doing the work. I would wager a guess that some less constructive individuals on this list and others play a role in what choices people make. :) So, if you're trying to change the dynamics, please call out and help put an end to unconstructive and unprofessional behavior just as much as you ask for public collaboration. -- Erik Möller Deputy Director, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
Erik Moeller wrote: All of us at WMF care and follow discussions like this, especially clearly laid out and well thought-out analysis like Aryeh's original post. We don't always agree. :-) I know Danese is planning to weigh in, so I won't write too much at this time, but will point to this post from a couple of months ago where I laid out my view on some (not all) of these topics. In general, I think most of us are in favor of more public communications, including public lists, participation in public IRC channels, wikis, etc. I don't agree with unrealistic suggestions (e.g. face-to-face meetings have very real and very serious productivity advantages that we don't want to lose), but we've generally been trending in this direction. Finally, most of these decisions aren't made by executive fiat -- Wikimedia is a very collaborative organization, and virtually all the decisions about how/where to communicate and work have been made by the people doing the work. I realize that management styles can and will differ, but here's the breakdown for posts by Danese to wikitech-l since she was hired in late January / early February: Feb 2010: 0 Mar 2010: 5 Apr 2010: 0 May 2010: 0 Jun 2010: 2 Jul 2010: 1 Aug 2010: 2 That's ten posts to the central Wikimedia development mailing list in seven months. Are you suggesting Danese is just a very quiet person? And while I have these tabs open, your stats for the same time period, Erik: Feb 2010: 0 Mar 2010: 3 Apr 2010: 1 May 2010: 0 Jun 2010: 0 Jul 2010: 1 Aug 2010: 1 That would be six posts in seven months. Do you think this is acceptable? Do you think it leaves anyone on the outside (or on the inside) with the perception that the Wikimedia technical executive team is concerned with being engaged with the community? When you contrast Danese's stats or your stats with Brion's or Tim's, what do you think the underlying message is? MZMcBride ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On 9/2/2010 9:04 PM, Roan Kattouw wrote: Oh really? So I guess we have dozens of people capable of and available for reviewing and deploying code? We don't. As you have said yourself and Aryeh has pointed out, review and deployment has been a problem for a long time, and if one order could have solved it that would've happened long ago. A long time ago, we didn't have this much of a problem with code review (except extensions/branches) and deployment. A couple years ago, we updated the site to trunk at least once a month from what I recall. We still had big features (CentralAuth, preprocessor rewrite) and still ran a fundraiser. The foundation now has probably 3 times as many staff members as then, but from the community's POV seems to get less done. -- Alex (wikipedia:en:User:Mr.Z-man) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 37-01--10 03:59 PM, Alex wrote: The foundation now has probably 3 times as many staff members as then, but from the community's POV seems to get less done. Seems true to me, except in the case of the Vector project where the red carpet gets laid out. Obviously a concern - especially since the Foundation has consistently said they aim to hire smart instead of fast. Is the tech team really doing better now than they were then? - -Mike -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAkyAXV4ACgkQst0AR/DaKHtg6wCfUpcMSnRU78ZLGvWrSuh3GuaT K2kAn1RO6OcRNEuBNIkBGHETZ4NCEUPI =bUhE -END PGP SIGNATURE- ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
On Fri, Sep 3, 2010 at 3:05 AM, Aryeh Gregor simetrical+wikil...@gmail.com wrote: * Consider what to do about code review. This is pretty much the hardest problem on this list, which is why I don't propose a specific solution here, but there has to be a better solution than assume a bunch of employees are trusted enough to sync their own code, force everyone else to wait months for central review. There are three ways this problem may be dealt with: * People responsible for code review focus on code review. * People responsible for code review involve more people to review code. * People responsible for code review don't do code review and don't want to lose the control over code review process -- what is done now, does not work. The review backlog is one of the things I stopped contributing at some point. * Stop concentrating tech employees in San Francisco. Either have most of them work from home, or perhaps establish other small offices so that they're split up. The point is, make them rely on telecommunication, because if you put people in the same office they'll talk a lot face-to-face, and volunteers simply cannot participate. The purpose of putting people together in an office is so that they work together as a team, and this is exactly what you do *not* want, because volunteers cannot be part of that team. This is the second-hardest problem, or maybe the hardest, and I can't give a full solution for it either. I'd suggest checking with Mozilla about how they do it, because I know they do have offices, but they're a perfect example of community-oriented development. That won't work. Face-to-face communications are extremely efficient for many things (like Roan pointed out, it works well with code review). * Explicitly encourage all paid developers to do everything in public and to treat volunteer developers as they would paid ones. I'm not saying this should be enforced in any particular manner, but it should be clearly stated so that everyone knows how things are intended to be. Or at least document all their decision in public. * Shut down the secret staff IRC channel. Development discussion can take place in #mediawiki, ops in #wikimedia-tech, other stuff in #wikimedia or whatever. If users interfere with ops' discussions sometimes in #wikimedia-tech during outages or such, set all sysadmins +v and set the channel +m as necessary. That's worked in the past. AFAIK this is mostly a sysadmin channel. * Shut down #wikimedia-dev (formerly #wikipedia_usability, kind of). The explicit purpose of the channel is to allow development discussion with less noise, but noise here means community involvement. In community development, you do get a lot more discussion, but that's not something you should try avoiding. In general, use existing discussion fora wherever possible, and if you do fragment them, make sure you don't have too much of a staff-volunteer split in which fora people use. I'd rather rename it to #mediawiki-dev. * Stop using private e-mail for development, at least to any significant extent. If there are any internal development mailing lists or aliases or whatever used for development, retire them. Or at least make usabil...@wikimedia.org a publicly logged mailing list. I see no reason why it should not be (you may create a separate internal mailing list). ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
2010/9/3 Victor Vasiliev vasi...@gmail.com: Or at least make usabil...@wikimedia.org a publicly logged mailing list. I see no reason why it should not be (you may create a separate internal mailing list). It's not really being used anymore because the Stanton grant is over now, and the people that used to work on it are now working on other tasks while still supporting the existing UsabilityInitiative / Vector software and performing rollouts such as the one this week. Other such project-specific mailing lists should preferably be public, yes. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Community vs. centralized development
The quotes below are illustrative excerpts, my replies are to the whole post. On 03/09/10 09:05, Aryeh Gregor wrote: That's what leads to things like http://www.mediawiki.org/wiki/Special:Code/MediaWiki/67299. Some people said that maybe that could have been phrased better, or something. But the revert wasn't the problem, it was a symptom of the problem. The problem was that the design was decided on somewhere that volunteers couldn't or wouldn't participate. Of course you revert something that contradicts an agreed-upon design -- the problem is that the agreed-upon design was only agreed upon by a small group of employees. How are volunteers supposed to contribute in that environment, if they don't know what tune they're supposed to be dancing to? The usability team has shown some amount of arrogance and aloofness in their dealings with the developer community, and I understand that you were upset by that. But I don't think arrogance is a trait which is confined to employees, in fact I think it's a near-universal fault. Being able to get stuff done in spite of it is an essential skill. I think that all developers care about usability, and that UI design should be a part of every project team. I don't think we should have one team writing bad interfaces, and another team rewriting them to be usable, it's inefficient. Ideally, UI experts should be available to be assigned to any project, and this is already happening to some extent. Project-based teams should hopefully be more open and accessible than the usability team was. As for fundraising, the work is uninspiring, and I don't think we've ever managed to get volunteers interested in it regardless of how open we've been. * Shut down the secret staff IRC channel. Development discussion can take place in #mediawiki, ops in #wikimedia-tech, other stuff in #wikimedia or whatever. If users interfere with ops' discussions sometimes in #wikimedia-tech during outages or such, set all sysadmins +v and set the channel +m as necessary. That's worked in the past. Your recommendations seem insensitive and unrealistic. What works for you does not necessarily work for everyone. Some contributors (both employees and volunteers) are not comfortable talking in a public forum, and prefer to not say anything at all than to broadcast their ideas to the world on a publically logged IRC channel or mailing list. I used to rail against such sensitivities, but I've come to see that as unproductive. I still think that we should encourage people to use public forums, but only to a point, and that point should not be use the public forum or don't contribute at all, as you seem to be suggesting. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
