Re: [Wikitech-l] Making developer access easier to get
On 10/29/2011 04:01 AM, K. Peachey wrote: On Sat, Oct 29, 2011 at 9:54 AM, Sumana Harihareswara suma...@wikimedia.org wrote: * However, I am now ensuring that we are more lenient with extensions developers than we are with people applying for core commit access. We still, of course, watch out for security issues in submitted code samples! I might be reading this wrong, But isn't that exactly what we don't want, We want people building extensions even if they have bad security habits in SVN so we can teach them on how to correct them (and other generally improvements) so we know that they have improved the code they are offering to people. K. Peachey, thanks for making that point. I talked with Tim, Aaron, and Chad about this, and you're right. For extensions, the only criteria for commit access are, should be, and now will be as listed at https://www.mediawiki.org/wiki/Commit_access_requests#Requesting_commit_access : A demonstration that your request is made in good faith. For example, someone may be employing you to work on MediaWiki, or you may be known to the Wikimedia volunteer community by way of past work. Plans to contribute to MediaWiki or extensions in a substantial way. Programming skills appropriate for the type of work you propose to do. Our community will help new participants, especially with MediaWiki-specific skills, but we don't have time to train programmers from scratch. An account on this wiki (www.mediawiki.org) with an authenticated email address set in your Special:Preferences. We will tell people about security issues we see in their code, and ask them to promise to fix them in the future, but we won't use security problems as a reason to turn them away. We've also now unified the tools and extensions Subversion groups; there were a few developers (declerambaul, giovanni, halfak, swalker, whym) who were only in the tools group, and those developers now have access to commit to extensions. -- Sumana Harihareswara Volunteer Development Coordinator Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On Sat, Oct 29, 2011 at 9:54 AM, Sumana Harihareswara suma...@wikimedia.org wrote: * However, I am now ensuring that we are more lenient with extensions developers than we are with people applying for core commit access. We still, of course, watch out for security issues in submitted code samples! I might be reading this wrong, But isn't that exactly what we don't want, We want people building extensions even if they have bad security habits in SVN so we can teach them on how to correct them (and other generally improvements) so we know that they have improved the code they are offering to people. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On 10/04/2011 12:02 AM, Sumana Harihareswara wrote: On 10/03/2011 07:19 PM, Benjamin Lees wrote: On Mon, Oct 3, 2011 at 6:27 PM, Rob Lanphier ro...@wikimedia.org wrote: Since a lot of people are applying, there are quite a few to get through. Out of curiosity, how many is a lot in this context? From viewing the OTRS queue archives: 25 requests in the last 40 days. That's about 4.3 applications per week. This is up from about 29 in the last 60 days (3.4 apps/week), and 82 in the past 180 days (3.2 apps/week, also the rate over the entire 20 months we've been using the OTRS queue). I believe all these stats exclude spam. You see why I'd like more developers reviewing commit access requests; the more requests we get, the more time the code review takes, and I'd like to spread that out more -- both as a TODO and as a learning opportunity. And -- yay for our community that we're getting more frequent commit access requests! Along with the line on http://toolserver.org/~robla/crstats/crstats.trunkall.html going down, it's a sign of our development community's health and momentum. Hmm, it looks like the web archive http://lists.wikimedia.org/pipermail/wikitech-l/2011-October/055502.html left out my reply and thus the statistics. :-( To continue the rest of the thread: I recently talked with Chad, Tim and Aaron about this thread, and about the situation regarding developer commit access. My conclusions: * I'm not going to add any new technical process (such as a new queue just for extensions developers, or more granular kinds of permissions on svn.wikimedia.org) that focuses on SVN access/process, because that's not worth it, because we are migrating to Git this year. * However, I am now ensuring that we are more lenient with extensions developers than we are with people applying for core commit access. We still, of course, watch out for security issues in submitted code samples! * I'm also starting to explicitly encourage developers whom we're denying core access to still work on core *in branches* and request review. This way, they can suggest improvements via SVN (instead of having to use Bugzilla patches) and get feedback via CR. * I have already been reaching out to other core developers to ask them whether I should encourage certain patch submitters to apply for commit access. Now, I am increasing how often I privately ask other core and extensions developers to review an applicant's code samples, and (if I have reason to believe they've worked with the applicant) ask them whether s/he's a reasonable person and a competent developer. Getting someone we trust to vouch for an applicant will enhance our web of trust, and help Chad, Tim, and Aaron decide in an applicant's favor. I believe these steps make developer access easier to get, while still not adding substantially more post-commit code review workload. Thanks. -- Sumana Harihareswara Volunteer Development Coordinator Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On 11-10-03 07:08 PM, Olivier Beaton wrote: It seems pretty straight forward to me, #1 Core access? Keep existing review process in place, could take 6 months, sorry. Patches that will never get committed for you! #2 Extension access? Has a working extension attached to submission (has some .php files in it, no need for a code review here, the community will handle that) . Granted. Create their account, and give them access to _only_ their extension directory. SVN Doesn't work that way. We can apply limited restrictions like making the wmf branch and trunk only accessible by specific groups. But we can't create efficiently (I don't even know if it's possible at all) create a separate right for each and every extension and give new committers only access to their own extension. So anyone that gets commit implicitly has the ability to screw up any extension or tool other than phase3 or the wmf branch. Besides, such a pattern would suddenly mean that instead of an extension author going through review once, they'd have to go through review for each new extension they make so that we can add the new directory and right. If you want to foster a more communal and helping extension community, when it comes to do core access reviews, use a script to spit out active people on svn, and if any of them only have access to their extension directory, pro-actively have a short sentence or two about if you should give them access to all extensions. If yes, let them know the happy news, maybe they will do something with it (likely not, even if they had it in the first place). Another metric might be if they have a lot of tickets for other core extensions with patches attached. They can't be that destructive in their own space, and if they have a complete working extension, chances are people in the community will start using it, and invariably come into irc complaining if the quality is low, better to nip this one in the bud and get them into the code review process early. Like all the people being so helpful to me in IRC. I don't want to wait months to find out I've been doing things all wrong. For extension access there isn't many reasons why you can't make them an account on the spot and see how it goes, whoever is around and sees it first (even Sumana). I haven't used git, but won't you need to decide what gets comitted in git as well, no matter what they comit on their own HD? It will still be 'get it moved into a extension branch on git, have your revisions reviewed, get it into the extension distributor'. I see this applying no matter which way you go. In git each extension, core, or other project is a separate repo. Because of that whatever model we use with git it's possible to efficiently let a user have access just to their extensions. So that model you describe where anyone can have access to their own restricted area, that's the potential git model, not reasonably possible with svn. Additionally the plan seams to be to use gerrit to a large degree. Apparently the plan is what Ryan calls a gated trunk. For something like core when you push your commits in; Instead of the changes actually being applied right away the changesets are instead put into the repo but not applied to the master branch, and a new changeset shows up in the gerrit interface. At that point the change can be reviewed by other people, and will probably also be reviewed by scripts checking to see if the change breaks any tests. After a reviewer has okay'd a changeset gerrit merges that change into the main part of the repo. At least that's my understanding of how the gerrit setup works from my discussion with Ryan. I do need to bring up an e-mail about that. Just my 2c to the pile, now banking a dollar. - Olivier -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
I'm going to nip this one in the bud right away, and given that core and extension SVN access is already separate already, I already know we're making use of it. In your SVN auth file on the server: [/mediawiki/trunk/extensions/SomeExtension] username = rw [/mediawiki/tags/extensions/SomeExtension] username = rw [/mediawiki/branches/extensions/SomeExtension] username = rw Done. Right now I imagine for extension only access there is just 3 entries like the above, but for the whole extensions directory (I'm guessing) and one for the whole repo. And what review? You're not looking at code quality here, if they have a working extension that they are releasing, do you really want them pasting it onto the page on mw.org? I keep hearing people saying they want to check all those in. Where's the review there? At least with a checkin people have the opportunity to give them real feedback instead of being ignored and stagnate into obsoleteness on mw.org. And really I don't think you can really comit to maintaining every extension every person makes forever. Let's be realistic here, it's their extension, they will maintain it, and if not and it breaks after a long period of time, you archive it with no activity somewhere in /branches/archives or another plan. What I'm saying is requesting access + having a finished extension should mean someone can immediately just add them access. You're right this means one request per extension, but after 2 you should be asking yourself okay this person is contributing a lot to the community, maybe we can give them access to the whole ext dir? and actually review them. On Tue, Oct 4, 2011 at 3:35 AM, Daniel Friesen li...@nadir-seen-fire.com wrote: On 11-10-03 07:08 PM, Olivier Beaton wrote: #2 Extension access? Has a working extension attached to submission (has some .php files in it, no need for a code review here, the community will handle that) . Granted. Create their account, and give them access to _only_ their extension directory. SVN Doesn't work that way. We can apply limited restrictions like making the wmf branch and trunk only accessible by specific groups. But we can't create efficiently (I don't even know if it's possible at all) create a separate right for each and every extension and give new committers only access to their own extension. So anyone that gets commit implicitly has the ability to screw up any extension or tool other than phase3 or the wmf branch. Besides, such a pattern would suddenly mean that instead of an extension author going through review once, they'd have to go through review for each new extension they make so that we can add the new directory and right. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On 11-10-04 04:29 AM, Olivier Beaton wrote: I'm going to nip this one in the bud right away, and given that core and extension SVN access is already separate already, I already know we're making use of it. In your SVN auth file on the server: [/mediawiki/trunk/extensions/SomeExtension] username = rw [/mediawiki/tags/extensions/SomeExtension] username = rw [/mediawiki/branches/extensions/SomeExtension] username = rw Done. Right now I imagine for extension only access there is just 3 entries like the above, but for the whole extensions directory (I'm guessing) and one for the whole repo. Ok. Let's see. Firstly most extensions don't use tags so we can omit that, but we don't use /branches/extensions that way, it's more like /mediawiki/branches/REL#_##/extensions/SomeExtension. We'll need an extra line for each release, so say an extension was committed in 1.15 we'll need 4 entries now, and another when we branch 1.19. Ok, pretending that we only give one user access to an extension, that all extensions were committed in 1.15, and none use tags. We have 586 extensions inside /trunk/extensions/ currently. Doing the math, 586 (extension count) * 5 (entries per extension; trunk + 4 REL branches) * 2 (lines per entry) = 5860... So are you trying to seriously argue that ops should have to maintain 5860+ lines of config and add an extra 1172+ for every release we make, just to restrict extension authors into their own extensions? You are sane, right? And what review? You're not looking at code quality here, if they have a working extension that they are releasing, do you really want them pasting it onto the page on mw.org? I keep hearing people saying they want to check all those in. Where's the review there? At least with a checkin people have the opportunity to give them real feedback instead of being ignored and stagnate into obsoleteness on mw.org. And really I don't think you can really comit to maintaining every extension every person makes forever. Let's be realistic here, it's their extension, they will maintain it, and if not and it breaks after a long period of time, you archive it with no activity somewhere in /branches/archives or another plan. There's a beautiful tool we have called 'grep'. Plenty of the breaking changes we make in core are ones we make on purpose to improve the system and are ones that it's rather simple to find extensions using the old interface by doing a quick grep. Then we go through the ones using the old problematic interface, change them to use the new code, then commit. Tbh, if we threw unit testing into extensions we could make batch changes and ensure extension compatibility with new versions of MW even more confidently. Plenty of extensions are rather simple. Plenty already work when they commit. Plenty will continue working until a minor interface change means they need a tweak to be made to them. And plenty of those tweaks can be made en-masse. Along those lines we have plenty of extensions that can be 'maintained' indefinitely simply by sitting in trunk with no specific maintainer getting automatic i18n updates and small tweaks when an interface changes. The idea that only the extension's creator / maintainer would maintain an extension is flawed in the first place. People like to use extensions and people notice when they break, usually as the result of minor core changes. Hence, even if an extension no longer has an actual maintainer it may have people who have been using it. And all it takes to fix most issues is one of them to poke someone to make a minor tweak to the extension. A number of extensions may even be used by people who can develop but wouldn't bother taking the task of maintaining the extension. When an extension breaks generally one of them might happen to notice it breaking, fix it, and commit that fix into trunk so that they can use the functioning extension inside their own wiki. What I'm saying is requesting access + having a finished extension should mean someone can immediately just add them access. ;) and what I'm saying in our planned git workflow you probably won't even need that much to get access. You're right this means one request per extension, but after 2 you should be asking yourself okay this person is contributing a lot to the community, maybe we can give them access to the whole ext dir? and actually review them. On Tue, Oct 4, 2011 at 3:35 AM, Daniel Friesen li...@nadir-seen-fire.com wrote: On 11-10-03 07:08 PM, Olivier Beaton wrote: #2 Extension access? Has a working extension attached to submission (has some .php files in it, no need for a code review here, the community will handle that) . Granted. Create their account, and give them access to _only_ their extension directory. SVN Doesn't work that way. We can apply limited restrictions like making the wmf branch and trunk only accessible by specific groups. But we can't create efficiently (I don't even know if it's possible at
Re: [Wikitech-l] Making developer access easier to get
On Tue, Oct 4, 2011 at 7:29 AM, Olivier Beaton olivier.bea...@gmail.com wrote: I'm going to nip this one in the bud right away, and given that core and extension SVN access is already separate already, I already know we're making use of it. In your SVN auth file on the server: [/mediawiki/trunk/extensions/SomeExtension] username = rw [/mediawiki/tags/extensions/SomeExtension] username = rw [/mediawiki/branches/extensions/SomeExtension] username = rw Done. Right now I imagine for extension only access there is just 3 entries like the above, but for the whole extensions directory (I'm guessing) and one for the whole repo. And I'm going to nip this one in the bud right here. Yes it is technically feasible, but it isn't a scalable process at all. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
Oh boy. That was a bit uncalled for there, I'll try to reply to the parts that were civil. On Tue, Oct 4, 2011 at 9:51 AM, Daniel Friesen li...@nadir-seen-fire.com wrote: Ok. Let's see. Firstly most extensions don't use tags so we can omit that, Why not? Extensions have versions, why shouldn't they use tags? but we don't use /branches/extensions that way, it's more like /mediawiki/branches/REL#_##/extensions/SomeExtension. We'll need an Ah, I wasn't aware of that. That changes thing a little, as you so nicely pointed out. Clearly from my example I wasn't proposing adding 5000 lines to a config file. Clearly. Why you insinuate that is beyond me. Like my mention of tags, I was assuming extension authors might want to write branches for their code. I know I've done that a few times when I wanted to try out bigger new ideas or directions. The whole copy every extension at the point of branching as far as I understand it, is so that you can always get the extension at the version that existed during that release (and hopefully works and has less bugs). There's a beautiful tool we have called 'grep'. Plenty of the breaking That's fine for small updates and will undoubtedly keeps things running for awhile, but I don't see that working out long term for any reasonably complex extension. But I don't think I have to argue this point, because either way you look at it, you want that code in svn. You don't want to be sending people away, you want them checking it into svn so you can do those small mass updates. My ideas about access restrictions is meant to address this concern with giving people svn access. It's so that you can give more people access, more quickly, with less overhead (code review). The idea that only the extension's creator / maintainer would maintain an extension is flawed in the first place. People like to use extensions Right, you want other people to take over maintaining an extension, especially popular ones, if it gets abandoned. This is hard if you don't get people to check in their extensions, for which you need to give them access first and that's what this is all about. ;) and what I'm saying in our planned git workflow you probably won't even need that much to get access. As I understand git (which is not very much), it will solve some of these problems. Here's a workflow: a) I'm coding my extension on my machine with my own repo for my extension, I'm ready to do a release, I send it to the mw repo for merging into the extensions branch b) Who approves the merging into mw.org? Does it have to pass a code review just like core changes? Who will have the rights to do code reviews and say ok it's ready now? I'm all for code reviews, if there was a place I could submit my extensions for review, I'd do it in a heartbeat, I'm new to mw.org and I'd rather find out now if I'm doing things wrong or could do them better, than in a few months maybe if someone happens to look at it and feel I'm approachable enough. In the end though if you make that code-review a show stopper you may end up splitting the community a lot, instead of authors helping authors it will be some authors gate keeping and blocking code from others. If I make a feature or improvement to my extension that I've coded and whoever is reviewing doesn't think it needs the feature... really? MW.org is offering, or attempting to offer a really unique and great service. Versioning, code reviews, and distribution. Even We'll help you keep it up to date with core api changes!. Sounds fantastic, but if you can't get people to check in their code, and be able to have those checkins happen in a timely basis, people won't use the service, and just see it as elitist instead of the great thing it should be. In the short term my suggestion stands. don't give them access to branches/relX/extensions. That's not what I said. If you'd rather skip branches and tags altogether even that would be okay. Just give them access to trunk/phase3/extensions -- once they prove themselves, then extend that to all extensions and the rel branches extensions folders like you currently do. - Olivier ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
Is that a reply to Dan's branches/REL_*/extensions madness or a legitimate having 1-3 lines in the config per extension author we are trying out until they get full extension access is not a scalable process? Depending on how many people are requesting access, I can see how that could still be true. On Tue, Oct 4, 2011 at 9:57 AM, Chad innocentkil...@gmail.com wrote: On Tue, Oct 4, 2011 at 7:29 AM, Olivier Beaton olivier.bea...@gmail.com wrote: I'm going to nip this one in the bud right away, and given that core and extension SVN access is already separate already, I already know we're making use of it. In your SVN auth file on the server: [/mediawiki/trunk/extensions/SomeExtension] username = rw [/mediawiki/tags/extensions/SomeExtension] username = rw [/mediawiki/branches/extensions/SomeExtension] username = rw Done. Right now I imagine for extension only access there is just 3 entries like the above, but for the whole extensions directory (I'm guessing) and one for the whole repo. And I'm going to nip this one in the bud right here. Yes it is technically feasible, but it isn't a scalable process at all. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
Hi, It's great to hear that Git, once MediaWiki switches over to it, will solve some of the issues with granting access. Still, I assume no one knows exactly when the change-over will come - and I also assume that, even with Git in place, there will still be issues of access to the official repository for each set of code. So let me propose a technical solution that is hopefully fully feasible: instead of separating out access by core MediaWiki vs. extensions, separate it out as: 1) core MediaWiki, 2) extensions used on public Wikimedia wikis, and 3) all other extensions. Or the separation could even be just MediaWiki core + WMF-used extensions vs. other extensions - two sets of code, separated by whether they're used on Wikipedia and the like. The group of extensions used on WMF sites, and the group of those that aren't, are almost in two separate worlds - a bug or security hole in the former could potentially impact hundreds of millions of people, while such a bug in the latter would probably impact a few hundred or thousand people at most. If that kind of split were made, I think access to the non-WMF group could then be given almost immediately to anyone who demonstrated a basic understanding of MediaWiki coding, without any real fear of harm. The original extension under discussion, AdManager, for instance, is not going to end up anytime soon on a Wikimedia wiki, as you could guess from its name alone - so any bugs in that code are literally about a million times less important than bugs in extensions like FlaggedRevs. Any new extensions would simply be grouped into the non-Wikimedia group - the Wikimedia group would be an opt-in thing. There is a potential gray area, of extensions that aren't used in Wikipedia and the like but are used on helper wikis like translatewiki.net and the WMF infrastructure wiki - Replace Text and Semantic MediaWiki are two such extensions. Those could end up in either slot; I think it would be fine either way. This ties in to something else I've thought about for a while: that maybe the CodeReview protocol should similarly be bifurcated, so that non-WMF extensions like, say, SocialProfile, don't get the same level of scrutiny as extensions like ParserFunctions. As an extension developer, I'm grateful for all the helpful reviews that my commits get - but if all that reviewing work is coming at the expense of the reviewing of code that's used on WMF sites, then it seems like a waste of resources. Any thoughts on this idea? -Yaron -- WikiWorks · MediaWiki Consulting · http://wikiworks.com ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On 4 October 2011 22:42, Yaron Koren ya...@wikiworks.com wrote: Hi, So let me propose a technical solution that is hopefully fully feasible: instead of separating out access by core MediaWiki vs. extensions, separate it out as: 1) core MediaWiki, 2) extensions used on public Wikimedia wikis, and 3) all other extensions. Or the separation could even be just MediaWiki core + WMF-used extensions vs. other extensions - two sets of code, separated by whether they're used on Wikipedia and the like. This would be great, apart from one thing: new extensions are installed on WMF wikis on a fairly regular basis. Moving extensions between SVN repos, or even between separate branches, would be extremely disruptive right across the board, so this disctinction would at best be accurate only at the time the repository was reconfigured. At worst, we'd end up forking extensions once they were installed on the cluster, with all new development being made to the WMF version while users of the old version are left to rot, blisfully unaware that the code they are using is no longer getting any TLC. This ties in to something else I've thought about for a while: that maybe the CodeReview protocol should similarly be bifurcated, so that non-WMF extensions like, say, SocialProfile, don't get the same level of scrutiny as extensions like ParserFunctions. As an extension developer, I'm grateful for all the helpful reviews that my commits get - but if all that reviewing work is coming at the expense of the reviewing of code that's used on WMF sites, then it seems like a waste of resources. This already happens, although more by default than through design. Whole swathes of commits to extensions and branches are already automatically marked deferred in CR; usually that means that the amount of code review it will receive is nil. Our problems with code review are not due to the distraction of random extensions! --HM ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
Happy Melon - I should have clarified how I thought this should be implemented. I don't mean that any extensions should move locations - the fact that everything is in one big /extensions directory is quite helpful. I just mean that there should be some SVN settings so that only users with a certain permission level can modify the code in pre-specified directories under /extensions. I don't know much about SVN administration, but I assume that you can restrict access in that sort of semi-fine-grained way. -Yaron ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On Tue, Oct 4, 2011 at 2:42 PM, Yaron Koren ya...@wikiworks.com wrote: It's great to hear that Git, once MediaWiki switches over to it, will solve some of the issues with granting access. Still, I assume no one knows exactly when the change-over will come - Current schedule is to start doing prelim conversion tests this month -- expect some stuff to play with by the New Orleans hackathon -- and start seriously converting in November: http://www.mediawiki.org/wiki/Roadmap -- brion ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On Tue, Oct 4, 2011 at 4:29 AM, Olivier Beaton olivier.bea...@gmail.comwrote: I'm going to nip this one in the bud right away, and given that core and extension SVN access is already separate already, I already know we're making use of it. In your SVN auth file on the server: [/mediawiki/trunk/extensions/SomeExtension] username = rw [/mediawiki/tags/extensions/SomeExtension] username = rw [/mediawiki/branches/extensions/SomeExtension] username = rw Branched extensions are a bit differently handled, but as we're about to see most of the time I think we can just skip worrying about them entirely. :) I had a good talk with Olivier on IRC, and I think we understand a bit bit better where we're coming from. The big thing is to make sure that people feel like they're being fairly communicated with. I think we do have an available spot between full extensions read/write access and host your files elsewhere for now -- and that go ahead and commit to your extension directory X spot is a place we want to be to attract and retain newbie developers. (I should say, newbie members of the developer community! They may be old hands we've just never interacted with before because they've been MediaWiki *admins* and *users* out of our sight.) The git stuff should place us squarely there: it should be trivial for people to set up their own repositories to commit to that share our infrastructure, with a little less work to do to migrate things into the main community-maintained repositories. In the meantime though we're still working in SVN; people who need SVN commit access to get stuff done still need it, and we're still accepting general requests, so it's good to make sure that people know what to expect from us when they ask. I suspect the primary case that would be nice to handle goes something like this: Bob has written a MediaWiki extension BobsAwesomeExt that he'd like to share with others. He wants it to be in MediaWiki's source control system for archival purposes, making downloads easy, and to help it get future maintenance (localization, security fixes, compatibility fixes) even if Bob ends up wandering off later. If Bob ends up staying (which would be great!) then this gives him an account to start with and a visible stream in CodeReview to start interacting with other developers. In SVN land, Bob mainly needs to be able to commit into trunk/extensions/BobsAwesomeExt. In theory people could maintain version branches for extensions but in practice almost nobody does except for core contributors merging fixes so we don't really have to worry about permissions on those branch or tag directories. So it would probably not be too terrible a burden for us to stick some more one-off extensions in in the short term, just with an eye for making sure that we're capturing and keeping the code and developers who are already knocking on the door. What we *do* want to avoid is ending up with a full-blown complicated-permissions setup on the current SVN; but I don't think we'll be forced into slippery-slope territory by adding a few single-directory single-user permission grants. (In future git land, giving Bob full read-write access to the BobsAwesomeExt main repository will let him do all the branch maintenance he wants, so this would not be a permanent restriction.) Now we'll still want to make sure we can consistently manage permissions on the git repos, but we'll want to make sure we know how all that works before speccing out that setup. :) If you're not in a hurry but would like to prepare for moving an extension into our git space when it's ready: * stick your code on a public git host like github or gitorious * you'll be able to push it -- with all history -- into MediaWiki's hosted git space later -- brion ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On Mon, Oct 3, 2011 at 4:30 PM, Yaron Koren ya...@wikiworks.com wrote: I think developer accounts on the Wikimedia SVN repository should be easier to get. I was under the impression that getting one *was* easy...but admittedly things have changed quite a bit since 2008. When he requested access, this was the relevant part of the response from Sumana: Right now, we are not approving your request for commit access. I'm sorry. We'd like for you to get more practice writing code for MediaWiki, submit patches for review via Bugzilla attachments, and ask us for comments... Please come back and request access again in a few months. I don't know whether this is WMF policy now, or a personal decision from Sumana, or a decision made by someone else, but in any case I don't understand it. I don't think it was Sumana's personal decision, as I believe that a group of people decide what to do with pending commit access requests. In any case, I would have to agree with your conclusion, as I did some review and fixing on the mentioned AdManager extension. It seems to me that there are two valid reasons for not simply allowing everyone to get a developer account: the first, and major, reason is to prevent malicious users from vandalizing or deleting code. The second is to prevent well-intentioned but incompetent developers from checking in buggy and/or badly-written code that requires lots of fixes and review time by the reviewers. In both cases, the person's presence in SVN would cause more harm than good. I'm sure that we've had some incompetent developers in the past, but becoming a competent dev takes some time and work. :-) Clearly a higher bar is being applied here than what's spelled out in the mediawiki.org documentation - which only says that we don't have time to train programmers from scratch: http://www.mediawiki.org/wiki/Commit_access#Prerequisites Yeah, it certainly seems like that. If and when we want to encourage people to share their code and receive various fixes, rejecting valid commit access requests isn't the way to do that. It seems to me that if someone writes an extension that basically matches the MediaWiki guidelines, works, and does something useful, they should pretty much be granted automatic access to an account, because they will have proved that their presence will be a net positive overall. Any thoughts on this? +1 And out of curiosity - is there a new policy in place? I wouldn't know, as the process has changed over the years, but I have to say that I liked it when commit access requests were on the MediaWiki.org wiki ([[mw:Commit access requests]]) -- IMO it was a better and more transparent way to manage commit access requests than an OTRS queue or whatever is used nowadays; then again, I'm just giving suggestions here, I'm not here to make any decisions as I'm not employed by the Foundation. Thanks and regards, -- Jack Phoenix MediaWiki developer ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On 3 October 2011 18:00, Jack Phoenix j...@countervandalism.net wrote: And out of curiosity - is there a new policy in place? I wouldn't know, as the process has changed over the years, but I have to say that I liked it when commit access requests were on the MediaWiki.org wiki ([[mw:Commit access requests]]) -- IMO it was a better and more transparent way to manage commit access requests than an OTRS queue or whatever is used nowadays; then again, I'm just giving suggestions here, I'm not here to make any decisions as I'm not employed by the Foundation. The biggest problem of the old mw.org queue was that it was simply neglected for months at a time; my own commit access request was up there for over six months before *anyone* looked at it *at all*. I agree that it was more transparent and maybe 'better'; but the most important requirement of the system is that it *works* and is used. If the OTRS queue works for the current svn admins, then that's an important merit. --HM ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
Yaron Koren wrote: (...) Neither of those cases apply here - the Ad Manager code was well-written, and it works. If you're curious, you can see for yourself the kinds of fixes and changes that were made to the code after it was checked in - all minor stuff, the only major thing being that the extension originally included support for MediaWiki 1.15, which people thought was unnecessary. Clearly a higher bar is being applied here than what's spelled out in the mediawiki.org documentation - which only says that we don't have time to train programmers from scratch: http://www.mediawiki.org/wiki/Commit_access#Prerequisites Maybe the request wasn't clear pointing to the existing Ad Manager extension, already copied to our svn. Or perhaps Sumana read it too quickly and missed it. Looks it shouldn't have been rejected. Note, by the way, that if there's a more stringent policy in place now, it's not being applied consistently, because the students in this year's Google Summer of Code got developer access after much less proof of programming ability. I agree. GSoC students get accounts much easier than other people, and are generally worse developers at the beginning. OTOH, there are developers assigned to mentor them, so that may strike it out. Equally, given your vouch for Ike, I think that the application should accepted. It seems to me that if someone writes an extension that basically matches the MediaWiki guidelines, works, and does something useful, they should pretty much be granted automatic access to an account, because they will have proved that their presence will be a net positive overall. Any thoughts on this? Yes. We want developer access to be easy. We want mediawiki extensions to live in our svn as far as we can. There is even a project to use virtual machines as a development infrastructure. And extension access is not a big deal anyway. And out of curiosity - is there a new policy in place? Not that I know of. Although Sumana is probably applying some consistent rules. Review of developer requests was more random before. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On Mon, Oct 3, 2011 at 6:30 AM, Yaron Koren ya...@wikiworks.com wrote: I think developer accounts on the Wikimedia SVN repository should be easier to get. I say this because a consultant of ours at WikiWorks, Ike Hecht, asked for a developer account last week and was rejected. He created his first major MediaWiki extension, Ad Manager, recently, which I added to the repository a few weeks ago - you can see it here: [snip] Specifically as regards stand-alone extensions we've generally been much more liberal than for core -- perhaps Ike forgot to mention that this is for maintenance of an extension that's already been committed and that several people are vouching for him and his code? Generally speaking... TL;DR summary: we also want to make it easier to get involved; this is a work in progress! :) We're currently in a sort of no-mans-land in trying to make sure our policies are reasonably responsive and consistent, knowing that we're sitting just ahead of a planned migration from SVN to Git which will change the permissions landscape. In a Git world, it should become a *lot* easier to fully participate in the development ecosystem without having to get an account manually approved and created. Part of what us coders need about having a SVN account now is simply that without direct checkin ability, you can't, well, check anything in. :) You otherwise have to wait on other people to look at your code before it even gets into the system, and your commit won't even have your name on it when it gets there. :( In git-land though, we can basically eliminate most of the distinction between a submitted patch (floating around from Bugzilla, mailing list, or IRC) and something that's been committed-but-not-yet-reviewed (the scary world of CodeReview, where bad code can live on breaking other things until people figure out how to fix it months later ;). Commits will be fully labeled with the names of their creators, whether they got committed straight to core by Tim Starling himself or whether they came in as a pull request from someone who's forwarding work from someone we've never seen before. Review and fixes can happen on a custom branch -- fully versioned -- and be merged in to mainline after further commits are made to tweak them. Being able to maintain extensions as standalone git repositories will further reduce the difficulty of getting in: setting yourself up with a git account and creating repos for your extensions will become entirely self-serve (no need to ask for every individual git permission). We should end up in a better position to do both totally self-serve and semi-curated work (eg, shared maintenance of translations and security updates). -- brion ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On Mon, Oct 3, 2011 at 6:30 AM, Yaron Koren ya...@wikiworks.com wrote: I don't know whether this is WMF policy now, or a personal decision from Sumana, or a decision made by someone else, but in any case I don't understand it. It seems to me that there are two valid reasons for not simply allowing everyone to get a developer account: the first, and major, reason is to prevent malicious users from vandalizing or deleting code. The second is to prevent well-intentioned but incompetent developers from checking in buggy and/or badly-written code that requires lots of fixes and review time by the reviewers. In both cases, the person's presence in SVN would cause more harm than good. I'm going to leave it to Sumana for the more complete reply here (she and I just spoke). Sumana inherited our existing process, and she's been running it about as well as it can be run. I think she's probably been uncomfortable breaking with tradition while she's relatively new and there's a lot of stakeholders, but I think we've got some room to break with tradition. As it stands, we kinda backed into our current process. When I came on board, Tim had just moved over to OTRS. I spent some time with him understanding the process and figuring out how to speed things up, eventually pulling in other developers. Sumana took over the process at this point, which basically involves a handful of folks (Tim, Chad, and Aaron, I believe) quickly reviewing applications, and Sumana helping to dispatch responses. Since a lot of people are applying, there are quite a few to get through. I think the process could benefit from some transparency on both sides of the equation. I know from talking to her Sumana would love to have a wider pool of developers to vet the list, and a more transparent process would make that easier. Those requesting access can help things by being more transparent, too. Developers who post to the mailing list and participate in IRC will have a lot easier time getting access (assuming they say sensible things in those venues). It's a lot less scary giving access to someone when you have enough information to speculate on what they're likely to do with it, and they've proven that they play well with others. Since we don't have a process for reversing our decision (has anyone ever had their svn access revoked?), we're naturally going to be more conservative about who we let in. Anyway, as Brion says, we're likely to change things around pretty substantially when we migrate to Git, anyway, so we shouldn't spend too much time worrying about what svn access looks like for everyone. I don't want to use that as an excuse to shut down any improvement, but merely want to remind people of why we aren't likely to do anything really radical in the short term. Rob ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On Mon, Oct 3, 2011 at 6:27 PM, Rob Lanphier ro...@wikimedia.org wrote: Since a lot of people are applying, there are quite a few to get through. Out of curiosity, how many is a lot in this context? ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
It seems pretty straight forward to me, #1 Core access? Keep existing review process in place, could take 6 months, sorry. Patches that will never get committed for you! #2 Extension access? Has a working extension attached to submission (has some .php files in it, no need for a code review here, the community will handle that) . Granted. Create their account, and give them access to _only_ their extension directory. If you want to foster a more communal and helping extension community, when it comes to do core access reviews, use a script to spit out active people on svn, and if any of them only have access to their extension directory, pro-actively have a short sentence or two about if you should give them access to all extensions. If yes, let them know the happy news, maybe they will do something with it (likely not, even if they had it in the first place). Another metric might be if they have a lot of tickets for other core extensions with patches attached. They can't be that destructive in their own space, and if they have a complete working extension, chances are people in the community will start using it, and invariably come into irc complaining if the quality is low, better to nip this one in the bud and get them into the code review process early. Like all the people being so helpful to me in IRC. I don't want to wait months to find out I've been doing things all wrong. For extension access there isn't many reasons why you can't make them an account on the spot and see how it goes, whoever is around and sees it first (even Sumana). I haven't used git, but won't you need to decide what gets comitted in git as well, no matter what they comit on their own HD? It will still be 'get it moved into a extension branch on git, have your revisions reviewed, get it into the extension distributor'. I see this applying no matter which way you go. Just my 2c to the pile, now banking a dollar. - Olivier ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
Caution: long! On 10/03/2011 09:30 AM, Yaron Koren wrote: Hi, I think developer accounts on the Wikimedia SVN repository should be easier to get. I agree. I say this because a consultant of ours at WikiWorks, Ike Hecht, asked for a developer account last week and was rejected. He created his first major MediaWiki extension, Ad Manager, recently, which I added to the repository a few weeks ago - you can see it here: http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/AdManager/ When he requested access, this was the relevant part of the response from Sumana: Right now, we are not approving your request for commit access. I'm sorry. We'd like for you to get more practice writing code for MediaWiki, submit patches for review via Bugzilla attachments, and ask us for comments... Please come back and request access again in a few months. I checked with Ike to ensure that he was okay with us discussing his rejection in public -- he hadn't known, but said it was okay. So I'm pasting here in full the response I wrote to Isaac, after having Rob, Chad, Tim, and Aaron review his request and code: Thank you for requesting commit access, and my apologies on the delay in response. (Developers have had their attention taken by issues around the deployment of MediaWiki 1.18 on a bunch of wikis.) Right now, we are not approving your request for commit access. I'm sorry. We'd like for you to get more practice writing code for MediaWiki, submit patches for review via Bugzilla attachments, and ask us for comments. Please read http://www.mediawiki.org/wiki/Manual:Coding_conventions http://www.mediawiki.org/wiki/Security_for_developers Please come back and request access again in a few months. Thank you for being part of the MediaWiki community, and please feel free to find us in the #mediawiki channel on FreeNode IRC for more detailed feedback. The bits that were left out of the excerpt, but that I do think are relevant: I pointed him to further reading, to help him improve his code quality, and suggested that he talk with us on IRC for more detailed feedback, again to help improve his code quality. In general, this is the kind of rejection notice I send, when Chad, Tim, and Aaron decide to reject an applicant. If an applicant has SQL injection or cross-site scripting issues in their code sample, then I specifically advise them about that using a templated text. I would like to provide more constructive and specific feedback, but since that would demand more time per request and we're usually backlogged, I've chosen speed of response over length, and suggested to rejected applicants that they ask for more feedback if they want it. About a third do. I don't know whether this is WMF policy now, or a personal decision from Sumana, or a decision made by someone else As others in this thread pointed out before I got to it -- I'm not the one making the decisions on whether to grant access. Right now it's Chad, Tim, and Aaron. but in any case I don't understand it. It seems to me that there are two valid reasons for not simply allowing everyone to get a developer account: the first, and major, reason is to prevent malicious users from vandalizing or deleting code. The second is to prevent well-intentioned but incompetent developers from checking in buggy and/or badly-written code that requires lots of fixes and review time by the reviewers. In both cases, the person's presence in SVN would cause more harm than good. That second point sometimes doesn't get enough attention: yes, when we grant commit access, we are committing to keeping up with post-commit review of that person's code. If the person looks likely to make a lot of mistakes, that adds to our load pretty substantially. And so reviewers get a bit fearful about saying yes. (This of course ties into our ongoing exhortations to code reviewers to be revert-happy, but that's a little beyond the scope of this email.) Neither of those cases apply here - the Ad Manager code was well-written, and it works. If you're curious, you can see for yourself the kinds of fixes and changes that were made to the code after it was checked in - all minor stuff, the only major thing being that the extension originally included support for MediaWiki 1.15, which people thought was unnecessary. I believe the reviewers' opinion of this code differed markedly from Yaron's, which is the reason for the rejection. Clearly a higher bar is being applied here than what's spelled out in the mediawiki.org documentation - which only says that we don't have time to train programmers from scratch: http://www.mediawiki.org/wiki/Commit_access#Prerequisites Yes, and that's wrong. Once we straighten out what the actual criteria should be (and I suspect they'll include a division between core and extensions access), I'll want to update that page. Note, by the way, that if there's a more stringent policy in place
Re: [Wikitech-l] Making developer access easier to get
On 10/03/2011 01:48 PM, Happy Melon wrote: On 3 October 2011 18:00, Jack Phoenix j...@countervandalism.net wrote: ... I liked it when commit access requests were on the MediaWiki.org wiki ([[mw:Commit access requests]]) -- IMO it was a better and more transparent way to manage commit access requests than an OTRS queue or whatever is used nowadays; then again, I'm just giving suggestions here, I'm not here to make any decisions as I'm not employed by the Foundation. The biggest problem of the old mw.org queue was that it was simply neglected for months at a time; my own commit access request was up there for over six months before *anyone* looked at it *at all*. I agree that it was more transparent and maybe 'better'; but the most important requirement of the system is that it *works* and is used. If the OTRS queue works for the current svn admins, then that's an important merit. --HM The current system gets applicants responses usually within a week, sometimes two or three weeks. During the old system it was sometimes months. Right now, we're backlogged, mostly because we skipped one of the weekly commit access queue review meetings during the 1.18 deploy crunch. I think another reason the current system works is because that weekly meeting is only 15-20 minutes, so admins consistently attend them. :-) Almost all of that time is code review and discussion of the candidate, because I try to go into the queue ahead of time and look for incomplete applications, ask for code samples, ask for ssh keys, and take care of other administrivia. So if we implement my proposal and you participate in one of these meetings, your time won't be wasted. -- Sumana Harihareswara Volunteer Development Coordinator Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On 10/03/2011 05:18 PM, Brion Vibber wrote: On Mon, Oct 3, 2011 at 6:30 AM, Yaron Koren ya...@wikiworks.com wrote: I think developer accounts on the Wikimedia SVN repository should be easier to get. I say this because a consultant of ours at WikiWorks, Ike Hecht, asked for a developer account last week and was rejected. He created his first major MediaWiki extension, Ad Manager, recently, which I added to the repository a few weeks ago - you can see it here: [snip] Specifically as regards stand-alone extensions we've generally been much more liberal than for core -- perhaps Ike forgot to mention that this is for maintenance of an extension that's already been committed and that several people are vouching for him and his code? He mentioned the ad manager extension, and linked to it, but did not mention that others were vouching for him. Generally speaking... TL;DR summary: we also want to make it easier to get involved; this is a work in progress! :) We're currently in a sort of no-mans-land in trying to make sure our policies are reasonably responsive and consistent, knowing that we're sitting just ahead of a planned migration from SVN to Git which will change the permissions landscape. In a Git world, it should become a *lot* easier to fully participate in the development ecosystem without having to get an account manually approved and created. Part of what us coders need about having a SVN account now is simply that without direct checkin ability, you can't, well, check anything in. :) You otherwise have to wait on other people to look at your code before it even gets into the system, and your commit won't even have your name on it when it gets there. :( In git-land though, we can basically eliminate most of the distinction between a submitted patch (floating around from Bugzilla, mailing list, or IRC) and something that's been committed-but-not-yet-reviewed (the scary world of CodeReview, where bad code can live on breaking other things until people figure out how to fix it months later ;). Commits will be fully labeled with the names of their creators, whether they got committed straight to core by Tim Starling himself or whether they came in as a pull request from someone who's forwarding work from someone we've never seen before. Review and fixes can happen on a custom branch -- fully versioned -- and be merged in to mainline after further commits are made to tweak them. Being able to maintain extensions as standalone git repositories will further reduce the difficulty of getting in: setting yourself up with a git account and creating repos for your extensions will become entirely self-serve (no need to ask for every individual git permission). We should end up in a better position to do both totally self-serve and semi-curated work (eg, shared maintenance of translations and security updates). -- brion I am looking forward to this. But in the meantime, let's make sure we clarify our ruleset. -- Sumana Harihareswara Volunteer Development Coordinator Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Making developer access easier to get
On 10/03/2011 07:19 PM, Benjamin Lees wrote: On Mon, Oct 3, 2011 at 6:27 PM, Rob Lanphier ro...@wikimedia.org wrote: Since a lot of people are applying, there are quite a few to get through. Out of curiosity, how many is a lot in this context? From viewing the OTRS queue archives: 25 requests in the last 40 days. That's about 4.3 applications per week. This is up from about 29 in the last 60 days (3.4 apps/week), and 82 in the past 180 days (3.2 apps/week, also the rate over the entire 20 months we've been using the OTRS queue). I believe all these stats exclude spam. You see why I'd like more developers reviewing commit access requests; the more requests we get, the more time the code review takes, and I'd like to spread that out more -- both as a TODO and as a learning opportunity. And -- yay for our community that we're getting more frequent commit access requests! Along with the line on http://toolserver.org/~robla/crstats/crstats.trunkall.html going down, it's a sign of our development community's health and momentum. -- Sumana Harihareswara Volunteer Development Coordinator Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
