Re: [Wikitech-l] On third-party users of MediaWiki and collaboration with upstream
On Wed, Aug 17, 2011 at 2:07 PM, Jack Phoenix j...@countervandalism.net wrote: What's common to third-party users of MediaWiki is not just custom extensions, but sadly core changes, or as they're better known, core hacks -- unsupported changes to the core of the MediaWiki software. I think that everyone will agree with me when I say that we will want to reduce the amount of core hacking by third-parties and instead increase collaboration with us, the upstream developers of MediaWiki. Essentially I'd like to see all major third-party users contributing code to the upstream version of MediaWiki and everyone keeping their copies of MediaWiki on the official MediaWiki Subversion repository at svn.wikimedia.org Jack Phoenix MediaWiki developer I've been reading this discussion with interest and thought a few updates/links might be useful: Sam Reed has put together a list of the Wikia diffs that need to be upstreamed[0], and Sean Colombo's working on it. Markus Glaser is coming to the upcoming New Orleans hackathon[1] and will chat about third party development upstream to MediaWiki SVN, among other issues. Olivier Finlay Beaton put together a survey of what versions of MediaWiki are being run out in the wild[2] and found that the majority are on 1.16.x. Some administrators stay on old versions because they have extension compatibility issues, or because they don't know or don't care that new versions are available. We should address those obstacles in other ways. But to the extent that administrators don't upgrade because of their core hacks -- that's the friction that Sean, Jack, Reuben, Joachim, and Markus can especially help us reduce. From outside our immediate community, an essay on the cost of *not* upstreaming[3]. [0] http://www.mediawiki.org/wiki/Wikia_code [1] http://mediawiki.org/wiki/NOLA_Hackathon [2] http://www.mediawiki.org/wiki/Manual:Extension_support#What.27s_out_there [3] http://blogs.gnome.org/bolsh/2011/09/01/the-cost-of-going-it-alone/ Sumana Harihareswara Volunteer Development Coordinator Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On third-party users of MediaWiki and collaboration with upstream
Hi Joachim, thank you for your message. I agree that it'd be brilliant if third-party users wouldn't have to branch MediaWiki for their code! Hiding some things behind a configuration variable, like how Markus suggested, is certainly doable, to an extent. However, there will be things that need to be rethought and maybe even completely rewritten; patches like http://trac.wikia-code.com/changeset/11900/wikia/trunk/includes/api come to mind. Likewise, both Wikia and wikiHow have some custom methods and member variables in some core files, such as includes/User.php; some can be moved to a custom class and/or done via hooks, but there are some things that I'm not too sure of, such as wikiHow's getBotIDs() method in includes/User.php, for example. I actually sent this message to wikitech-l and CC'd various third-party developers on it; I don't know why your message isn't showing up in the wikitech-l archives ( http://lists.wikimedia.org/pipermail/wikitech-l/2011-August/thread.html), though. Like K. Peachey mentioned earlier, using SVN's external property might be a good way of including libraries that we don't want to host on svn.wikimedia.org for one reason or another. I'd like to know more about frameworks used by third parties and why you use them. If I recall correctly, wikiHow uses Prototype for some of their JavaScript code, but they're moving towards jQuery. Wikia has various different frameworks (http://trac.wikia-code.com/browser/wikia/trunk/lib) on their SVN, such as phpFlickr, PHPUnit, Stomp and more. One interesting point to consider is the edge case of having to edit library code for some reason. For example, you might need to fix a PHP notice (like what was done in http://trac.wikia-code.com/changeset?new=40808@wikia%2Ftrunk%2Flib%2FHTTPold=33163@wikia%2Ftrunk%2Flib%2FHTTP) or even a fatal or something like that. What'd be the best solution in such case? I think that submitting a patch to the upstream developers of the library would be the best approach, obviously, but it might be that some libraries aren't even maintained anymore or the maintainers do not have very much time...maybe then we should note somewhere that you need to patch the code. Let's say that we have an extension called FooBar in svn.wikimedia.org/mediawiki/trunk/extensions/FooBar. It has an external dependency on a library called Baz, which is hosted on Google Code. However, line 15 of Baz's BazClass.php contains code that throws a notice on modern PHP versions. In this case, I'd keep the Baz library as the svn:external of the FooBar extension but I'd add a README to the extension directory, to explain what needs patching in the external library and why and how to apply the patch. On Fri, Aug 26, 2011 at 7:08 PM, Joachim Bode joachim.b...@twoonix.comwrote: Hi Jack, hi all, I think it is indeed a good idea to provide a possibility for 3rd party devs to store branches at /mediawiki/branches/. To me the attempt of creating a way to add additional hooks *without* having to branch, that is tied to this question, is even more important. I like Markus' idea of a $wgExecuteExpensiveHooks Setting. How about CC-ing the wikitech-l list for this? We use GIT internally, but I don't Know about an option that allows for hooking a remote repository. There are hooks, but afaik - like in mw - only to call e.g. pre and post commit operations. But I doubt that would make us get rid of the legal aspects anyway. For inclusion of frameworks I would prefer providing a framework management that integrates them in one single location for the whole installation. This would check for version inconsistencies especially for JS fws and make sure that the same fw is not stored inside multiple extensions which does not work for different JS fw versions anyway. All the best, Achim Sent from my iPhone Thanks and regards, -- Jack Phoenix MediaWiki developer ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On third-party users of MediaWiki and collaboration with upstream
Hello, at Hallo Welt!, we are currently discussing some similar issues. The question is how deep we should and could tie in with mediawiki svn in general. Here are some things that came to my mind: Changes to core: While BlueSpice is essentially a MediaWiki extension and has a policy of accepting MW core as it is, we do have a handful core hacks where necessary. I doubt, though, that these will make it into core. For example, we use read protection of pages. A list of changes to core code that are necessary in this respect can already be found on mediawiki.org, however, they are not permanently implemented, I assume because of performance questions. Similar issues arise when you want to use real names instead of usernames. Here we'd have to hook at places that seem to be rather performance critical. Having said this, I admit, I never tried to pace those hooks in core. You might call that preemptive policy compliance... or just laziness ;). So the question is, can there be 'flavors' of MediaWiki that do not have to consider the requirements of a high performance high traffic environment? I assume that flavoring MW would get rather complicated. But we could introduce hooks that are only executed if, say, $wgExecuteExpensiveHooks is set. Another question is about extensions: I would very much like to see BlueSpice in mediawiki svn. There are some difficulties, though. One of them is that we use large third party frameworks and I am not sure I want to maintain them in mediawiki svn (e.g. TinyMCE or ExtJs), nor if I am allowed to do so legally. These could be left out, but then the extension would not be in a working condition when downloaded. Another issue is that we want to maintain release versions of the extension and backport some bug- and security fixes. That basically means branching. And afaik we cannot do that in mediawiki svn. I faintly remember having heard about an architecture, where you can kind of hook a remote repository into some subversion repository. That might be a solution here, since we could keep our own repo and branches while making sure there is an upstream of our trunk version. Actually, this sounds very much like DVCS, but I admit I don't know enough about git and others to know if that would solve this problem. I assume, though, that e.g. Semantic MediaWiki has some similar versioning issues, so I would be very interested to hear their thoughts on that. Cheers, Markus (mglaser) Von: Jack Phoenix [mailto:j...@countervandalism.net] Gesendet: Mittwoch, 17. August 2011 20:07 An: Wikimedia developers; Sean Colombo; Jack Herrick; reu...@wikihow.com; joachim.b...@twoonix.com; Markus Glaser Betreff: On third-party users of MediaWiki and collaboration with upstream Hi all, while MediaWiki has been and is developed primarily with Wikimedia Foundation's interests in mind, there are some big third-party users of MediaWiki out there; while Wikia and wikiHow are the biggest and most well-known, they certainly aren't the only ones. What's common to third-party users of MediaWiki is not just custom extensions, but sadly core changes, or as they're better known, core hacks -- unsupported changes to the core of the MediaWiki software. I think that everyone will agree with me when I say that we will want to reduce the amount of core hacking by third-parties and instead increase collaboration with us, the upstream developers of MediaWiki. Reducing the amount of core hacks is generally a good idea for third parties, because it will allow them to upgrade to the latest stable version of MediaWiki easily and things like new hooks can and in many cases are useful to other users of MediaWiki. For example, the MakeGlobalVariablesScript hook (http://www.mediawiki.org/wiki/Manual:Hooks/MakeGlobalVariablesScript) was originally introduced by Wikia (under the name 'ExtendJSGlobalVars'); in r38397 I added the hook into core under its current name and right now there are many extensions using the hook, including ones used by Wikimedia Foundation sites (see http://www.mediawiki.org/wiki/Category:MakeGlobalVariablesScript_extensions). This is a fine example of how a third-party core hack became a part of the MediaWiki core and thus something useful to other users of MediaWiki, including the Wikimedia Foundation. Another factor to take into account is security. According to the Version lifecycle page on MediaWiki.org (see http://www.mediawiki.org/wiki/Version_lifecycle), The release manager has also issued a strong recommendation that versions not listed above as current version or legacy version should not be used in a productive environment. They may contain critical security vulnerabilities and other major bugs, including the threat of possible data loss and/or corruption. For example, wikiHow is running MediaWiki 1.12.0, which was released on 21 March 2008 -- over three years ago. While I'm sure that the wikiHow developers have applied plenty
