Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-06-05 Thread Keisial
Daniel Friesen wrote: I wanted to use runkit but had issues installing it. So I ended up messing with php's horrid proc_open to sandbox it in another process to act as the vm in the case my system needs to extract info from the wiki (not for virtualizing the actual wiki, that is done

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-06-04 Thread Happy-melon
Dmitriy Sintsov ques...@rambler.ru wrote in message news:1006208056.1275619880.71836632.61...@mcgi66.rambler.ru... * Happy-melon happy-me...@live.com [Fri, 4 Jun 2010 00:33:30 +0100]: One way to achieve this would be to develop the MediaWiki class to actually be what it originally

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-06-04 Thread Dmitriy Sintsov
* Happy-melon happy-me...@live.com [Fri, 4 Jun 2010 10:03:14 +0100]: Dmitriy Sintsov ques...@rambler.ru wrote in message news:1006208056.1275619880.71836632.61...@mcgi66.rambler.ru... * Happy-melon happy-me...@live.com [Fri, 4 Jun 2010 00:33:30 +0100]: One way to achieve this would be

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-06-04 Thread Happy-melon
-- From: Dmitriy Sintsov ques...@rambler.ru Sent: Friday, June 04, 2010 11:01 AM To: Happy-melon happy-me...@live.com; Wikimedia developers wikitech-l@lists.wikimedia.org Subject: Re: [Wikitech-l] Reasonably efficient interwiki transclusion

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-06-04 Thread Happy-melon
-- From: Dmitriy Sintsov ques...@rambler.ru Sent: Friday, June 04, 2010 11:35 AM To: Happy-melon happy-me...@live.com Subject: Re: [Wikitech-l] Reasonably efficient interwiki transclusion * Happy-melon happy-me...@live.com [Fri, 4 Jun 2010 11:10

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-06-03 Thread Dmitriy Sintsov
* Roan Kattouw roan.katt...@gmail.com [Tue, 25 May 2010 20:58:54 +0200]: 2010/5/25 Aryeh Gregor simetrical+wikil...@gmail.com: Having Wikimedia servers send HTTP requests to each other instead of just doing database queries does not sound like a great idea to me. You're hitting several

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-06-03 Thread Daniel Friesen
Dmitriy Sintsov wrote: * Roan Kattouw roan.katt...@gmail.com [Tue, 25 May 2010 20:58:54 +0200]: 2010/5/25 Aryeh Gregor simetrical+wikil...@gmail.com: Having Wikimedia servers send HTTP requests to each other instead of just doing database queries does not sound like a great idea to

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-06-03 Thread Platonides
Daniel Friesen wrote: ^_^ hackish isn't that bad in some sense. I'm currently experimenting with some farm code that works completely outside of MediaWiki rather than as a extension sitting inside of it. Using a sandbox it can get access to the MediaWiki install and extract info from it in

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-06-03 Thread Happy-melon
Dmitriy Sintsov ques...@rambler.ru wrote in message news:830714463.1275562997.168145444.10...@mcgi21.rambler.ru... * Roan Kattouw roan.katt...@gmail.com [Tue, 25 May 2010 20:58:54 +0200] Having a something like FarmRequest or FarmApi classes would be a great think for wiki farms (I run a

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-06-03 Thread Daniel Friesen
Platonides wrote: Daniel Friesen wrote: ^_^ hackish isn't that bad in some sense. I'm currently experimenting with some farm code that works completely outside of MediaWiki rather than as a extension sitting inside of it. Using a sandbox it can get access to the MediaWiki install and

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-06-03 Thread Dmitriy Sintsov
* Happy-melon happy-me...@live.com [Fri, 4 Jun 2010 00:33:30 +0100]: One way to achieve this would be to develop the MediaWiki class to actually be what it originally promised: an object representing a wiki, of which there can in principle be more than one instantiated at any one time.

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-28 Thread Peter17
I have updated my proposal with a fourth version [1] I am still waiting for comments from Tim Starling. I have contacted him on IRC for this. [1] http://www.mediawiki.org/wiki/User:Peter17/Reasonably_efficient_interwiki_transclusion#Fourth_version_.(to_be_discussed) -- Peter Potrowl

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-26 Thread Alex Brollo
@peter: here a recent thread into MediaWiki-API ml about API and sections: http://lists.wikimedia.org/pipermail/mediawiki-api/2010-May/subject.html No mention of labelled sections used by #lst exstesion ... :-( but remember the name of ThomasV as a reference. Alex

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-26 Thread Platonides
Peter17 wrote: I didn't set $wgUploadPath. Just $wgUseInstantCommons = true; The images URLs are actually transformed to remote URLs: I work on my own local wiki, which address is http://localhost/mediawiki/ and transcluding {{mediawikiwiki::User:Peter17}} which contains

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Peter17
2010/5/25 Platonides platoni...@gmail.com: Seems it doesn't work so well. It was inadvertedly broken for wikitext transclusions when the interwiki points to the nice url. See 'wgEnableScaryTranscluding and Templates/Images?' thread at mediawiki-l Well, in my tests, images are well included

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Alex Brollo
About the question from Alex about transcluding sections: is it possible to request only a section through the API? I searched about this but didn't find :( -- Peter Potrowl Ask ThomasV, #lst is particularly cared by him, to deepest level of knowledge! I guess he met too your same

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Chad
On Tue, May 25, 2010 at 7:41 AM, Peter17 peter...@gmail.com wrote: Mmmh sorry, I'm not really sure I understand... My suggestion is to use a shared database that would store the remote calls, not the content of the pages... In my mind, fetching the distant pages would be done through the

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Platonides
Aryeh Gregor wrote: On Mon, May 24, 2010 at 8:27 PM, Q overlo...@gmail.com wrote: I would have to suggest to not go the shared database route unless the code can be fixed so that shared databases actually work with all of the DB backends. I don't see why it shouldn't be easy to get it

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Platonides
church.of.emacs.ml wrote: However, you'd have to worry that each distant wiki uses only a fair amount of the home wiki server's resources. E.g. set a limit of inclusions (that limit would have to be on the home-wiki-server-side) and disallow infinite loops (they're always fun). Infinite loops

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Tei
On 25 May 2010 15:30, Platonides platoni...@gmail.com wrote: church.of.emacs.ml wrote: However, you'd have to worry that each distant wiki uses only a fair amount of the home wiki server's resources. E.g. set a limit of inclusions (that limit would have to be on the home-wiki-server-side) and

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Aryeh Gregor
(I'm going to use local wiki here for what Peter is calling distant wiki, and foreign wiki for what he's calling home wiki. This seems to better match the terminology we use for Commons.) On Tue, May 25, 2010 at 7:41 AM, Peter17 peter...@gmail.com wrote: Yes. The shared database would be only

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Roan Kattouw
2010/5/25 Aryeh Gregor simetrical+wikil...@gmail.com: Having Wikimedia servers send HTTP requests to each other instead of just doing database queries does not sound like a great idea to me. You're hitting several extra servers for no reason, including extra requests to an application server.  

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Aryeh Gregor
On Tue, May 25, 2010 at 2:58 PM, Roan Kattouw roan.katt...@gmail.com wrote: This is true if, indeed, all parsing is done on the distant wiki. However, if parsing is done on the home wiki, you're not simply requesting data that's ready-baked in the DB and API calls make sense. That's true -- if

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Roan Kattouw
2010/5/25 Aryeh Gregor simetrical+wikil...@gmail.com: Templates will often miss the parser cache, because different invocations will use different parameters.  Even *with* the parser cache, parsing is *still* one of the most expensive operations Wikimedia does, so I'm not so sanguine. I

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Marco Schuster
On Tue, May 25, 2010 at 8:58 PM, Roan Kattouw roan.katt...@gmail.comwrote: To the point of whether parsing on the on the distant wiki makes more sense: I guess there are points to be made both ways. I originally subscribed to the idea of parsing on the home wiki so expanding the same template

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Aryeh Gregor
On Tue, May 25, 2010 at 3:48 PM, Roan Kattouw roan.katt...@gmail.com wrote: Also note that you wouldn't technically be parsing, just preprocessing on the home wiki, which is certain to be less expensive (how much less I don't know) This is a good point. and that you'd be doing this on some

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Platonides
Aryeh Gregor wrote: Ok, I will keep this in mind. Parsing the template on the home wiki seems necessary because it can use other templates hosted on that wiki to render correctly... I think it is the most logical way to do, isn't it? I think parsing the template on the local wiki is better,

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Chad
On Tue, May 25, 2010 at 5:50 PM, Platonides platoni...@gmail.com wrote: But I guess that's much better handled by just using a proper export, and having the templates included in that, so never mind. Yes. Perhaps they could have a Special:ImportFromRemote to do one-click imports. And this

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Aryeh Gregor
On Tue, May 25, 2010 at 5:50 PM, Platonides platoni...@gmail.com wrote: There are imho fewer variables set by the caller wiki, which could be passed with the query. I don't get what you're saying here. For intra-Wikimedia query, they could directly ask an apache. They can even send the query

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Jim Tittsler
On 2010-05-25 23:41, Peter17 wrote: 2010/5/25 Platonides platoni...@gmail.com: Seems it doesn't work so well. It was inadvertedly broken for wikitext transclusions when the interwiki points to the nice url. See 'wgEnableScaryTranscluding and Templates/Images?' thread at mediawiki-l Well,

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-25 Thread Peter17
2010/5/26 Jim Tittsler j...@onnz.net: On 2010-05-25 23:41, Peter17 wrote: 2010/5/25 Platonides platoni...@gmail.com: Seems it doesn't work so well. It was inadvertedly broken for wikitext transclusions when the interwiki points to the nice url. See 'wgEnableScaryTranscluding and

[Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-24 Thread Peter17
Hello to all! I'm a French student and I am participating the Google Summer of Code this year on Mediawiki! My mentor is Roan Kattouw (Catrope) and my subject is Reasonably efficient interwiki transclusion. You can see my application page here: [1]. I have already discussed with my mentor and

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-24 Thread Alex Brollo
2010/5/24 Peter17 peter...@gmail.com Hello to all! I'm a French student and I am participating the Google Summer of Code this year on Mediawiki! My mentor is Roan Kattouw (Catrope) and my subject is Reasonably efficient interwiki transclusion. You can see my application page here: [1].

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-24 Thread Amir E. Aharoni
On Mon, May 24, 2010 at 17:44, Peter17 peter...@gmail.com wrote: Hello to all! I'm a French student and I am participating the Google Summer of Code this year on Mediawiki! My mentor is Roan Kattouw (Catrope) and my subject is Reasonably efficient interwiki transclusion. You can see my

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-24 Thread Chad
On Mon, May 24, 2010 at 11:18 AM, Amir E. Aharoni amir.ahar...@mail.huji.ac.il wrote: On Mon, May 24, 2010 at 17:44, Peter17 peter...@gmail.com wrote: Hello to all! I'm a French student and I am participating the Google Summer of Code this year on Mediawiki! My mentor is Roan Kattouw

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-24 Thread Amir E. Aharoni
On Mon, May 24, 2010 at 18:48, Chad innocentkil...@gmail.com wrote: On Mon, May 24, 2010 at 11:18 AM, Amir E. Aharoni amir.ahar...@mail.huji.ac.il wrote: The title of the subject is a bit confusing. Interwiki, for better or worse, refers to interlanguage links. No it doesn't. Interwiki links

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-24 Thread Platonides
http://www.mediawiki.org/wiki/User:Peter17/Reasonably_efficient_interwiki_transclusion#Good_points Seems it doesn't work so well. It was inadvertedly broken for wikitext transclusions when the interwiki points to the nice url. See 'wgEnableScaryTranscluding and Templates/Images?' thread at

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-24 Thread church.of.emacs.ml
Hi, On 05/24/2010 04:44 PM, Peter17 wrote: It is now open for comments, so, could you please read it and let me know about your remarks and suggestions, on this list and/or on the talk page? first of all, I let me tell you that I'm really excited about this project. It may very well

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-24 Thread Chad
On Mon, May 24, 2010 at 7:42 PM, church.of.emacs.ml church.of.emacs...@googlemail.com wrote: Hi, On 05/24/2010 04:44 PM, Peter17 wrote: It is now open for comments, so, could you please read it and let me know about your remarks and suggestions, on this list and/or on the talk page? first

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-24 Thread Q
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 5/24/2010 6:42 PM, church.of.emacs.ml wrote: Hi, On 05/24/2010 04:44 PM, Peter17 wrote: It is now open for comments, so, could you please read it and let me know about your remarks and suggestions, on this list and/or on the talk page?

Re: [Wikitech-l] Reasonably efficient interwiki transclusion

2010-05-24 Thread Aryeh Gregor
On Mon, May 24, 2010 at 8:27 PM, Q overlo...@gmail.com wrote: I would have to suggest to not go the shared database route unless the code can be fixed so that shared databases actually work with all of the DB backends. I don't see why it shouldn't be easy to get it working with all DB