On 15/03/11 13:34, Dan Bolser wrote: > Anyone know / guess why a) the limit is 20, b) the query goes awry at the > limit?
MediaWiki has a global restriction on the number of parser functions that may be used on one page. Maybe this or a similar MW "emergency break" could be a reason for this limitation. I am not sure if the current embedding limit in SMW is still needed, since there are now a number of additional global controls that prevent arbitrary multiplication of transclusion in such queries (the concern originally was not transclusion performance but multiplication of the same embedded page due to its appearance in a great number of results). It might be that it would be safe to increase the limit by default. Cheers, Markus > > On 11 March 2011 01:16, Dan Bolser<dan.bol...@gmail.com> wrote: >> Result: Recursion here isn't terrible for performance. >> >> Details: >> >> Yaron kindly set "SMWResultPrinter::$maxRecursionDepth = 40" on >> http://scratchpad.referata.com and I did some testing. >> >> For some reason, the level of recursion now stops at level 20 (??). >> I'm doing everything as before: >> * http://scratchpad.referata.com/wiki/Piston >> * http://scratchpad.referata.com/wiki/Template:Part_of_what >> >> Pages higher in the tree don't hit this limit, and the template >> 'terminates' correctly, e.g. >> * http://scratchpad.referata.com/wiki/Foo >> >> I benchmarked the performance hit due to recursion by requesting each >> page in the tree 20 times using curl, and parsing out the '<!-- Served >> in 0.451 secs. -->' string in the returned URL. Every page request was >> done with the 'action=purge' CGI setting. >> >> The results are quite interesting, suggesting that each recursion adds >> 0.02 seconds to the 'serve time' on scratchpad.referata.com: >> * http://imagepaste.nullnetwork.net/img/1299805257test_benchmark.png >> * http://oi56.tinypic.com/2gx41oy.jpg # duplicate upload >> >> >> About the above plot, upper and lower triangles are +/- s.d. of the >> mean load time (squares) in 20 trials. After depth 20 the behaviour >> changes because the template doesn't deal well with this limit >> condition and a large (null) results set is generated each time (see >> the bottom of the 'Piston' page linked above). The 'solid' regression >> includes just the points up to and including depth 20, the 'dashed' >> regression includes the points above 20. >> >> >> Cheers, >> Dan. >> >> On 10 March 2011 13:54, Dan Bolser<dan.bol...@gmail.com> wrote: >>> On 10 March 2011 12:27, Alex M. Hendler<amh.pub...@ontolawgy.com> wrote: >>>> Hi Dan, >>>> >>>> I am not sure what "your own risk!" means, but whenever I see an >>>> exclamation point in the body of a comment that is not telling you how >>>> super-cool a feature is, I get a bit scared. >>> >>> I don't see that online here: >>> http://semantic-mediawiki.org/doc/SMW__QueryPrinter_8php-source.html >>> >>> Perhaps it's a version issue, but in any case, I take your point. >>> >>> >>>> That said, I can foresee a problem if you set recursion depth too high >>>> and someone made an error such that a page that gets returned by the >>>> query is "part of" the querying page, then you just end up with loopy >>>> (literally) query results. >>> >>> Yeah, of course a page shouldn't be 'part of' itself, but there is >>> nothing to stop a user doing that. >>> >>> >>>> As for performance, in my experience (still yet to be confirmed by >>>> others) maxRecursionDepth> 2 may cause odd property inheritance >>>> behavior if you embed or transclude pages. See >>>> https://bugzilla.wikimedia.org/show_bug.cgi?id=27445 >>> >>> Weird bug! Seems to be a 'feature' of the embed format only with nested >>> queries? >>> >>> >>>> I hope this helps point you in the right direction in terms of doing >>>> what you need to do (it looks like maybe a template with known levels of >>>> properties, e.g., [[-Part of::x]] passed through sufficient levels of >>>> arrays/arraymaps and #if: statements would be a good bet); hopefully >>> >>> Yeah, I don't have a fixed number of levels. >>> >>> >>>> someone else will be able to better answer your performance question. >>>> >>>> As for SemanticTreeView, it looks like it must be installed with Halo, >>>> but the Ontoprise folks may have a different view of that. >>> >>> >>> Did you get this from the docs, or have you tried installing? >>> >>> Thanks for your help, >>> Dan. >>> >>>> -Alex >>>> >>>> >>>> - >>>> ----Original Message----- >>>> From: Dan Bolser<dan.bol...@gmail.com> >>>> To: Alex M. Hendler<amh.pub...@ontolawgy.com> >>>> Cc: Semantic MediaWiki Users<semediawiki-u...@lists.sourceforge.net> >>>> Subject: Re: [Semediawiki-user] Recursive ask using format=template, >>>> stuck at depth of 3? >>>> Date: Thu, 10 Mar 2011 10:00:32 +0000 >>>> >>>> Thanks Alex. >>>> >>>> I read the document you suggested: >>>> http://semantic-mediawiki.org/doc/SMW__QueryPrinter_8php-source.html >>>> >>>> but I expected to see why the settings are what they are (i.e. why so >>>> low), which I don't see. >>>> >>>> Do you think those 'odd' issues that you mention would to this simple >>>> recursive query? Perhaps I'm being naive, but it seems to work so well >>>> up to the given depth... would performance degrade super-linearly with >>>> increased depth? >>>> >>>> >>>> Alternatively, the Semantic TreeView extension looks promising: >>>> >>>> * >>>> http://smwforum.ontoprise.com/smwforum/index.php/Help:Description_of_the_tree_parser_function >>>> * >>>> http://smwforum.ontoprise.com/smwforum/index.php/Help:Description_of_the_generateTree_parser_function >>>> >>>> >>>> but it isn't clear from the documentation if you need to install halo >>>> or not (some places state halo as a requirement, while other >>>> information suggests that it isn't) ... I guess I'll just suck it and >>>> see... The biggest problem is that it expects a tree while I have a >>>> DAG... Not sure what that'll do to it. >>>> >>>> Any other suggested extensions for navigating relations? (Other than >>>> Special:Browse of course!) >>>> >>>> >>>> Cheers, >>>> Dan. >>>> >>>> On 10 March 2011 07:39, Alex M. Hendler<amh.pub...@ontolawgy.com> wrote: >>>>> Dan, >>>>> >>>>> I believe the setting you're looking for is >>>>> >>>>> SMWResultPrinter::$maxRecursionDepth = x; >>>>> >>>>> Place that after your enableSemantics statement in LocalSettings.php, >>>>> where x = [desired depth]-1. >>>>> >>>>> The reason you can't get beyond 3 levels on Referata is that (I believe) >>>>> it uses the default max. depth of 2. includes/SMW_QueryPrinter.php >>>>> explains the recursion settings very well. >>>>> >>>>> The developers can correct me if I am wrong, but I believe that >>>>> recursion is intentionally not more publicly documented (i.e., outside >>>>> of source code) because it can do odd and perhaps unpredictable things >>>>> to semantics as well as performance if you increase it above 2. >>>>> >>>>> You can sometimes fool the system into recursing beyond >>>>> maxRecursionDepth, but that is not something I would encourage anyone to >>>>> do intentionally (it also prints out very messy results). >>>>> >>>>> Hope this helps, >>>>> Alex >>>>> >>>>> ----Original Message----- >>>>> From: Dan Bolser<dan.bol...@gmail.com> >>>>> To: Semantic MediaWiki Users<semediawiki-u...@lists.sourceforge.net> >>>>> Subject: [Semediawiki-user] Recursive ask using format=template, stuck >>>>> at depth of 3? >>>>> Date: Wed, 9 Mar 2011 23:58:14 +0000 >>>>> >>>>> Hi, >>>>> >>>>> I wrote a nice 'recursive' template to traverse a DAG here: >>>>> http://scratchpad.referata.com/wiki/Template:Part_of_what >>>>> >>>>> It works by calling itself with an ask using it's given input and then >>>>> the result of the ask and so on up the DAG ... (recursion always makes >>>>> me a little dizzy). >>>>> >>>>> It works great except it only calls itself a maximum of 3 times, as >>>>> evidenced by comparing the results of these two pages: >>>>> * http://scratchpad.referata.com/wiki/Piston >>>>> * http://scratchpad.referata.com/wiki/Engine >>>>> >>>>> I couldn't find the appropriate setting in the manual to bump this >>>>> 'maximum recursion' value. The equivalent value for MediaWiki can be >>>>> found here, but note the default setting is a whopping 40: >>>>> >>>>> http://www.mediawiki.org/wiki/Manual:$wgMaxTemplateDepth >>>>> >>>>> >>>>> Any tips appreciated. >>>>> >>>>> Cheers, >>>>> Dan. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Colocation vs. Managed Hosting >>>>> A question and answer guide to determining the best fit >>>>> for your organization - today and in the future. >>>>> http://p.sf.net/sfu/internap-sfd2d >>>>> _______________________________________________ >>>>> Semediawiki-user mailing list >>>>> semediawiki-u...@lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user >>>>> >>>>> >>>> >>>> >>> >> >> ------------------------------------------------------------------------------ >> Colocation vs. Managed Hosting >> A question and answer guide to determining the best fit >> for your organization - today and in the future. >> http://p.sf.net/sfu/internap-sfd2d >> _______________________________________________ >> Semediawiki-user mailing list >> semediawiki-u...@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/semediawiki-user >> > > ------------------------------------------------------------------------------ > Colocation vs. Managed Hosting > A question and answer guide to determining the best fit > for your organization - today and in the future. > http://p.sf.net/sfu/internap-sfd2d > _______________________________________________ > Semediawiki-devel mailing list > Semediawiki-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/semediawiki-devel > ------------------------------------------------------------------------------ Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d _______________________________________________ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
