Ok, SIO works fine when page purging, and using SMW_refreshData. However after looking like it's going fine for awhile running runJobs.php after starting an update job from SMWAdmin, SIO starts to go berserk.
I'm sure of one thing. This has something to do with pages being parsed multiple times. For some reason SMW_refreshData only parses a page once for data, while when using a refresh job instead it will for some reason end up going back and parsing the page again multiple times. Because it ends up parsed multiple times and SIO only does a simplistic $main_page_name == self::$cur_page_name check, because the page name is the same on the multiple subsequent parses it thinks its still inside the same parse, instead of been reset and as a result the internal object index ends up getting incremented beyond the actual number of sio calls in the page which leads to even more smw_ids, which leads to even more page parses, which causes the job queue to loop over and over making exponentially larger parses, which add a number of new sio entries into smw_ids each time. There are multiple things to fix here. The job queue should not be re-parsing the page over and over (I'm having a hard time finding why the job queue does this, but the refresh script doesn't; The only two things different is the script executes refreshData with a count of 1, and the refresh job uses a count of 2, and the script disables $smwgEnableUpdateJobs, however I don't see that used in any relevant locations). And SIO should not be using a simple page title check, it should use a proper parser hook to understand when the parser has started parsing a new page. So that SIO doesn't think that it's still in the same parse, when really it's being asked to parse the same page twice (though of course as noted before, this honestly shouldn't be allowed to happen). ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name] Daniel Friesen wrote: > Ok, it looks like SIO has inserted 1428 Module/1.1#... items into > smw_ids, and 12159 Module/1.0#... items into smw_ids. > This does affect all pages with SIO, I just couldn't see them before > because there were so many smw_ids fo Modules/1.0 that it never even got > to the other pages. > Module/1.0 had been purged more times than Modules/1.1 had, so my guess > is that SIO was inserting more and more items for each time the page was > parsed. > I'm 100% certain that I never ever let 12159 sio calls show up on any > page..... The pages wouldn't even render trying to do that many. > > I'm going to delete all the SMW data, set SMW back up, refresh the data. > Then install SIO after hacking it to spam a list of the smw_ids items it > tries to create into the page output. And put my sio calls back in > place. And see what circumstances sio goes berserk adding smw_ids items. > > ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name] > > Daniel Friesen wrote: > >> Tried another update job with SIO uninstalled, the berserk job has >> returned... it worked fine before when SIO was gone, but it's back for >> some strange reason. >> >> ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name] >> >> >> >> Daniel Friesen wrote: >> >> >>> MW 1.15, SMW trunk 1.5.0.a-SVN svn info says r63655, SIO trunk (that's >>> the only thing in SVN anyways). >>> Everything cept SIO can be seen at >>> http://wiki.commonjs.org/wiki/Special:Version (Note that FlaggedRevs was >>> installed AFTER the SIO issue) >>> #set_internal is commented out but can be seen at >>> http://wiki.commonjs.org/index.php?title=Template:Spec/implementation&action=edit >>> >>> <http://wiki.commonjs.org/index.php?title=Template:Spec/implementation&action=edit> >>> Only page affected was http://wiki.commonjs.org/wiki/Modules/1.0 >>> The same SIO call was probably included in one or two other pages, but >>> the job queue didn't obsess over them. Though don't know if their data >>> made it into the cache. >>> SMWAdmin's update data was also used, don't know if that was before or >>> after SIO going berserk. >>> Berserk job didn't go away even after uninstall until the refresh job >>> was manually removed from the db. However the berserk job only came into >>> existence when SIO was installed. >>> >>> >>> ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name] >>> >>> Yaron Koren wrote: >>> >>> >>> >>>> This may be the least helpful bug report I've ever received... what >>>> versions of MW, SMW and SIO are you using? And did this happen for >>>> every internal object you tried? And what does the relevant >>>> #set_internal call look like? >>>> >>>> -Yaron >>>> >>>> On Fri, Mar 12, 2010 at 11:14 PM, Daniel Friesen >>>> <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> I found a bit of a bug in Semantic Internal Objects. >>>> >>>> Using SIO caused my Job queue to go berserk. Jobs would run constantly >>>> centered on a single page that was using SIO. When I took a closer >>>> look >>>> at the database the page names actually had #'s in them, in other >>>> words >>>> they were for internal objects. however these numbers were up above >>>> 3000, SIO was basically generating insane jobs to refresh data that >>>> shouldn't exist and going berserk on the job queue running forever. >>>> >>>> -- >>>> ~Daniel Friesen (Dantman, Nadir-Seen-Fire) >>>> [http://daniel.friesen.name] >>>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Download Intel® Parallel Studio Eval >>>> Try the new software tools for yourself. Speed compiling, find bugs >>>> proactively, and fine-tune applications for parallel performance. >>>> See why Intel Parallel Studio got high marks during beta. >>>> http://p.sf.net/sfu/intel-sw-dev >>>> _______________________________________________ >>>> Semediawiki-devel mailing list >>>> [email protected] >>>> <mailto:[email protected]> >>>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel >>>> >>>> >>>> >>>> >>>> -- >>>> WikiWorks · MediaWiki Consulting · http://wikiworks.com >>>> ------------------------------------------------------------------------ >>>> >>>> ------------------------------------------------------------------------------ >>>> Download Intel® Parallel Studio Eval >>>> Try the new software tools for yourself. Speed compiling, find bugs >>>> proactively, and fine-tune applications for parallel performance. >>>> See why Intel Parallel Studio got high marks during beta. >>>> http://p.sf.net/sfu/intel-sw-dev >>>> ------------------------------------------------------------------------ >>>> >>>> _______________________________________________ >>>> Semediawiki-devel mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel >>>> >>>> >>>> >>>> >>> ------------------------------------------------------------------------------ >>> Download Intel® Parallel Studio Eval >>> Try the new software tools for yourself. Speed compiling, find bugs >>> proactively, and fine-tune applications for parallel performance. >>> See why Intel Parallel Studio got high marks during beta. >>> http://p.sf.net/sfu/intel-sw-dev >>> >>> >>> >> ------------------------------------------------------------------------------ >> Download Intel® Parallel Studio Eval >> Try the new software tools for yourself. Speed compiling, find bugs >> proactively, and fine-tune applications for parallel performance. >> See why Intel Parallel Studio got high marks during beta. >> http://p.sf.net/sfu/intel-sw-dev >> >> > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Semediawiki-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
