Hi,
I added some memory_get_usage() calls to the code
'before' is the return value before the call to
SMWQueryProcessor::getResultFromQueryString (with different offset values)
'#1' value in getInstanceQueryResult() after
$result = new SMWQueryResult()
'#2' value in getInstanceQueryResult() after the foreach loops
assigning $row[] = new SMWResultArray
Offset:0 before:15,295,804
#1: 16,775,940
#2: 17,468,932
Offset:10 before:17,557,768
#1: 17,581,712
#2: 18,729,208
Offset:20 before:18,701,124
#1: 18,733,052
#2: 20,044,556
:
Offset:480 before:128,938,860
#1: 128,981,036
#2: 131,807,872
Offset:490 before:131,779,456
#1: 131,821,492
<b>Fatal error</b>: Allowed memory size of 134217728 bytes
Basically there are app. 240K memory used up for my query after calling
getResultFromQueryString.and at present that limits me to less than 500 returned
results.
It would of course be nice if there would be a way to release some of that
memory again after the result has been processed (but I guess first one would
have to know where it went, to the SMWQueryResult object or somewhere else?)
Another question is, if (as I only just need the page titles from the query
result) there is a simpler way to do that than calling getResultFromQueryString?
Cheers,
Gu
Quoting Markus Krötzsch <[email protected]>:
> Hi,
>
> I am not aware of a good way of debugging memory issues in PHP yet. Can
> anyone
> recommend a free profiler or the like? As for your problem, there is a
> general
> tendency of PHP to hoard memory during (very) long runs. This is a known bug
>
> that apparently can only be avoided by re-engineering the object dependencies
>
> in affected code [1]. However, without a proper memory profiling, I have no
> feasible approach for finding out where this should be done.
>
> Of course, it is also possible that some other problem causes memory to be
> eaten without PHP being to blame. Usually this is caused by look-up arrays
> that are used as caches in PHP. For example, MediaWiki has a LinkCache that
> can grow without control; if you do long runs that may (indirectly) use this
>
> cache, then it is required to purge it manually from time to time. Again,
> there might be further such problems, though I have tried to limit all
> caches'
> size in SMW. Ideas for better debugging are welcome.
>
> Regards,
>
> Markus
>
> [1] http://paul-m-jones.com/?p=262
>
>
> On Mittwoch, 7. Januar 2009, [email protected] wrote:
> > Hi,
> >
> > I wonder if anyone can tell me if the functions used when I call
> > SMWQueryProcessor::getResultFromQueryString should release all the memory
> > after they finished?
> >
> > I thought I could avoid a memory problem by calling
> > getResultFromQueryString with different 'offset' values to retrieve
> smaller
> > chunks of articles. However, at some point I still run out of memory
> during
> > the loop, even if I just call the function and ignore the result.
> >
> > $maxresults = 500;
> > $limit = $params['limit']; # is 10
> > for ($ofs = 0; $ofs < $maxresults ; $ofs += $limit) {
> > $params['offset'] = $ofs;
> > $dummy = SMWQueryProcessor::getResultFromQueryString($querystring,
> > $params, $printouts, SMW_OUTPUT_WIKI);
> > }
> >
> > When $ofs reaches 490 I run out of memory (Fatal error: Allowed memory
> size
> > of 134217728 bytes exhausted).
> >
> > The test query is '[[MyProp::~*;?;?]][[Category:MainArticle]]' with about
> > 19000 articles matching (as using [[MyProp::?;?;?]] gives the error 'The
> > value of property "MyProp" was not understood', I have to replace one '?'
> > with '~*').
> >
> > MediaWiki 1.13.1
> > PHP 5.2.1
> > MySQL 6.0.45
> > SMW 1.5a SVN
> >
> > Thanks,
> > Gu
> >
> >
> >
> >
> >
> ---------------------------------------------------------------------------
> >--- Check out the new SourceForge.net Marketplace.
> > It is the best place to buy or sell services for
> > just about anything Open Source.
> > http://p.sf.net/sfu/Xq1LFB
> > _______________________________________________
> > Semediawiki-devel mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>
>
> --
> Markus Krötzsch
> Semantic MediaWiki http://semantic-mediawiki.org
> http://korrekt.org [email protected]
>
>
------------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It is the best place to buy or sell services for
just about anything Open Source.
http://p.sf.net/sfu/Xq1LFB
_______________________________________________
Semediawiki-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
