Bugs item #2518276, was opened at 2009-01-19 00:39
Message generated for change (Comment added) made by jflokstra
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2518276&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: PFtijah
Group: Pathfinder "stable"
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Loredana Afanasiev (lafanasi)
Assigned to: Henning Rode (hrode)
Summary: memory leak
Initial Comment:
Dear developers,
When running the simple keyword query below ~100 times, the MServer's memory
footprint grows to 100% and the server crashes with no error messages. My
collection has ~1GB.
let $opt := <TijahOptions ft-index="polietiekedata" ir-model="NLLR"/>
let $c := collection("HAN")
let $qid := tijah:query-id($c, "//spreker[about(.,%KEYWORD%)]", $opt)
for $res in tijah:nodes($qid)
return <pair>{( string($res/@naam), tijah:score($qid, $res))}</pair>
thank you!
l.
OS: Linux, 2.6.24.3-50.fc8
Mem: 19GB
Compiler: gcc 4.1.2
Parser tools: ?
autoconf (GNU Autoconf) 2.61
automake (GNU automake) 1.10
ltmain.sh (GNU libtool) 1.5.24
----------------------------------------------------------------------
>Comment By: Jan Flokstra (jflokstra)
Date: 2009-02-03 11:40
Message:
I had a more systematic look at the problem here. I think I solved the
problem by:
1) removing 2 unnecessary BBPfixes()
2) assigning the nexi_result variable in the generated pftijah script to
nil;
This was a very strange error. The bat is removed from the buffer pool
when the script is run throught "mclient -lmil" and we thought our
generated script was memory leak free. But when we called it from "C" by
"monet_exec(script_str);" the bat was not cleared and we have to assign nil
manually. To prevent further problems in the futrue we plan to put the
entire scripts between { }. This way the MIL interpreter should kill all
used variables also when called from "C"
We reduced memory consumption for 'big' queries' (eg. 50.000 results) from
850K to 16K which is about avarage for Pathfinder,
I wait for conformation from Loredana before I close the bug.
JanF.
----------------------------------------------------------------------
Comment By: Henning Rode (hrode)
Date: 2009-01-19 16:55
Message:
i just checked it together with jan f..
result: we have indeed a memory leak here.
5K per simple pathfinder query.
1.8M per simple pf/tijah query.
26K per pf/tijah query with new query processor.
the 1.8M are of course quite bad, but since it only occurs in the old
version of the query processor, we won't fix that anymore.
the 26K in the new version are now on my todo list. i still have to find
out, whether it occurs in the C code of the query processor, or whether we
simply do not give enough hints to MonetDB to free MIL vars.
loredana, you can maybe use as an additional option to your queries:
<TijahOptions newversion="true"/>
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2518276&group_id=56967
------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
