Bugs item #2683759, was opened at 2009-03-11 20:57
Message generated for change (Comment added) made by stmane
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2683759&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: PF general
Group: Pathfinder "stable"
>Status: Open
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Loredana Afanasiev (lafanasi)
Assigned to: Peter Boncz (boncz)
Summary: PF:memory leak when fireing multiple simultaneous clients
Initial Comment:
Hi team,
After running one query in a multi-user scenario (N clients/threads
simultaneously) the memory footprint of mserver (% of the memory that the
server occupies) grew. After repeating the experiment M times the memory grew
each time and eventually crashed. Running the same query sequentially the same
number of times leaves the footprint of mserver constant. Could it be a faulty
memory cleaning? Details below.
Thanks,
l.
Query: q3.xq
let $col := fn:collection("MotiesTweedeKamer")
let $years := fn:distinct-values(
for $date in $col//hiddendatum
return fn:substring(fn:string($date),1,4))
for $y in $years
order by $y ascending
return <result year="{$y}" count="{
count($col//document[fn:substring(fn:string(.//hiddendatum),1,4) = $y])
}"/>
N = number of parallel users
M = number of sequential runs
* CVS HEAD 2009-03-04
Multithreaded
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
N=60,M=1
5793 lafanasi 20 0 11.5g 9.5g 10m S 2 48.4 16:53.90 Mserver
N=60,M=1
5793 lafanasi 20 0 14.7g 12g 10m S 0 61.6 32:56.46 Mserver
N=60,M=1
5793 lafanasi 20 0 16.4g 13g 10m S 0 67.9 48:10.48 Mserver
N=60,M=1
5793 lafanasi 20 0 17.1g 13g 7968 S 1 68.2 64:07.58 Mserver
N=60,M=1
5793 lafanasi 20 0 17.5g 13g 7564 S 2 67.4 78:18.33 Mserver
N=60,M=4
5793 lafanasi 20 0 20.4g 14g 5320 S 2 72.0 143:11.85 Mserver
N=60,M=6
MAPI = mone...@localhost:50000
QUERY = let $col := fn:collection("MotiesTweedeKamer")
ERROR = !ERROR: BBPdecref: 1000061400_prop_val does not have pointer fixes.
!ERROR: BBPdecref: 1000061400_attr_qn does not have pointer fixes.
!ERROR: BBPdecref: 1000061400_attr_prop does not have pointer fixes.
QUERY = let $col := fn:collection("MotiesTweedeKamer")
ERROR = !ERROR: BBPdecref: 1000061400_rid_level does not have pointer fixes.
!ERROR: BBPdecref: 1000061400_rid_prop does not have pointer fixes.
!ERROR: BBPdecref: 1000061400_prop_text does not have pointer fixes.
!ERROR: BBPdecref: 1000061400_prop_val does not have pointer fixes.
!ERROR: BBPdecref: 1000061400_attr_qn does not have pointer fixes.
!ERROR: BBPdecref: 1000061400_attr_prop does not have pointer fixes.
N=60,M=9
Setup connection failed
$stopMserver
$startMserver
Sequential
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
N=1, M=60
13895 lafanasi 20 0 899m 152m 7924 S 0 0.8 9:18.75 Mserver
N=1, M=200
13895 lafanasi 20 0 930m 181m 7924 S 1 0.9 20:46.68 Mserver
N=1, M=200
13895 lafanasi 20 0 930m 181m 7924 S 2 0.9 39:15.99 Mserver
* Pathfinder Stable version
Multithreaded
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
N=60,M=1
20370 ilps_bg 20 0 3783m 430m 9084 S 2 2.1 126:07.23 Mserver
N=60,M=10
20370 ilps_bg 20 0 8377m 889m 9092 S 1 4.4 133:33.95 Mserver
N=60,M=20
20370 ilps_bg 20 0 17.1g 1.7g 9092 S 2 8.4 147:48.97 Mserver
N=60,M=100
20370 ilps_bg 20 0 62.4g 5.7g 9092 S 2 28.8 223:33.26 Mserver
N=60,M=200
20370 ilps_bg 20 0 153g 13g 9092 S 2 69.6 373:01.82 Mserver
N=60,M=200
Server crash
Setup connection failed
MAPI = mone...@localhost:50100
ACTION= read_line
QUERY = MAPI = mone...@localhost:50100
Sequential
N=1,M=60
19694 ilps_bg 20 0 922m 418m 10m S 0 2.1 0:27.42 Mserver
N=1,M=200
19694 ilps_bg 20 0 926m 422m 10m S 2 2.1 1:52.17 Mserver
Machine name:ZOOKST4
OS: Linux, 2.6.24.3-50.fc8
Mem: 19GB
Compiler: gcc 4.1.2
Parser tools: ?
autoconf (GNU Autoconf) 2.61
automake (GNU automake) 1.10
ltmain.sh (GNU libtool) 1.5.24
----------------------------------------------------------------------
>Comment By: Stefan Manegold (stmane)
Date: 2009-04-18 11:06
Message:
re-opened to remind us that we should consider adding a test,
or at least give a reason why not test can be added.
----------------------------------------------------------------------
Comment By: Peter Boncz (boncz)
Date: 2009-04-17 23:54
Message:
Ah, the BBPdecref error you got was also fixed by a recent bug reported by
Wouter (BBPtrim would unload logical views)
----------------------------------------------------------------------
Comment By: Peter Boncz (boncz)
Date: 2009-04-17 23:53
Message:
Mserver creates a new thread for each new entering request (to a certain
limit)
with 60 concurrent requests you will get 60 threads, which execute the
query.
Mserver until a few weeks ago in Linux used the normal globc ptmalloc()
library for much of its work. However, this library tends to create a
different memory arena for each different thread, and tends to cause huge
fragmentation. That is what you were observing.
To reduce memory pressure, indepently from this report, we already changed
over to the direct use of anonymous mmap (vmalloc) for much of the BAT
requests. This means that memory pressure on malloc() is now much less.
Therefore, this issue has disappeared by itself.
However, when re-running your query scrip, I actually got quite a few
crashes, as I was easily able to overwhelm Mserver with too many threads. I
fixed this (in HEAD) and now it does not crash anymore. However, under
overload, mclient might produce an error in trying to connect with a
server. This is a fact of life (deinal of service), though Mserver could
theoretically be made to be much more resilient against this.
----------------------------------------------------------------------
Comment By: Loredana Afanasiev (lafanasi)
Date: 2009-03-12 12:39
Message:
btw,
<collection updatable="false" size="120 MiB"
numDocs="27946">MotiesTweedeKamer</collection>
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2683759&group_id=56967
------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
