Bugs item #1976341, was opened at 2008-05-28 11:43
Message generated for change (Comment added) made by lsidir
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1976341&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: PF/runtime
Group: MonetDB4 4.24
Status: Open
Resolution: None
Priority: 7
Private: No
Submitted By: Wouter Alink (vzzzbx)
Assigned to: Stefan Manegold (stmane)
Summary: XQ: leftovers after deleting document
Initial Comment:
Using todays stable build (preJune, May28, 2008) but it happens with earlier
builds as well), the dbfarm isn't cleaned up as expected after deleting a
document with 'pf:del-doc()'.
When repeatedly adding and deleting documents, the leftovers start piling up,
and eventually the system runs out of disk-space.
See example below.
bash-3.2$ wc -c /tmp/aap.xml
292890833 /tmp/aap.xml
bash-3.2$ echo 'pf:add-doc("/tmp/aap.xml","aap.xml")' | mclient -lxq
bash-3.2$ du -sh /tmp/dbfolder/
367M /tmp/dbfolder/
bash-3.2$ echo 'pf:add-doc("/tmp/aap.xml","aap_1.xml")' | mclient -lxq
bash-3.2$ du -sh /tmp/dbfolder/
733M /tmp/dbfolder/
bash-3.2$ echo 'pf:add-doc("/tmp/aap.xml","aap_2.xml")' | mclient -lxq
bash-3.2$ du -sh /tmp/dbfolder/
1.1G /tmp/dbfolder/
bash-3.2$
bash-3.2$ echo 'pf:del-doc("aap_2.xml")' | mclient -lxq
bash-3.2$ du -sh /tmp/dbfolder/
997M /tmp/dbfolder/
bash-3.2$ echo 'pf:del-doc("aap_1.xml")' | mclient -lxq
bash-3.2$ du -sh /tmp/dbfolder/
895M /tmp/dbfolder/
bash-3.2$ du -sh /tmp/dbfolder/
895M /tmp/dbfolder/
bash-3.2$ du -sh /tmp/dbfolder/
895M /tmp/dbfolder/
bash-3.2$ echo 'pf:del-doc("aap.xml")' | mclient -lxq
bash-3.2$ du -sh /tmp/dbfolder/
794M /tmp/dbfolder/
bash-3.2$ du -sh /tmp/dbfolder/
794M /tmp/dbfolder/
bash-3.2$
----------------------------------------------------------------------
>Comment By: Lefteris Sidirourgos (lsidir)
Date: 2008-05-29 15:28
Message:
Logged In: YES
user_id=1856546
Originator: NO
Before adding the doc:
232K MonetDB4
12K MonetDB4/xquery_logs
8.0K MonetDB4/xquery_logs/demo
216K MonetDB4/dbfarm
212K MonetDB4/dbfarm/demo
204K MonetDB4/dbfarm/demo/bat
4.0K MonetDB4/dbfarm/demo/bat/LEFTOVERS
8.0K MonetDB4/dbfarm/demo/bat/BACKUP
4.0K MonetDB4/dbfarm/demo/bat/03
44K MonetDB4/dbfarm/demo/bat/01
4.0K MonetDB4/dbfarm/demo/bat/05
4.0K MonetDB4/dbfarm/demo/bat/04
After adding the doc:
366M MonetDB4
12K MonetDB4/xquery_logs
8.0K MonetDB4/xquery_logs/demo
366M MonetDB4/dbfarm
366M MonetDB4/dbfarm/demo
366M MonetDB4/dbfarm/demo/bat
4.0K MonetDB4/dbfarm/demo/bat/LEFTOVERS
8.0K MonetDB4/dbfarm/demo/bat/BACKUP
4.0K MonetDB4/dbfarm/demo/bat/03
44K MonetDB4/dbfarm/demo/bat/01
179M MonetDB4/dbfarm/demo/bat/05
188M MonetDB4/dbfarm/demo/bat/04
After deleting the doc:
265M MonetDB4
12K MonetDB4/xquery_logs
8.0K MonetDB4/xquery_logs/demo
265M MonetDB4/dbfarm
265M MonetDB4/dbfarm/demo
265M MonetDB4/dbfarm/demo/bat
4.0K MonetDB4/dbfarm/demo/bat/LEFTOVERS
8.0K MonetDB4/dbfarm/demo/bat/BACKUP
4.0K MonetDB4/dbfarm/demo/bat/03
44K MonetDB4/dbfarm/demo/bat/01
77M MonetDB4/dbfarm/demo/bat/05
188M MonetDB4/dbfarm/demo/bat/04
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-05-29 12:12
Message:
Logged In: YES
user_id=572415
Originator: NO
could you please also report the results of
`find .../MonetDB4 -type d | xargs du -sh`
?
Thanks!
----------------------------------------------------------------------
Comment By: Lefteris Sidirourgos (lsidir)
Date: 2008-05-29 12:01
Message:
Logged In: YES
user_id=1856546
Originator: NO
Hi, I repeated this bug, and I am attaching the results. This is not a
query compile time problem but a runtime, but because the mps and algebra
use a slightly different "play docmgm tape" I did the experiments with both
back-ends. The problem is there for both back-ends, and the disk is cleaned
only when the mserver *starts* again (and not when it is killed). The
document I used is around 280MB.
I am re-assigning this bug to Stefan:) (or Peter?)
File Added: leftovers_after_del.report
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-05-28 12:40
Message:
Logged In: YES
user_id=572415
Originator: NO
Moreover, does /tmp/dbfolder/ only contain the dbfarm or also the log
directory?
How does the size of the log directory evolve/change in above example?
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-05-28 12:38
Message:
Logged In: YES
user_id=572415
Originator: NO
Wouter,
could you please check
(1) which of the sub directories of /tmp/dbfolder/ contains the
left-overs, and
(2) whether stopping and re-starting Mserver does change (reduce) the size
of /tmp/dbfolder/ (and/or its subdirectories)
?
Thanks!
----------------------------------------------------------------------
Comment By: Wouter Alink (vzzzbx)
Date: 2008-05-28 11:50
Message:
Logged In: YES
user_id=621590
Originator: YES
(increased priority a little, as it seems a showstopper for us)
additional information:
- restarting mserver does clean up the data as expected
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1976341&group_id=56967
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
