Bugs item #1872685, was opened at 2008-01-16 12:38
Message generated for change (Comment added) made by boncz
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1872685&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Core
Group: MonetDB4 CVS Head
Status: Open
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Roberto Cornacchia (cornuz)
Assigned to: Nobody/Anonymous (nobody)
Summary: PF(tijah): HEAPsetmmap: Assertion `p' failed.
Initial Comment:
.../gdk/gdk_bat.mx:2501: HEAPsetmmap: Assertion `p' failed.
This happens while indexing a (not small) xml file in pftijah.
It is relatively new, it did not happen before Xmas (same data, same
operations).
Not completely sure whether this is a M4 or a pftijah bug, but I saw some
recent changes to gdk_heap..
I enclose the script used to index (a part of) wikipedia collection in pftijah.
To reproduce the bug:
- run the server with pathfinder module
- MODIFY the 'DST' variable in the script
- run the scrip
I am running the head version of everything, compiled with default options and
32bit OID's.
----------------------------------------------------------------------
>Comment By: Peter Boncz (boncz)
Date: 2008-02-11 01:36
Message:
Logged In: YES
user_id=591107
Originator: NO
Recent lowering te gdk_vm_minsize threshold, that determines when BAT heap
allocs
(due to a new or extending BAT) start directly mapping on disk,
exposed this bug that has been there for 4 years already.
The gdk_vm_minsize threshold is now 64MB on Linux (it is higher on
Windows,
because of high cost).
Direct-mmaps usually are only allowed for non-writable bats, but in this
case we get it in a temporary, but non-persistent bat.
Such cases are (DIRTY) recognized by filename extension: the mode
is STORE_MMAP, it the filename ends with .priv.
The bug occurred when making the bat non-writable again. The assert
goes off because it expects the .priv extension, but it is gone.
Turns out it was gone for two reasons:
(1) the initial HEAPalloc/HEAPextend failed to attach the privext
extension in the first place
(2) on a subsequent HEAPextend, the .priv extension would be
lost due to the behavior of decompose_filename()
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-02-10 15:04
Message:
Logged In: YES
user_id=572415
Originator: NO
Increasing vm_minsize to <=512 MB (`Mserver --set
gdk_vm_minsize=$[512*1024*1024]`) does NOT avoid the assertion / segfault.
Increasing vm_minsize to >=768 MB (`Mserver --set
gdk_vm_minsize=$[768*1024*1024]`) DOES avoid the assertion / segfault.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-02-10 13:00
Message:
Logged In: YES
user_id=572415
Originator: NO
some more debug info:
========
(gdb) print committed
$6 = 32
(gdb) print *hp
$7 = {maxsize = 268435456, free = 153429276, size = 268435456, base =
0x2aaafb320000 "FII\002EII\002", filename = 0x1b895d8 "07/743.tail",
storage = 1 '\001', copied = 0 '\0', newstorage = 1 '\001', dirty = 0
'\0'}
========
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-02-10 12:58
Message:
Logged In: YES
user_id=572415
Originator: NO
Just a quick update:
also after disabling GDKvmalloc for sizes that exceed GDK_mem_bigsize,
this probelm persists on the latest Stable release branch (see below;
segfault iso. assertion as assertions are disabled); i.e., the problem is
indeed related to GDK_vm_minsize, not to GDK_mem_bigsize ...
========
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1115703632 (LWP 30275)]
0x00002aaaac6dafc4 in HEAPsetmmap (hp=0x1288c40, committed=32,
remap=0x427f098c) at
/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB/src/gdk/gdk_bat.mx:2536
2536 *p = 0; /* cut off .priv */
2517 /* transition heap from writable (or appendable) to readonly */
2518 static int
2519 HEAPsetmmap(Heap *hp, int committed, int *remap)
2520 {
2521 assert(hp);
2522
2523 if (hp->base == NULL || hp->storage == STORE_MEM ||
hp->filename == NULL || !committed) {
2524 return hp->storage;
2525 }
2526 *remap = TRUE;
2527 if (hp->storage == STORE_MMAP) {
2528 /* state(2), goto state (3) */
2529 long_str src, dst;
2530 str p = strstr(hp->filename, ".priv");
2531 int ret;
2532
2533 assert(p); /* absence of .priv would indicate
an illegal heap state */
2534
2535 GDKfilepath(src, BATDIR, hp->filename, NULL);
2536 *p = 0; /* cut off .priv */
2537 GDKfilepath(dst, BATDIR, hp->filename, NULL);
(gdb) print hp->filename
$3 = 0x1b895d8 "07/743.tail"
(gdb) print p
$4 = 0x0
(gdb) print hp->storage
$5 = 1 '\001'
========
----------------------------------------------------------------------
Comment By: jzhang (juntezhang)
Date: 2008-01-26 16:30
Message:
Logged In: YES
user_id=1992393
Originator: NO
I have encountered this problem when building an index. I am using a
default
installation of MonetDB + Pathfinder module. Compiled for
x86_64-suse-linux-gnu/64bit with 64bit OIDs; dynamically linked. If anyone
knows a work-around for this, please let me know.
----------------------------------------------------------------------
Comment By: Roberto Cornacchia (cornuz)
Date: 2008-01-22 11:03
Message:
Logged In: YES
user_id=883451
Originator: YES
After some investigation,
I found out that the bug is triggered by the
commit in gdk_utils.mx, 1.200 -> 1.201, dated 14 Dec 2007:
- lower the threshold for creation of mmap-backed heaps to 64MB
[...]
Unfortunately this doesn't say much (to me, at least)
on the real cause of the problem.
Lowering the GDK_vm_minsize has probably had the effect of
triggering this bug earlier (i.e. with smaller data size)
than before the change.
About PF / PfTijah, it is indeed triggered by PfTijah and not by PF
alone:
Server:
$ Mserver --dbinit="module(pathfinder);"
Client:
$ echo 'pf:add-doc("/export/scratch0/roberto/wikipedia_XML/part-0.xml",
"part-0.xml")' | mclient -lx
[Succeeds]
$ echo "tijah:create-ft-index()" | mclient -lx
[Fails]
So, the bug shows up during PfTijah indexing.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-01-21 16:47
Message:
Logged In: YES
user_id=572415
Originator: NO
Indicate in the Summary, that this bug is triggered by Pathfinder; PFtijah
to be precise.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1872685&group_id=56967
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
