Bugs item #1872685, was opened at 2008-01-16 12:38
Message generated for change (Comment added) made by cornuz
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1872685&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Core
Group: MonetDB4 CVS Head
>Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Roberto Cornacchia (cornuz)
Assigned to: Nobody/Anonymous (nobody)
Summary: PF(tijah): HEAPsetmmap: Assertion `p' failed.
Initial Comment:
.../gdk/gdk_bat.mx:2501: HEAPsetmmap: Assertion `p' failed.
This happens while indexing a (not small) xml file in pftijah.
It is relatively new, it did not happen before Xmas (same data, same
operations).
Not completely sure whether this is a M4 or a pftijah bug, but I saw some
recent changes to gdk_heap..
I enclose the script used to index (a part of) wikipedia collection in pftijah.
To reproduce the bug:
- run the server with pathfinder module
- MODIFY the 'DST' variable in the script
- run the scrip
I am running the head version of everything, compiled with default options and
32bit OID's.
----------------------------------------------------------------------
>Comment By: Roberto Cornacchia (cornuz)
Date: 2008-02-13 15:12
Message:
Logged In: YES
user_id=883451
Originator: YES
The assertion doesn't pop up anymore in my test, so I'll close this bug
report.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-02-11 08:36
Message:
Logged In: YES
user_id=572415
Originator: NO
Peter,
Thanks for the fix. Works fine for me.
Roberto.
could you please check and report, whether the latest Stable release
branches also work for you, and if so, close this bug report?
Unfortunately, Roberto's example is to big / resource demanding to be
added as test to CVS.
Hence, unless we find a "reasonable" test case, we might have to close
this one without having a test monitoring it...
----------------------------------------------------------------------
Comment By: Peter Boncz (boncz)
Date: 2008-02-11 01:36
Message:
Logged In: YES
user_id=591107
Originator: NO
Recent lowering te gdk_vm_minsize threshold, that determines when BAT heap
allocs
(due to a new or extending BAT) start directly mapping on disk,
exposed this bug that has been there for 4 years already.
The gdk_vm_minsize threshold is now 64MB on Linux (it is higher on
Windows,
because of high cost).
Direct-mmaps usually are only allowed for non-writable bats, but in this
case we get it in a temporary, but non-persistent bat.
Such cases are (DIRTY) recognized by filename extension: the mode
is STORE_MMAP, it the filename ends with .priv.
The bug occurred when making the bat non-writable again. The assert
goes off because it expects the .priv extension, but it is gone.
Turns out it was gone for two reasons:
(1) the initial HEAPalloc/HEAPextend failed to attach the privext
extension in the first place
(2) on a subsequent HEAPextend, the .priv extension would be
lost due to the behavior of decompose_filename()
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-02-10 15:04
Message:
Logged In: YES
user_id=572415
Originator: NO
Increasing vm_minsize to <=512 MB (`Mserver --set
gdk_vm_minsize=$[512*1024*1024]`) does NOT avoid the assertion / segfault.
Increasing vm_minsize to >=768 MB (`Mserver --set
gdk_vm_minsize=$[768*1024*1024]`) DOES avoid the assertion / segfault.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-02-10 13:00
Message:
Logged In: YES
user_id=572415
Originator: NO
some more debug info:
========
(gdb) print committed
$6 = 32
(gdb) print *hp
$7 = {maxsize = 268435456, free = 153429276, size = 268435456, base =
0x2aaafb320000 "FII\002EII\002", filename = 0x1b895d8 "07/743.tail",
storage = 1 '\001', copied = 0 '\0', newstorage = 1 '\001', dirty = 0
'\0'}
========
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-02-10 12:58
Message:
Logged In: YES
user_id=572415
Originator: NO
Just a quick update:
also after disabling GDKvmalloc for sizes that exceed GDK_mem_bigsize,
this probelm persists on the latest Stable release branch (see below;
segfault iso. assertion as assertions are disabled); i.e., the problem is
indeed related to GDK_vm_minsize, not to GDK_mem_bigsize ...
========
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1115703632 (LWP 30275)]
0x00002aaaac6dafc4 in HEAPsetmmap (hp=0x1288c40, committed=32,
remap=0x427f098c) at
/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB/src/gdk/gdk_bat.mx:2536
2536 *p = 0; /* cut off .priv */
2517 /* transition heap from writable (or appendable) to readonly */
2518 static int
2519 HEAPsetmmap(Heap *hp, int committed, int *remap)
2520 {
2521 assert(hp);
2522
2523 if (hp->base == NULL || hp->storage == STORE_MEM ||
hp->filename == NULL || !committed) {
2524 return hp->storage;
2525 }
2526 *remap = TRUE;
2527 if (hp->storage == STORE_MMAP) {
2528 /* state(2), goto state (3) */
2529 long_str src, dst;
2530 str p = strstr(hp->filename, ".priv");
2531 int ret;
2532
2533 assert(p); /* absence of .priv would indicate
an illegal heap state */
2534
2535 GDKfilepath(src, BATDIR, hp->filename, NULL);
2536 *p = 0; /* cut off .priv */
2537 GDKfilepath(dst, BATDIR, hp->filename, NULL);
(gdb) print hp->filename
$3 = 0x1b895d8 "07/743.tail"
(gdb) print p
$4 = 0x0
(gdb) print hp->storage
$5 = 1 '\001'
========
----------------------------------------------------------------------
Comment By: jzhang (juntezhang)
Date: 2008-01-26 16:30
Message:
Logged In: YES
user_id=1992393
Originator: NO
I have encountered this problem when building an index. I am using a
default
installation of MonetDB + Pathfinder module. Compiled for
x86_64-suse-linux-gnu/64bit with 64bit OIDs; dynamically linked. If anyone
knows a work-around for this, please let me know.
----------------------------------------------------------------------
Comment By: Roberto Cornacchia (cornuz)
Date: 2008-01-22 11:03
Message:
Logged In: YES
user_id=883451
Originator: YES
After some investigation,
I found out that the bug is triggered by the
commit in gdk_utils.mx, 1.200 -> 1.201, dated 14 Dec 2007:
- lower the threshold for creation of mmap-backed heaps to 64MB
[...]
Unfortunately this doesn't say much (to me, at least)
on the real cause of the problem.
Lowering the GDK_vm_minsize has probably had the effect of
triggering this bug earlier (i.e. with smaller data size)
than before the change.
About PF / PfTijah, it is indeed triggered by PfTijah and not by PF
alone:
Server:
$ Mserver --dbinit="module(pathfinder);"
Client:
$ echo 'pf:add-doc("/export/scratch0/roberto/wikipedia_XML/part-0.xml",
"part-0.xml")' | mclient -lx
[Succeeds]
$ echo "tijah:create-ft-index()" | mclient -lx
[Fails]
So, the bug shows up during PfTijah indexing.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-01-21 16:47
Message:
Logged In: YES
user_id=572415
Originator: NO
Indicate in the Summary, that this bug is triggered by Pathfinder; PFtijah
to be precise.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1872685&group_id=56967
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
