Bugs item #1607210, was opened at 2006-12-02 01:27 Message generated for change (Settings changed) made by stmane You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1607210&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: PF/runtime Group: Pathfinder CVS Head >Status: Closed Resolution: Fixed Priority: 8 Private: No Submitted By: Peter Boncz (boncz) Assigned to: Peter Boncz (boncz) Summary: XQ: server-side compilation crash (member benchmark) Initial Comment: Philippe Michiels reported at dagstuhl that the simple CHILD test of the MemBeR benchmark crashes when run in MapiClient -lx; crashing during the *compilation* process It works with pf standalone, translating to MIL explicitly http://ilps.science.uva.nl/Resources/MemBeR/CHILD.html The query used in this micro-benchmark navigates downward from the root, performing n successive child navigation steps, where n is to be varied. For every integer n, the corresponding query, named CHILD(n), is: /t1/t2/.../tn THE BUG ONLY OCCURS WITH n>15 AND WITH SERVER SIDE COMPILATION may well be some buffer overflow.. ---------------------------------------------------------------------- >Comment By: Stefan Manegold (stmane) Date: 2009-11-27 08:36 Message: After some more fixes (see below), the test now appears to work fine on all platforms except - timeout on http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/Int.64.32.d.1-Fedora10/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/Int.64.64.d.1-Fedora10/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/Int.64.32.d.1-Windows6.0/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/Int.64.64.d.1-Windows6.0/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html This needs to be analyzed separately. - abort too deep recursion on http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.32.32.d.1-SunOS5.11/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html This occurs with numerous tests on that platform and needs to be analyzed separately. Closing. =================================================================== 2009/11/23 - stmane: pathfinder/runtime/pathfinder.mx,1.462.4.13 try even more conservative stack limit (PFmaxstack) for too deep recusion check (PFrecursion_fence()) to check whether this helps to prevent the remaining segfaults with test tests/BugTracker/Tests/server-side_compilation_crash.SF-1607210.* on some platforms; cf. http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.32.32.d.1-Debian4.0/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Gentoo2.0.1/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.32.32.d.1-SunOS5.11/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html =================================================================== 2009/11/24 - stmane: pathfinder/tests/BugTracker/Tests/server-side_compilation_crash.SF-1607210_s00.xq,1.4.22.1 pathfinder/tests/BugTracker/Tests/server-side_compilation_crash.SF-1607210_s02.xq,1.4.22.1 after " try even more conservative stack limit (PFmaxstack) for too deep recusion check (PFrecursion_fence()) to check whether this helps to prevent the remaining segfaults with test tests/BugTracker/Tests/server-side_compilation_crash.SF-1607210.* on some platforms; cf. http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.32.32.d.1-Debian4.0/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Gentoo2.0.1/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.32.32.d.1-SunOS5.11/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html " this test does not segfault any more, but runs into "too deep recursion" on the mentioned platforms earlier than on others; hence, we shrink parts of the test a bit. =================================================================== 2009/11/26 - stmane: pathfinder/tests/BugTracker/Tests/server-side_compilation_crash.SF-1607210_s00.xq,1.4.22.2 pathfinder/tests/BugTracker/Tests/server-side_compilation_crash.SF-1607210_s02.xq,1.4.22.2 shrink queries even more to avoid "aborted too deep recursion" also on 32-bit Debian4.0 & 32-bit SunOS5.11 =================================================================== ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-11-21 15:35 Message: =================================================================== 2009/11/21 - stmane: pathfinder/runtime/pathfinder.mx,1.462.4.12 use more conservative stack limit (PFmaxstack) for too deep recusion check (PFrecursion_fence()) to prevent segfault (due to too deep recursion and consequent stack overflow) in tests tests/BugTracker/Tests/collection_management.SF-1726599.* tests/BugTracker/Tests/crash_on_concatenated_query.SF-1730547.* cf., https://sourceforge.net/tracker/?func=detail&aid=1607210&group_id=56967&atid=482468 https://sourceforge.net/tracker/?func=detail&aid=1730547&group_id=56967&atid=482468 =================================================================== Report can be closed once nightly testing confirms the fix for all platforms. ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-11-16 22:18 Message: re-opened as the respective test fails on all platform with timeout or segfault at least since 2009.03.25 cf., http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/tests_BugTracker/server-side_compilation_crash.SF-1607210.out.00.html ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2008-02-10 14:12 Message: Logged In: YES user_id=572415 Originator: NO Appears to work fine on most platforms, again; exceptions: Segfault on RedHat (on Itanium): http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/Int.64.64.d-RedHat4WS/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html Timeout on Windows: http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/Int.64.32.d-Windows5.2/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2008-01-22 14:58 Message: Logged In: YES user_id=572415 Originator: NO re-opened as it fails, again, since 2008.01.12, i.e., after checkins on 2008.01.11, now with ERROR = !fatal error: aborted too deep recursion http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests103/GNU.32.32.d-Fedora6/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html ---------------------------------------------------------------------- Comment By: Peter Boncz (boncz) Date: 2007-06-05 18:37 Message: Logged In: YES user_id=591107 Originator: YES fixed ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2007-06-04 21:19 Message: Logged In: YES user_id=572415 Originator: NO See also [ 1730547 ] XQ: Mserver crashes on concatenated query http://sourceforge.net/tracker/index.php?func=detail&aid=1730547&group_id=56967&atid=482468 ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2007-05-29 10:24 Message: Logged In: YES user_id=572415 Originator: NO See also http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests103/GNU.64.64.d-Fedora6/tests_BugTracker/server-side_compilation_crash.SF-1607210.err.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests103/GNU.64.64.d-Fedora6/tests_BugTracker/server-side_compilation_crash.SF-1607210.out.00.html ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2007-05-28 12:33 Message: Logged In: YES user_id=572415 Originator: NO Added test in pathfinder/tests/BugTracker/Tests/server-side_compilation_crash.SF-1607210* As opposed to [ 1207048 ] XQuery: crashtest http://sourceforge.net/tracker/index.php?func=detail&aid=1207048&group_id=56967&atid=482468 http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests103/GNU.64.64.d-Fedora6/tests_BugDay_2005-11-09_0.9.3/xquery_crashtest.SF-1207048.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests103/GNU.64.64.d-Fedora6/tests_BugDay_2005-11-09_0.9.3/xquery_crashtest.SF-1207048.err.00.html it still crashes with a segfault, i.e., the new fence does not catch the stack overflow in this case, yet. Not analysed, yet. Re-opend to keep us reminded to have another look at this one. ---------------------------------------------------------------------- Comment By: Peter Boncz (boncz) Date: 2007-05-25 23:55 Message: Logged In: YES user_id=591107 Originator: YES fixed using a fence ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2006-12-11 16:10 Message: Logged In: YES user_id=572415 Originator: NO I am pretty sure, this is a problem with recursive burg-generated code running out of stack space, which in unfortunately out of out stack space control, and hence IMHO very hard to prevent/fix... With "pf" standalone, it happens as well, just with a longer path expression (29 child steps in my case): $ n=28 ; i=0 ; q='doc("MyDoc.xml")' ; while [ $i -lt $n ] ; do i=$[i+1] ; q="$q`printf "/t%02d" $i`" ; done ; echo $q | tee /tmp/q.xq ; pf /tmp/q.xq >/dev/null doc("MyDoc.xml")/t01/t02/t03/t04/t05/t06/t07/t08/t09/t10/t11/t12/t13/t14/t15/t16/t17/t18/t19/t20/t21/t22/t23/t24/t25/t26/t27/t28 $ n=29 ; i=0 ; q='doc("MyDoc.xml")' ; while [ $i -lt $n ] ; do i=$[i+1] ; q="$q`printf "/t%02d" $i`" ; done ; echo $q | tee /tmp/q.xq ; pf /tmp/q.xq >/dev/null doc("MyDoc.xml")/t01/t02/t03/t04/t05/t06/t07/t08/t09/t10/t11/t12/t13/t14/t15/t16/t17/t18/t19/t20/t21/t22/t23/t24/t25/t26/t27/t28/t29 Segmentation fault $ gdb --args pf /tmp/q.xq GNU gdb Red Hat Linux (6.3.0.0-1.84rh) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu"...Using host libthread_db library "/lib64/libthread_db.so.1". (gdb) r Starting program: /net/corona.ins.cwi.nl/export/scratch0/manegold/Monet/Testing/Stable/.GNU.64.32.d.--disable-optimize_--enable-debug_--enable-assert.PREFIX./PATHFINDER/bin/pf /tmp/q.xq [Thread debugging using libthread_db enabled] [New Thread 46912496328800 (LWP 30516)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 46912496328800 (LWP 30516)] 0x0000000000483666 in reduce (p=0x2aaaaaae0b90, goalnt=50) at /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/core/fs.brg:874 874 rule = PFfs_rule (STATE_LABEL (p), goalnt); (gdb) li 869 PFpnode_t *kids[MAX_KIDS]; /* leaf nodes of this rule */ 870 bool topdown; /* is this a top-down rule? */ 871 PFcnode_t *c; /* temporary helper variable */ 872 873 /* determine rule that matches for this non-terminal */ 874 rule = PFfs_rule (STATE_LABEL (p), goalnt); 875 876 /* PFinfo (OOPS_NOTICE, "match for rule %i", rule); */ 877 assert (rule); 878 (gdb) up #1 0x00000000004837d9 in reduce (p=0x2aaaaaae0b90, goalnt=49) at /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/core/fs.brg:929 929 reduce (kids[i], nts[i]); (gdb) li 924 /* 925 * Recursively invoke compilation. This means bottom-up compilation. 926 */ 927 if (!topdown) 928 for (unsigned short i = 0; nts[i]; i++) 929 reduce (kids[i], nts[i]); 930 931 /* 932 PFinfo (OOPS_NOTICE, "processing rule %i", rule); 933 */ (gdb) bt #0 0x0000000000483666 in reduce (p=0x2aaaaaae0b90, goalnt=50) at /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/core/fs.brg:874 #1 0x00000000004837d9 in reduce (p=0x2aaaaaae0b90, goalnt=49) at /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/core/fs.brg:929 [...] #680 0x00000000004837d9 in reduce (p=0x2aaaaaae5f00, goalnt=1) at #680 0x00000000004837d9 in reduce (p=0x2aaaaaae5f00, goalnt=1) at /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/core/fs.brg:929 #681 0x0000000000491eaf in PFfs (r=0x2aaaaaae5f00) at /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/core/fs.brg:4209 #682 0x00000000004065da in PFcompile (url=0x7ffffffb672e "/tmp/q.xq", pfout=0x38ecf318c0, status=0x731f40) at /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/compile.c:381 #683 0x0000000000405f11 in main (argc=2, argv=0x7ffffffb4e78) at /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/main.c:905 (gdb) /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/core/fs.brg:929 #681 0x0000000000491eaf in PFfs (r=0x2aaaaaae5f00) at /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/core/fs.brg:4209 #682 0x00000000004065da in PFcompile (url=0x7ffffffb672e "/tmp/q.xq", pfout=0x38ecf318c0, status=0x731f40) at /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/compile.c:381 #683 0x0000000000405f11 in main (argc=2, argv=0x7ffffffb4e78) at /ufs/manegold/_/scratch0/Monet/Testing/Stable/pathfinder/compiler/main.c:905 (gdb) ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2006-12-02 08:38 Message: Logged In: YES user_id=572415 Originator: NO How does it crash? Is there any error message or exit status? Does it work for stand-alone pf also for n >> 15? Might well be a stack-overflow in some burg generated code... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1607210&group_id=56967 ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
