Bugs item #2803460, was opened at 2009-06-09 14:07
Message generated for change (Comment added) made by yingying
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2803460&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: XML
Group: XML
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: francois guerin (fguerin)
Assigned to: Nobody/Anonymous (nobody)
Summary: #GDKmmap(...) fails, try to free up space
Initial Comment:
When i attempt to insert a file in my MonetDB database, the document shredding
crashes with the error:
xquery>pf:add-doc("/MonetDB/mercredi.xml","mercredi.xml","final")
more><>
#GDKmmap(430702592) fails, try to free up space [memory in
use=158280300,virtual memory in use=0]
#GDKmmap(430702592) result [mem=158280300,vm=0]
MAPI = mone...@localhost:50000
QUERY = pf:add-doc("/MonetDB/mercredi.xml","mercredi.xml","final")
ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples
0...@0.
!ERROR: [shred_url]: first error was:
!ERROR: HEAPextend: failed to extend to 430598008 for 05/522tail
!ERROR: shredBAT_extend: BATextend["_attr_own"](107649502 to 107649502)
fails
!ERROR: CMDshred_url: operation failed.
Timer 638806.145 msec
The xml file can be accessed at the address:
ftp://fguerin.inria.perso.neuf.fr/mercredi.xml.bz2
it's about 4-5Go and well-formed (i checked it)
I installed the lastest version of MonetDB ("May") with the options
--enable-xquery --enable-optimize
on several OS (mandriva, debian and macosx).
----------------------------------------------------------------------
>Comment By: Ying Zhang (yingying)
Date: 2009-06-17 12:39
Message:
This is indeed a 32bits related problem, it works with 64bits monet:
$ env CC="gcc -m64" CXX="g++ -m64" monetdb-install.sh
--prefix=/Users/jennie/mdb_may2009_64bits --enable-optimize
--enable-xquery
$ Mserver --dbinit="module(pathfinder);"
# MonetDB Server v4.30.0
# based on GDK v1.30.0
# Copyright (c) 1993-July 2008, CWI. All rights reserved.
# Copyright (c) August 2008-2009, MonetDB B.V.. All rights reserved.
# Compiled for i386-apple-darwin9.7.0/64bit with 64bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.
# PF/Tijah module v0.13.0 loaded. http://dbappl.cs.utwente.nl/pftijah
# MonetDB/XQuery module v0.28.3 loaded (default back-end is 'algebra')
# XRPC administrative console at http://127.0.0.1:50001/admin
MonetDB>xmldb_print();
#-----------------------------------------#
# collection alias URI size # name
# str str str lng # type
#-----------------------------------------#
MonetDB>shred_doc("/Users/jennie/tmp/mercredi.xml", "merc.xml",
"merc.xml", 10LL);
# shredding with holes: pagesize=16384, #fillsize=14746
# Elapsed time = 34m 05s 860ms 584us [008us/node]
# Shredded 1 XML document (merc.xml), total time after commit=2315.454s
The document might be just too large for 32bits, but it should give an
error, instead of segmentation fault, which still (i.e., today) happens:
$ monetdb-install.sh --prefix=/Users/jennie/mdb_may2009_32bits
--enable-optimize --enable-xquery --nightly=stable
[Wed Jun 17 12:14:17 CEST 2009]
[leopard:/Users/jennie/mdb_may2009_32bits/bin]
$ Mserver --dbinit="module(pathfinder);"
# MonetDB Server v4.30.1
# based on GDK v1.30.1
# Copyright (c) 1993-July 2008, CWI. All rights reserved.
# Copyright (c) August 2008-2009, MonetDB B.V.. All rights reserved.
# Compiled for i386-apple-darwin9.7.0/32bit with 32bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.
# PF/Tijah module v0.13.0 loaded. http://dbappl.cs.utwente.nl/pftijah
# MonetDB/XQuery module v0.28.3 loaded (default back-end is 'algebra')
# XRPC administrative console at http://127.0.0.1:50001/admin
MonetDB>shred_doc("/Users/jennie/tmp/mercredi.xml", "merc.xml",
"merc.xml", 10LL);
# shredding with holes: pagesize=16384, #fillsize=14746
#GDKmmap(301072384) fails, try to free up space [memory in
use=103246812,virtual memory in use=0]
#GDKmmap(301072384) result [mem=103246812,vm=0]
Segmentation fault
----------------------------------------------------------------------
Comment By: Fabian (mr-meltdown)
Date: 2009-06-09 16:45
Message:
env CC="gcc -m64" CXX="g++ -m64" ./monetdb-install.sh --ga-helemaal-los
(but that will fail on OSX, unless you have 64-bits variants of the
libraries libpcre and openssl)
----------------------------------------------------------------------
Comment By: Ying Zhang (yingying)
Date: 2009-06-09 16:41
Message:
On Mac OS, I suspect MonetID is compiled for 32-bits.
By default, (my) gcc on Mac produces 32-bits binaries, and with
monetdb-install.sh, there is no option to specify that it should produce
64-bits binaries. I installed the stable nightly build yesterday, and
tried to shred the same document, and I got similar error. Moreover, if
the document is shredded as updatable doc., Mserver segmentation faults.
I will (try to) compile monet on my mac for 64-bits to see if the problem
still exists.
Below are copy-paste of what I did, and the error messages:
$monetdb-install.sh --prefix=/Users/jennie/mdb_may2009 --enable-optimize
--enable-xquery --nightly=stable
[Mon Jun 8 23:31:50 CEST 2009]
[leopard:/Users/jennie/mdb_may2009/bin]
$ Mserver --dbinit="module(pathfinder);"
# MonetDB Server v4.30.1
# based on GDK v1.30.1
# Copyright (c) 1993-July 2008, CWI. All rights reserved.
# Copyright (c) August 2008-2009, MonetDB B.V.. All rights reserved.
# Compiled for i386-apple-darwin9.7.0/32bit with 32bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.
# PF/Tijah module v0.13.0 loaded. http://dbappl.cs.utwente.nl/pftijah
# MonetDB/XQuery module v0.28.3 loaded (default back-end is 'algebra')
# XRPC administrative console at http://127.0.0.1:50001/admin
MonetDB>shred_doc("/Users/jennie/tmp/mercredi.xml", "mercredi.xml",
"mercredi.xml", 10LL);
# shredding with holes: pagesize=16384, #fillsize=14746
#GDKmmap(376438784) fails, try to free up space [memory in
use=157277132,virtual memory in use=79298560]
#GDKmmap(376438784) result [mem=157277132,vm=79298560]
Segmentation fault
[Mon Jun 8 23:38:08 CEST 2009]
[leopard:/Users/jennie/mdb_may2009/bin]
$ ls -lh ~/tmp/mercredi.xml
-rw-r--r-- 1 jennie staff 4.7G May 14 13:25 /Users/jennie/tmp/
mercredi.xml
[Mon Jun 8 23:45:47 CEST 2009]
[leopard:/Users/jennie/mdb_may2009/bin]
$ Mserver --dbinit="module(pathfinder);"# MonetDB Server v4.30.1
# based on GDK v1.30.1
# Copyright (c) 1993-July 2008, CWI. All rights reserved.
# Copyright (c) August 2008-2009, MonetDB B.V.. All rights reserved.
# Compiled for i386-apple-darwin9.7.0/32bit with 32bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.
# PF/Tijah module v0.13.0 loaded. http://dbappl.cs.utwente.nl/pftijah
# MonetDB/XQuery module v0.28.3 loaded (default back-end is 'algebra')
# XRPC administrative console at http://127.0.0.1:50001/admin
MonetDB>xmldb_print();
#-----------------------------------------#
# collection alias URI size # name
# str str str lng # type
#-----------------------------------------#
MonetDB>shred_doc("/Users/jennie/tmp/mercredi.xml", "mercredi.xml",
"mercredi.xml", 0LL);
#GDKmmap(430702592) fails, try to free up space [memory in
use=157297100,virtual memory in use=0]
#GDKmmap(430702592) result [mem=157297100,vm=0]
!ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0...@0.
!ERROR: [shred_url]: first error was:
!ERROR: HEAPextend: failed to extend to 430598008 for 02/236tail
!ERROR: shredBAT_extend: BATextend["_attr_own"](107649502 to 107649502)
fails
!ERROR: CMDshred_url: operation failed.
MonetDB>exit();
[Mon Jun 8 23:55:35 CEST 2009]
[leopard:/Users/jennie/mdb_may2009/bin]
$
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2009-06-09 16:32
Message:
Is your OS and your MonetDB 32-bit or 64-bit?
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2803460&group_id=56967
------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
