Hi Peter,
Peter Boncz wrote: > Like Martin indicated, MonetDB runs out of memory here, when trying to obtain > a contiguous area of 14GB. But is a contiguous area of 14GB required for this? That is much memory! > The error messages show that MonetDB is pulling all the stops to try to get > this memory from the system. That is, a so-called "trim" buffer manager thread > tries to unload all unpinned other tables. To no avail here. You see the > actvity of the trim thread only when a GDKmalloc has failed already, and the > system tries again after unloading what it can. I'm now trying to import the relative 'smaller' integer only tables without references; but it ends up also in: #BBPTRIM_ENTER: memsize=180224,vmsize=4443275264 #BBPTRIM: memtarget=0 vmtarget=0 #BBPTRIM_EXIT: memsize=14925824,vmsize=4443275264 #BBPTRIM_ENTER: memsize=180224,vmsize=4443275264 #BBPTRIM: memtarget=0 vmtarget=0 #BBPTRIM_EXIT: memsize=14925824,vmsize=4443275264 #GDKmalloc(1374238248) fail => BBPtrim(enter) usage[mem=180224,vm=4443275264] #BBPTRIM_ENTER: memsize=180224,vmsize=4443275264 #BBPTRIM: memtarget=4611686018427387904 vmtarget=0 #BBPTRIM_EXIT: memsize=14925824,vmsize=4443275264 #GDKmalloc(1374238248) fail => BBPtrim(ready) usage[mem=180224,vm=4443275264] !ERROR: GDKmallocmax: failed for 1374238248 bytes !ERROR: GDKload: failed name=31/3141, ext=tail > The fact that though it tries to GDKmalloc you see the GDKload error message > indicates that it actually tries to get the 14GB by creating a file and memory > mapping on it. > > The cause for failure can thus be: > (1) your file system is full (test just after the error message appears, > because a MonetDB restarts first clears out all temp files it created on the > last run) MonetDB runs on 95GB of free data. Resulting in 16GB in MonetDB5, having inserted 19GB worth of textual content. > (2) you try to process such large loads on a 32-bts machine (naughty naughty) I'm not naughty, only have sick mind to try to run this amount of data on only 2GB of RAM. Linux srv1 2.6.27-gentoo-r4 #2 SMP Sat Nov 22 18:34:42 CET 2008 x86_64 Intel(R) Pentium(R) Dual CPU E2180 @ 2.00GHz GenuineIntel GNU/Linux Stefan ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
