Changeset: dc8066d67faa for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=dc8066d67faa
Modified Files:
gdk/gdk_utils.mx
Branch: Dec2011
Log Message:
GDKinit: align GDK_mem_bigsize with GDK_mmap_minsize
(on non-Windows systems, only!)
GDK_mmap_minsize is the limit as of which HEAPalloc()
switches to memory mapping rather than simple GDKmalloc().
GDK_mem_bigsize is the limit as of which GDKmalloc()
switches to virtual memory rather than simple malloc().
Currently,
GDK_mmap_minsize == 256 * 1024 * 1024 (i.e., 256 MB), and
GDK_mem_bigsize == 1 << 20 (i.e., 1 MB).
Intensive experimentation with TPCH, SSBM and other workloads
revealed that --- at least on Fedora Linux --- having GDKmalloc()
switch to virtual memory rather than simple malloc() as early as
with chunks as of 1 MB is a severe performance killer, as (explicite)
use of virtual memory appears to always require I/O, even if there
is ample physical memory.
Increasing GDK_mem_bigsize to GDK_mmap_minsize solves the problem
by effectively avoiding (anonymous) virtual memory completely.
The performance benefit with TPCH & SSBM size that fit in memory
is a factor 2 to 4 .
diffs (24 lines):
diff --git a/gdk/gdk_utils.mx b/gdk/gdk_utils.mx
--- a/gdk/gdk_utils.mx
+++ b/gdk/gdk_utils.mx
@@ -1675,7 +1675,11 @@ GDKinit(opt *set, int setlen)
/* Mserver by default takes 80% of all memory as a default */
GDK_mem_maxsize = GDK_mem_maxsize_max = (size_t) ((double) MT_npages()
* (double) MT_pagesize() * 0.815);
+#ifdef NATIVE_WIN32
GDK_mmap_minsize = GDK_mem_maxsize_max;
+#else
+ GDK_mmap_minsize = GDK_mem_bigsize = GDK_mem_maxsize_max;
+#endif
GDKmemchk(TRUE, TRUE);
GDKremovedir(DELDIR);
BBPinit();
@@ -1750,7 +1754,7 @@ GDKinit(opt *set, int setlen)
#else
/* WARNING: This unconditionally overwrites above settings, */
/* incl. setting via MonetDB env. var. "gdk_mmap_minsize" ! */
- GDK_mmap_minsize = 256 * 1024 * 1024;
+ GDK_mmap_minsize = GDK_mem_bigsize = 256 * 1024 * 1024;
#endif
#ifdef HAVE_POSIX_FADVISE
_______________________________________________
Checkin-list mailing list
[email protected]
http://mail.monetdb.org/mailman/listinfo/checkin-list
