Sjoerd,
this checkin seems to fix the following tests, while not harming any others
(on my 64-bit Fedora 10 desktop):
MonetDB4/src/gdk/malloc_mmap_priv.milS
MonetDB4/src/modules/plain/bat.py
sql/src/test/BugTracker-2009/union_leak.SF-2540169.SQL.sh
Stefan
On Thu, Aug 13, 2009 at 02:41:08PM +0000, Sjoerd Mullender wrote:
> Update of /cvsroot/monetdb/MonetDB/src/gdk
> In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv31891
>
> Modified Files:
> Tag: Aug2009
> gdk_heap.mx
> Log Message:
> Experimental checkin.
>
> Memory maps come in two flavors: normal and private
> (a.k.a. copy-on-write). The former are more efficient when saving
> since less data needs to be copied, but they can be a pain when
> dealing with pre-existing heaps, since then a backup needs to be made
> before starting to update the data.
>
> When switching from malloced to memory mapped allocation in a heap, we
> now see whether the heap file already existed, and if so, we use a
> private memory map (as before), but if the file did not exist, we now
> use a normal memory map. We don't need to make a backup first,
> because there is no pre-existing data to backup, so this should work.
>
> The result of this change is that loading data into fresh bats should
> be quicker. Since the new heaps are now "normal" memory maps, less
> copying takes place when the data is saved. A quick test suggests
> that this is indeed the case: the loading part of the 15M Periscope
> test took longer than the timeout (1800 seconds) allowed without this
> change, but was finished in about 1000 seconds with the change.
>
> Tonight's testing will hopefully show whether this idea does indeed
> work.
>
>
> U gdk_heap.mx
> Index: gdk_heap.mx
> ===================================================================
> RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_heap.mx,v
> retrieving revision 1.125
> retrieving revision 1.125.2.1
> diff -u -d -r1.125 -r1.125.2.1
> --- gdk_heap.mx 25 Jul 2009 21:51:40 -0000 1.125
> +++ gdk_heap.mx 13 Aug 2009 14:41:05 -0000 1.125.2.1
> @@ -206,13 +206,26 @@
> }
> /* too big: convert it to a disk-based temporary heap */
> if (can_mmap) {
> - FILE *fp = GDKfilelocate(nme, "wb", ext);
> + FILE *fp;
> char *of = h->filename;
> + int existing = 0;
> +
> + /* if the heap file already exists, we want to
> + switch to STORE_PRIV (copy-on-write memory
> + mapped files), but if the heap file doesn't
> + exist yet, the BAT is new and we can use
> + STORE_MMAP */
> + fp = GDKfilelocate(nme, "rb", ext);
> + if (fp != NULL) {
> + existing = 1;
> + fclose(fp);
> + }
> h->filename = NULL;
> + fp = GDKfilelocate(nme, "wb", ext);
> if (fp != NULL) {
> fclose(fp);
> if (h->storage == STORE_MEM) {
> - int newmode = (h->newstorage !=
> STORE_MMAP)?h->newstorage:STORE_PRIV;
> + int newmode = h->newstorage ==
> STORE_MMAP && existing ? STORE_PRIV : h->newstorage;
> /* make sure we realy MMAP */
> if (must_mmap && h->newstorage ==
> STORE_MEM)
> newmode = STORE_MMAP;
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Monetdb-checkins mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
>
>
>
>
>
--
| Dr. Stefan Manegold | mailto:[email protected] |
| CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
| 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
| The Netherlands | Fax : +31 (20) 592-4312 |
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Monetdb-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-developers
