On Sat, Feb 06, 2010 at 10:17:54AM +0000, Martin Kersten wrote:
> Update of /cvsroot/monetdb/MonetDB5/src/modules/kernel
> In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv685
>
> Modified Files:
> Tag: Feb2010
> batcast.mx batstr.mx
> Log Message:
> All string BATs should be preloaded to avoid random access to disk.
Preloading their string heap(s) is indeed a good idea (unless they exceeds
memory size, anyway) due to their inherently random access.
However, preloading sequentially traversed BUN heaps might just create an
unnecessarily huge memory footprint, in particular with large BATs.
Hence, we might want to consider extending BATpreload to not uncoditionally
preload all heaps of a BAT (up to four right now, since we ignore hash
tables),
but rather allow selective preload of those heaps that are (a) accessed at
all, and (b) access randomly (or not accessed sequentially).
I'll look into this later ...
Stefan
>
> Index: batstr.mx
> ===================================================================
> RCS file: /cvsroot/monetdb/MonetDB5/src/modules/kernel/batstr.mx,v
> retrieving revision 1.68
> retrieving revision 1.68.2.1
> diff -u -d -r1.68 -r1.68.2.1
> --- batstr.mx 7 Jan 2010 15:24:26 -0000 1.68
> +++ batstr.mx 6 Feb 2010 10:17:49 -0000 1.68.2.1
> @@ -239,6 +239,7 @@
>
> bi = bat_iterator(b);
>
> + BATpreload(b);
> BATloop(b, p, q) {
> ptr h = BUNhead(bi,p);
> x = (@3) BUNtail(bi,p);
> @@ -275,6 +276,7 @@
>
> bi = bat_iterator(b);
>
> + BATpreload(b);
> BATloop(b, p, q) {
> ptr h = BUNhead(bi,p);
> str y = (str)str_nil, *yp = &y;
> @@ -322,6 +324,7 @@
>
> if (strcmp(*cst, str_nil) == 0)
> isnil = 1;
> + BATpreload(b);
> BATloop(b, p, q) {
> ptr h = BUNhead(bi, p);
>
> @@ -369,6 +372,8 @@
> lefti = bat_iterator(left);
> righti = bat_iterator(right);
>
> + BATpreload(left);
> + BATpreload(right);
> BATloop(left, p, q) {
> bit y = TRUE;
> ptr h = BUNhead(lefti, p);
> @@ -411,6 +416,8 @@
> lefti = bat_iterator(left);
> righti = bat_iterator(right);
>
> + BATpreload(left);
> + BATpreload(right);
> BATloop(left, p, q) {
> ptr h = BUNhead(lefti,p);
> ptr tl = BUNtail(lefti,p);
> @@ -444,6 +451,7 @@
>
> lefti = bat_iterator(left);
>
> + BATpreload(left);
> BATloop(left, p, q) {
> ptr h = BUNhead(lefti,p);
> ptr tl = BUNtail(lefti,p);
> @@ -476,6 +484,8 @@
> lefti = bat_iterator(left);
> righti = bat_iterator(right);
>
> + BATpreload(left);
> + BATpreload(right);
> BATloop(left, p, q) {
> ptr h = BUNhead(lefti,p);
> str tl = (str) BUNtail(lefti,p);
> @@ -509,6 +519,7 @@
>
> lefti = bat_iterator(left);
>
> + BATpreload(left);
> BATloop(left, p, q) {
> ptr h = BUNhead(lefti,p);
> str tl = (str) BUNtail(lefti,p);
> @@ -539,6 +550,7 @@
>
> righti = bat_iterator(right);
>
> + BATpreload(right);
> BATloop(right, p, q) {
> ptr h = BUNhead(righti,p);
> str tr = (str) BUNtail(righti,p);
> @@ -579,6 +591,7 @@
>
> bi = bat_iterator(b);
>
> + BATpreload(b);
> BATloop(b, p, q) {
> ptr h = BUNhead(bi, p);
> str y = (str)str_nil, *yp = &y;
> @@ -622,6 +635,7 @@
>
> bi = bat_iterator(b);
>
> + BATpreload(b);
> BATloop(b, p, q) {
> ptr h = BUNhead(bi, p);
> ptr t = BUNtail(bi, p);
> @@ -666,6 +680,7 @@
> bn->tsorted = b->tsorted;
>
> bi = bat_iterator(b);
> + BATpreload(b);
> BATloop(b, p, q) {
> ptr h = BUNhead(bi, p);
> str t = (str) BUNtail(bi, p);
> @@ -720,6 +735,7 @@
> lefti = bat_iterator(left);
> starti = bat_iterator(start);
> lengthi = bat_iterator(length);
> + BATpreload(left);
> BATloop(left, p, q) {
> ptr h = BUNhead(lefti,p);
> str tl = (str) BUNtail(lefti,p);
> @@ -764,6 +780,7 @@
> bn->tsorted=0;
>
> li = bat_iterator(left);
> + BATpreload(left);
> BATloop(left, p, q) {
> ptr h = BUNhead(li,p);
> str tl = (str) BUNtail(li,p);
>
> Index: batcast.mx
> ===================================================================
> RCS file: /cvsroot/monetdb/MonetDB5/src/modules/kernel/batcast.mx,v
> retrieving revision 1.31
> retrieving revision 1.31.2.1
> diff -u -d -r1.31 -r1.31.2.1
> --- batcast.mx 7 Jan 2010 15:24:26 -0000 1.31
> +++ batcast.mx 6 Feb 2010 10:17:47 -0000 1.31.2.1
> @@ -253,6 +253,7 @@
> @:voidresultBAT(ty...@1,"batca...@1")@
> bi = bat_iterator(b);
>
> + BATpreload(b);
> BATloop(b, p, q) {
> ptr v = NULL;
> int len = 0;
>
>
> ------------------------------------------------------------------------------
> The Planet: dedicated and managed hosting, cloud storage, colocation
> Stay online with enterprise data centers and the best network in the business
> Choose flexible plans and management services without long-term contracts
> Personal 24x7 support from experience hosting pros just a phone call away.
> http://p.sf.net/sfu/theplanet-com
> _______________________________________________
> Monetdb-checkins mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
>
--
| Dr. Stefan Manegold | mailto:[email protected] |
| CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
| 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
| The Netherlands | Fax : +31 (20) 592-4199 |
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Monetdb-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-developers
