On Thu, Mar 05, 2009 at 10:26:17AM +0100, Martin Kersten wrote:
> Sjoerd Mullender wrote:
> > How is this not a new feature?
> Oops, yes, unless Stefan wants this in for McK
As I said, this was/is just an idea, has low(est) priority (for me),
effects/benefits are (at least to me) yet unknown, is not crucial for McK
--- hence, yes, it is a new feature that should be only in the development
trunk --- I can later take care of revoming it from the release branch ...
Stefan
> > Martin Kersten wrote:
> >> Update of /cvsroot/monetdb/MonetDB5/src/optimizer
> >> In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv2998/optimizer
> >>
> >> Modified Files:
> >> Tag: Feb2009
> >> Makefile.ag opt_prelude.mx
> >> Added Files:
> >> Tag: Feb2009
> >> opt_derivepath.mx
> >> Log Message:
> >> Added the framework for the commissioned derivepath optimizer.
> >>
> >>
> >> U opt_prelude.mx
> >> Index: opt_prelude.mx
> >> ===================================================================
> >> RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_prelude.mx,v
> >> retrieving revision 1.55.2.3
> >> retrieving revision 1.55.2.4
> >> diff -u -d -r1.55.2.3 -r1.55.2.4
> >> --- opt_prelude.mx 1 Mar 2009 19:15:30 -0000 1.55.2.3
> >> +++ opt_prelude.mx 5 Mar 2009 00:17:12 -0000 1.55.2.4
> >> @@ -57,6 +57,7 @@
> >> opt_export str deleteRef;
> >> opt_export str depositRef;
> >> opt_export str deriveRef;
> >> +opt_export str derivePathRef;
> >> opt_export str differenceRef;
> >> opt_export str divRef;
> >> opt_export str disconnectRef;
> >> @@ -216,6 +217,7 @@
> >> str deleteRef;
> >> str depositRef;
> >> str deriveRef;
> >> +str derivePathRef;
> >> str differenceRef;
> >> str divRef;
> >> str disconnectRef;
> >> @@ -371,6 +373,7 @@
> >> deleteRef = putName("delete",6);
> >> depositRef = putName("deposit",7);
> >> deriveRef = putName("derive",6);
> >> + derivePathRef = putName("derivePath",10);
> >> differenceRef= putName("difference",10);
> >> divRef = putName("/",1);
> >> disconnectRef= putName("disconnect",10);
> >>
> >> U Makefile.ag
> >> Index: Makefile.ag
> >> ===================================================================
> >> RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/Makefile.ag,v
> >> retrieving revision 1.66
> >> retrieving revision 1.66.2.1
> >> diff -u -d -r1.66 -r1.66.2.1
> >> --- Makefile.ag 18 Jan 2009 11:44:14 -0000 1.66
> >> +++ Makefile.ag 5 Mar 2009 00:17:12 -0000 1.66.2.1
> >> @@ -32,7 +32,7 @@
> >> opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \
> >> opt_accumulators.mx opt_qep.mx opt_mergetable.mx \
> >> opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \
> >> - opt_evaluate.mx opt_inline.mx opt_pushranges.mx \
> >> + opt_evaluate.mx opt_inline.mx opt_pushranges.mx
> >> opt_derivepath.mx \
> >> opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx
> >> opt_remap.mx \
> >> opt_statistics.mx opt_trace.mx opt_recycler.mx opt_dataflow.mx
> >> \
> >> opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx
> >> @@ -53,7 +53,7 @@
> >> opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \
> >> opt_accumulators.mx opt_qep.mx opt_mergetable.mx \
> >> opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \
> >> - opt_evaluate.mx opt_inline.mx opt_pushranges.mx \
> >> + opt_evaluate.mx opt_inline.mx opt_pushranges.mx
> >> opt_derivepath.mx \
> >> opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx
> >> opt_remap.mx \
> >> opt_statistics.mx opt_trace.mx opt_recycler.mx opt_dataflow.mx
> >> \
> >> opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx
> >> @@ -72,7 +72,7 @@
> >> opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \
> >> opt_accumulators.mx opt_qep.mx opt_mergetable.mx \
> >> opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \
> >> - opt_evaluate.mx opt_inline.mx opt_pushranges.mx \
> >> + opt_evaluate.mx opt_inline.mx opt_pushranges.mx
> >> opt_derivepath.mx \
> >> opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx
> >> opt_remap.mx \
> >> opt_statistics.mx opt_trace.mx opt_recycler.mx opt_dataflow.mx
> >> \
> >> opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx
> >>
> >> --- NEW FILE: opt_derivepath.mx ---
> >> @' The contents of this file are subject to the MonetDB Public License
> >> @' Version 1.1 (the "License"); you may not use this file except in
> >> @' compliance with the License. You may obtain a copy of the License at
> >> @' http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html
> >> @'
> >> @' Software distributed under the License is distributed on an "AS IS"
> >> @' basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
> >> @' License for the specific language governing rights and limitations
> >> @' under the License.
> >> @'
> >> @' The Original Code is the MonetDB Database System.
> >> @'
> >> @' The Initial Developer of the Original Code is CWI.
> >> @' Portions created by CWI are Copyright (C) 1997-July 2008 CWI.
> >> @' Copyright August 2008-2009 MonetDB B.V.
> >> @' All Rights Reserved.
> >>
> >> @f opt_derivepath
> >> @- Group derive paths
> >> The routine @sc{optimizer.derivepaths()}
> >> walks through the program looking for grouping operations
> >> and cascades them into a single multiple group paths.
> >> To illustrate, consider
> >> @verbatim
> >> (ext1,grp1) := group.new(b);
> >> (ext2,grp2) := group.derive(ext1,grp1, c);
> >> (ext3,grp3) := group.derive(ext2,grp2, d);
> >> @end verbatim
> >> The result becomes.
> >> @verbatim
> >> (ext3,grp3) := group.derivepath(b,c,d);
> >> @end verbatim
> >> @{
> >> @mal
> >> pattern optimizer.derivePath():str
> >> address OPTderivePath;
> >> pattern optimizer.derivePath(mod:str, fcn:str):str
> >> address OPTderivePath
> >> comment "Join path constructor";
> >> pattern group.derivePath(l:bat[:any,:any]...):bat[:any,:any]
> >> address ALGderivePath
> >> comment "internal routine to handle join paths.
> >> The type analysis is rather tricky.";
> >> @h
> >> #ifndef _OPT_DERIVEPATH_
> >> #define _OPT_DERIVEPATH_
> >> #include "opt_prelude.h"
> >> #include "opt_support.h"
> >> #include "mal_interpreter.h"
> >>
> >> /* #define DEBUG_OPT_DERIVEPATH */
> >> @c
> >> #include "mal_config.h"
> >> #include "opt_derivepath.h"
> >>
> >> static int
> >> OPTderivePathImplementation(Client cntxt, MalBlkPtr mb, MalStkPtr stk,
> >> InstrPtr p)
> >> {
> >> int i, actions=0;
> >> int *pc;
> >> str derivePathRef = putName("derivePath",10);
> >> InstrPtr q;
> >> InstrPtr *old;
> >> int limit,slimit;
> >>
> >> (void) cntxt;
> >> (void) stk;
> >> if (varGetProp(mb, getArg(mb->stmt[0], 0), inlineProp) != NULL)
> >> return 0;
> >>
> >> old= mb->stmt;
> >> limit= mb->stop;
> >> slimit= mb->ssize;
> >> newMalBlkStmt(mb,mb->ssize);
> >>
> >> /* beware, new variables and instructions are introduced */
> >> pc= (int*) GDKmalloc(sizeof(int)* mb->vtop * 2); /* to find last
> >> assignment */
> >> memset((char*) pc, 0, sizeof(int)* mb->vtop *2);
> >>
> >> for (i = 0; i<limit; i++){
> >> p= old[i];
> >> if( getModuleId(p)== groupRef && getFunctionId(p)== newRef ){
> >> pc[getArg(p,0)] = i;
> >> pc[getArg(p,1)] = i;
> >> }
> >> if( getModuleId(p)== groupRef && getFunctionId(p)== deriveRef ){
> >> @-
> >> Try to expand its argument list with what we have found so far.
> >> This creates a series of join paths, many of which will be removed during
> >> deadcode elimination.
> >> @c
> >> if (pc[getArg(p,2)]== pc[getArg(p,3)]){
> >> q=
> >> copyInstruction(getInstrPtr(mb,pc[getArg(p,2)]));
> >> q= pushArgument(mb,q, getArg(p,5));
> >> pc[getArg(p,0)] = i;
> >> pc[getArg(p,1)] = i;
> >> setFunctionId(q,derivePathRef);
> >> p= q;
> >> actions++;
> >> #ifdef DEBUG_OPT_DERIVEPATH
> >> stream_printf(cntxt->fdout,"new derivePath
> >> instruction\n");
> >> printInstruction(cntxt->fdout,mb, 0, q,
> >> LIST_MAL_ALL);
> >> #endif
> >> }
> >> }
> >> pushInstruction(mb,p);
> >> }
> >> for(; i<slimit; i++)
> >> if(old[i])
> >> freeInstruction(old[i]);
> >> GDKfree(old);
> >> GDKfree(pc);
> >> OPTDEBUG
> >> if ( actions)
> >> stream_printf(cntxt->fdout,"#opt_derivepath: %d statements
> >> glued\n",actions);
> >> return actions;
> >> }
> >> @include optimizerWrapper.mx
> >> @h
> >> @:exportOptimizer(derivePath)@
> >> #endif
> >> opt_export str ALGderivePath(Client cntxt, MalBlkPtr mb, MalStkPtr stk,
> >> InstrPtr pci);
> >> @c
> >> #include "opt_statistics.h"
> >> @:wrapOptimizer(derivePath,OPT_CHECK_ALL)@
> >> @-
> >> The derive path optimizer takes a derivation sequence and
> >> attempts to minimize the intermediate result.
> >> The choice depends on a good estimate of intermediate
> >> results using properties.
> >> For the time being, we use a simplistic model, based
> >> on the assumption that most joins are foreign key joins anyway.
> >> @c
> >> str
> >> ALGderivePath(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci)
> >> {
> >> (void) cntxt;
> >> (void) mb;
> >> (void) stk;
> >> (void) pci;
> >> throw(MAL,"group.derivePath","nyi");
> >> }
> >> @}
> >>
> >>
> >> ------------------------------------------------------------------------------
> >> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco,
> >> CA
> >> -OSBC tackles the biggest issue in open source: Open Sourcing the
> >> Enterprise
> >> -Strategies to boost innovation and cut costs with open source
> >> participation
> >> -Receive a $600 discount off the registration fee with the source code:
> >> SFAD
> >> http://p.sf.net/sfu/XcvMzF8H
> >> _______________________________________________
> >> Monetdb-checkins mailing list
> >> [email protected]
> >> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> >
> >
> >
> > ------------------------------------------------------------------------
> >
> > ------------------------------------------------------------------------------
> > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> > -Strategies to boost innovation and cut costs with open source participation
> > -Receive a $600 discount off the registration fee with the source code: SFAD
> > http://p.sf.net/sfu/XcvMzF8H
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Monetdb-checkins mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Monetdb-checkins mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
>
--
| Dr. Stefan Manegold | mailto:[email protected] |
| CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
| 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
| The Netherlands | Fax : +31 (20) 592-4312 |
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Monetdb-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-developers
