Changeset: d0f7c8e1270a for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d0f7c8e1270a
Modified Files:
        gdk/gdk.h
        gdk/gdk_atoms.c
        gdk/gdk_cbp.c
        gdk/gdk_col.c
        gdk/gdk_colop.c
        gdk/gdk_qsort.c
        gdk/gdk_rangejoin.c
        gdk/gdk_relop.c
        gdk/gdk_search.c
        gdk/gdk_setop.c
        gdk/gdk_ssort.c
        gdk/gdk_utils.c
        gdk/gdk_value.c
        geom/monetdb5/geom.mx
        monetdb5/extras/compiler/Makefile.ag
        monetdb5/extras/compiler/mal_compiler.c
        monetdb5/extras/compiler/mal_compiler.h
        monetdb5/extras/compiler/mal_compiler.mal
        monetdb5/extras/crackers/Makefile.ag
        monetdb5/extras/crackers/crackers_AVL_index.mx
        monetdb5/extras/crackers/crackers_core_ordered.mx
        monetdb5/extras/crackers/crackers_core_unordered.mx
        monetdb5/extras/crackers/crackers_crackmerge.mx
        monetdb5/extras/crackers/crackers_index.mx
        monetdb5/extras/crackers/crackers_joins.mx
        monetdb5/extras/crackers/crackers_joinselect_ops.mx
        monetdb5/extras/crackers/crackers_partial_sideways.mx
        monetdb5/extras/crackers/crackers_populate.mx
        monetdb5/extras/crackers/crackers_pq.c
        monetdb5/extras/crackers/crackers_pq.h
        monetdb5/extras/crackers/crackers_select_ops.mx
        monetdb5/extras/crackers/crackers_sideways.mx
        monetdb5/extras/crackers/crackers_sortmerge.mx
        monetdb5/extras/crackers/crackers_updates.mx
        monetdb5/extras/crackers/crackers_validation.mx
        monetdb5/extras/rdf/Makefile.ag
        monetdb5/extras/rdf/rdf.h
        monetdb5/extras/rdf/rdf_shredder.mx
        monetdb5/extras/sphinx/Makefile.ag
        monetdb5/extras/sphinx/sphinx.c
        monetdb5/extras/sphinx/sphinx.h
        monetdb5/extras/sphinx/sphinx.mal
        monetdb5/mal/Makefile.ag
        monetdb5/mal/mal.c
        monetdb5/mal/mal.h
        monetdb5/mal/mal.mal
        monetdb5/mal/mal_atom.c
        monetdb5/mal/mal_authorize.c
        monetdb5/mal/mal_authorize.h
        monetdb5/mal/mal_box.c
        monetdb5/mal/mal_builder.c
        monetdb5/mal/mal_builder.h
        monetdb5/mal/mal_client.c
        monetdb5/mal/mal_debugger.c
        monetdb5/mal/mal_errors.h
        monetdb5/mal/mal_exception.c
        monetdb5/mal/mal_exception.h
        monetdb5/mal/mal_factory.c
        monetdb5/mal/mal_factory.h
        monetdb5/mal/mal_function.c
        monetdb5/mal/mal_function.h
        monetdb5/mal/mal_import.c
        monetdb5/mal/mal_instruction.c
        monetdb5/mal/mal_interpreter.c
        monetdb5/mal/mal_linker.c
        monetdb5/mal/mal_linker.h
        monetdb5/mal/mal_module.c
        monetdb5/mal/mal_namespace.c
        monetdb5/mal/mal_namespace.h
        monetdb5/mal/mal_parser.c
        monetdb5/mal/mal_profiler.c
        monetdb5/mal/mal_properties.c
        monetdb5/mal/mal_properties.h
        monetdb5/mal/mal_readline.c
        monetdb5/mal/mal_recycle.c
        monetdb5/mal/mal_recycle.h
        monetdb5/mal/mal_resolve.c
        monetdb5/mal/mal_sabaoth.c
        monetdb5/mal/mal_sabaoth.h
        monetdb5/mal/mal_scenario.c
        monetdb5/mal/mal_scenario.h
        monetdb5/mal/mal_session.c
        monetdb5/mal/mal_session.h
        monetdb5/mal/mal_stack.c
        monetdb5/mal/mal_stack.h
        monetdb5/mal/mal_type.c
        monetdb5/mal/mal_type.h
        monetdb5/mal/mal_utils.c
        monetdb5/mal/mal_utils.h
        monetdb5/mal/mal_xml.c
        monetdb5/mal/mal_xml.h
        monetdb5/modules/atoms/Makefile.ag
        monetdb5/modules/atoms/blob.c
        monetdb5/modules/atoms/blob.h
        monetdb5/modules/atoms/blob.mal
        monetdb5/modules/atoms/color.c
        monetdb5/modules/atoms/color.h
        monetdb5/modules/atoms/color.mal
        monetdb5/modules/atoms/identifier.c
        monetdb5/modules/atoms/identifier.h
        monetdb5/modules/atoms/identifier.mal
        monetdb5/modules/atoms/inet.c
        monetdb5/modules/atoms/inet.h
        monetdb5/modules/atoms/inet.mal
        monetdb5/modules/atoms/mtime.c
        monetdb5/modules/atoms/str.c
        monetdb5/modules/atoms/streams.c
        monetdb5/modules/atoms/url.c
        monetdb5/modules/atoms/xml.c
        monetdb5/modules/atoms/xml.h
        monetdb5/modules/atoms/xml.mal
        monetdb5/modules/atoms/xml.sql
        monetdb5/modules/kernel/Makefile.ag
        monetdb5/modules/kernel/aggr.mx
        monetdb5/modules/kernel/aggr_be_avg.mx
        monetdb5/modules/kernel/aggr_be_count.mx
        monetdb5/modules/kernel/aggr_be_minmax.mx
        monetdb5/modules/kernel/aggr_be_prod.mx
        monetdb5/modules/kernel/aggr_be_sum.mx
        monetdb5/modules/kernel/aggr_bge_avg.mx
        monetdb5/modules/kernel/aggr_bge_count.mx
        monetdb5/modules/kernel/aggr_bge_minmax.mx
        monetdb5/modules/kernel/aggr_bge_prod.mx
        monetdb5/modules/kernel/aggr_bge_sum.mx
        monetdb5/modules/kernel/alarm.c
        monetdb5/modules/kernel/alarm.h
        monetdb5/modules/kernel/alarm.mal
        monetdb5/modules/kernel/algebra.c.mx
        monetdb5/modules/kernel/algebra.h.mx
        monetdb5/modules/kernel/algebra.mal.mx
        monetdb5/modules/kernel/array.mx
        monetdb5/modules/kernel/calc.c.mx
        monetdb5/modules/kernel/calc.h.mx
        monetdb5/modules/kernel/calc.mal.mx
        monetdb5/modules/kernel/colcalc.c.mx
        monetdb5/modules/kernel/colcalc.h.mx
        monetdb5/modules/kernel/colcalc.mal.mx
        monetdb5/modules/kernel/colcast.c
        monetdb5/modules/kernel/colcast.mal.mx
        monetdb5/modules/kernel/colcolor.c.mx
        monetdb5/modules/kernel/colcolor.h.mx
        monetdb5/modules/kernel/colifthen.c.mx
        monetdb5/modules/kernel/colifthen.h.mx
        monetdb5/modules/kernel/colifthen.mal.mx
        monetdb5/modules/kernel/colmath.c.mx
        monetdb5/modules/kernel/colmath.h.mx
        monetdb5/modules/kernel/colmath.mal.mx
        monetdb5/modules/kernel/colmtime.c.mx
        monetdb5/modules/kernel/colmtime.h.mx
        monetdb5/modules/kernel/colmtime.mal.mx
        monetdb5/modules/kernel/colstr.c
        monetdb5/modules/kernel/column.c.mx
        monetdb5/modules/kernel/column.h.mx
        monetdb5/modules/kernel/column.mal.mx
        monetdb5/modules/kernel/counters.c
        monetdb5/modules/kernel/counters.h
        monetdb5/modules/kernel/counters.mal
        monetdb5/modules/kernel/group.mx
        monetdb5/modules/kernel/lock.c
        monetdb5/modules/kernel/lock.h
        monetdb5/modules/kernel/lock.mal
        monetdb5/modules/kernel/logger.c
        monetdb5/modules/kernel/logger.mal
        monetdb5/modules/kernel/microbenchmark.c
        monetdb5/modules/kernel/microbenchmark.h
        monetdb5/modules/kernel/microbenchmark.mal
        monetdb5/modules/kernel/mmath.c.mx
        monetdb5/modules/kernel/mmath.h.mx
        monetdb5/modules/kernel/mmath.mal.mx
        monetdb5/modules/kernel/sema.c
        monetdb5/modules/kernel/sema.mal
        monetdb5/modules/kernel/status.c
        monetdb5/modules/kernel/unix.c
        monetdb5/modules/kernel/unix.mal
        monetdb5/modules/mal/Makefile.ag
        monetdb5/modules/mal/algebraExtensions.c
        monetdb5/modules/mal/algebraExtensions.h
        monetdb5/modules/mal/algebraExtensions.mal
        monetdb5/modules/mal/attach.c
        monetdb5/modules/mal/attach.h
        monetdb5/modules/mal/attach.mal
        monetdb5/modules/mal/batExtensions.c
        monetdb5/modules/mal/batExtensions.mal
        monetdb5/modules/mal/cbp.c
        monetdb5/modules/mal/chopper.c
        monetdb5/modules/mal/chopper.mal
        monetdb5/modules/mal/clients.c
        monetdb5/modules/mal/constraints.c
        monetdb5/modules/mal/constraints.h
        monetdb5/modules/mal/constraints.mal
        monetdb5/modules/mal/factories.c
        monetdb5/modules/mal/groupby.c
        monetdb5/modules/mal/groupby.h
        monetdb5/modules/mal/groupby.mal
        monetdb5/modules/mal/histogram.c
        monetdb5/modules/mal/histogram.h
        monetdb5/modules/mal/histogram.mal
        monetdb5/modules/mal/inout.c
        monetdb5/modules/mal/inspect.c
        monetdb5/modules/mal/language.c
        monetdb5/modules/mal/mal_init.mal
        monetdb5/modules/mal/manual.c
        monetdb5/modules/mal/manual.h
        monetdb5/modules/mal/manual.mal
        monetdb5/modules/mal/mat.c
        monetdb5/modules/mal/mdb.c
        monetdb5/modules/mal/mkey.c
        monetdb5/modules/mal/mkey.h
        monetdb5/modules/mal/mkey.mal
        monetdb5/modules/mal/pcre.c
        monetdb5/modules/mal/pcre.mal
        monetdb5/modules/mal/pqueue.mx
        monetdb5/modules/mal/profiler.c
        monetdb5/modules/mal/profiler.h
        monetdb5/modules/mal/profiler.mal
        monetdb5/modules/mal/recycle.c
        monetdb5/modules/mal/recycle.h
        monetdb5/modules/mal/recycle.mal
        monetdb5/modules/mal/remote.c
        monetdb5/modules/mal/remote.h
        monetdb5/modules/mal/remote.mal
        monetdb5/modules/mal/replication.c
        monetdb5/modules/mal/sabaoth.c
        monetdb5/modules/mal/sabaoth.h
        monetdb5/modules/mal/sabaoth.mal
        monetdb5/modules/mal/tablet.c
        monetdb5/modules/mal/tablet_mk.c
        monetdb5/modules/mal/tablet_si.c
        monetdb5/modules/mal/tablet_sql.c
        monetdb5/modules/mal/tokenizer.c
        monetdb5/modules/mal/trader.c
        monetdb5/modules/mal/trader.h
        monetdb5/modules/mal/trader.mal
        monetdb5/modules/mal/transaction.c
        monetdb5/modules/mal/transaction.mal
        monetdb5/modules/mal/txtsim.c
        monetdb5/modules/mal/txtsim.h
        monetdb5/modules/mal/txtsim.mal
        monetdb5/modules/mal/xmlcolumn.c
        monetdb5/modules/mal/zorder.c
        monetdb5/optimizer/Makefile.ag
        monetdb5/optimizer/opt_aliases.c
        monetdb5/optimizer/opt_cluster.c
        monetdb5/optimizer/opt_coercion.c
        monetdb5/optimizer/opt_commonTerms.c
        monetdb5/optimizer/opt_compression.c
        monetdb5/optimizer/opt_costModel.c
        monetdb5/optimizer/opt_datacyclotron.c
        monetdb5/optimizer/opt_dataflow.c
        monetdb5/optimizer/opt_deadcode.c
        monetdb5/optimizer/opt_derivepath.c
        monetdb5/optimizer/opt_dictionary.c
        monetdb5/optimizer/opt_emptySet.c
        monetdb5/optimizer/opt_evaluate.c
        monetdb5/optimizer/opt_factorize.c
        monetdb5/optimizer/opt_garbagecollector.c
        monetdb5/optimizer/opt_history.c
        monetdb5/optimizer/opt_inline.c
        monetdb5/optimizer/opt_joinpath.c
        monetdb5/optimizer/opt_macro.c
        monetdb5/optimizer/opt_mapreduce.c
        monetdb5/optimizer/opt_mergetable.c
        monetdb5/optimizer/opt_mitosis.c
        monetdb5/optimizer/opt_multiplex.c
        monetdb5/optimizer/opt_octopus.c
        monetdb5/optimizer/opt_pipes.c
        monetdb5/optimizer/opt_pipes.h
        monetdb5/optimizer/opt_prejoin.c
        monetdb5/optimizer/opt_prelude.c
        monetdb5/optimizer/opt_prelude.h
        monetdb5/optimizer/opt_pushranges.c
        monetdb5/optimizer/opt_qep.c
        monetdb5/optimizer/opt_recycler.c
        monetdb5/optimizer/opt_reduce.c
        monetdb5/optimizer/opt_remap.c
        monetdb5/optimizer/opt_remoteQueries.c
        monetdb5/optimizer/opt_reorder.c
        monetdb5/optimizer/opt_replication.c
        monetdb5/optimizer/opt_singleton.c
        monetdb5/optimizer/opt_statistics.c
        monetdb5/optimizer/opt_statistics.h
        monetdb5/optimizer/opt_statistics.mal
        monetdb5/optimizer/opt_strengthReduction.c
        monetdb5/optimizer/opt_support.c
        monetdb5/optimizer/opt_support.h
        monetdb5/optimizer/opt_support.mal
        monetdb5/optimizer/opt_trace.c
        monetdb5/optimizer/optimizer.c
        monetdb5/scheduler/Makefile.ag
        monetdb5/scheduler/run_adder.c
        monetdb5/scheduler/run_adder.h
        monetdb5/scheduler/run_adder.mal
        monetdb5/scheduler/run_centipede.c
        monetdb5/scheduler/run_centipede.h
        monetdb5/scheduler/run_centipede.mal
        monetdb5/scheduler/run_centipede.mx
        monetdb5/scheduler/run_isolate.c
        monetdb5/scheduler/run_isolate.h
        monetdb5/scheduler/run_isolate.mal
        monetdb5/scheduler/run_memo.c
        monetdb5/scheduler/run_memo.h
        monetdb5/scheduler/run_memo.mal
        monetdb5/scheduler/run_octopus.c
        monetdb5/scheduler/run_octopus.h
        monetdb5/scheduler/run_octopus.mal
        monetdb5/scheduler/run_pipeline.mx
        sql/backends/monet5/sql.mx
        sql/backends/monet5/sql_result.mx
Branch: headless
Log Message:

Merge with default branch.


diffs (truncated from 10008 to 300 lines):

diff --git a/geom/monetdb5/geom.mx b/geom/monetdb5/geom.mx
--- a/geom/monetdb5/geom.mx
+++ b/geom/monetdb5/geom.mx
@@ -17,15 +17,16 @@
 All Rights Reserved.
 @
 
-@f geom
-@a Wouter Scherphof, Niels Nes
-@* The simple geom module
-
+@c
+/*
+ * @f geom
+ * @a Wouter Scherphof, Niels Nes
+ * @* The simple geom module
+ */
 @mal
 module geom;
 
-@' overwrite lng needed for fixed size! Sizes are fixed in geom.prelude
-@mal
+# @' overwrite lng needed for fixed size! Sizes are fixed in geom.prelude
 atom mbr:lng;
 
 command tostr() :str           address mbrTOSTR;
@@ -64,6 +65,7 @@
        return x;
 end @1FromText;
 
+@
 @mal
 @:types(Point,point,1)@
 @:types(Curve,curve,2)@
@@ -92,13 +94,12 @@
 address wkbMBR
 comment "Creates the mbr for the given wkb.";
 
-@+
-command mbr(:col[:wkb]) :col[:mbr]
-address bat_wkbMBR
-comment "Creates the mbr's for the given wkb's.";
-
-@+ ogc basic methods
-@mal
+# @+
+# command mbr(:col[:wkb]) :col[:mbr]
+# address bat_wkbMBR
+# comment "Creates the mbr's for the given wkb's.";
+#
+# @+ ogc basic methods
 command Dimension(w:wkb) :int
 address wkbDimension
 comment "The inherent dimension of this geometry object, which must be less 
than or equal to the coordinate dimension. This specification is restricted to 
geometries in two-dimensional coordinate space.";
@@ -127,8 +128,7 @@
 address wkbBoundary
 comment "Returns the closure of the combinatorial boundary of this Geometry.";
 
-@+ ogc spatial relation methods
-@mal
+# @+ ogc spatial relation methods
 
 command Equals(a:wkb, b:wkb) :bit
 address wkbEquals
@@ -166,8 +166,7 @@
 address wkbRelate
 comment "Returns true if the Geometry a 'spatially related' to Geometry b, by 
testing for intersection between the Interior, Boundary and Exterior of the two 
geometries as specified by the values in the intersectionPatternMatrix.";
 
-@+ ogc constructor functions
-@mal
+# @+ ogc constructor functions
 command point(x:dbl,y:dbl) :wkb
 address wkbcreatepoint
 comment "Construct a point from two geometries";
@@ -178,8 +177,7 @@
 comment "Construct a point-BAT from two geometry-BATs";
 module geom;
 
-@+ ogc information methods
-@mal
+# @+ ogc information methods
 command X(g:wkb) :dbl
 address wkbgetcoordX
 comment "Returns the x-coordinate of a point geometry.";
@@ -188,8 +186,7 @@
 address wkbgetcoordY
 comment "Returns the y-coordinate of a point geometry.";
 
-@+ ogc Spatial Analysis methods
-@mal
+# @+ ogc Spatial Analysis methods
 
 command Area(g:wkb) :dbl
 address wkbArea
@@ -245,6 +242,7 @@
        x := wkb.FromText(wkt,@3);
        return x;
 end @2;
+@
 @mal
 @:ctypes(Point,point,1)@
 @:ctypes(Curve,curve,2)@
@@ -955,11 +953,11 @@
 
        return ret;
 }
+@
 @c
 @:getcoord(X)@
 @:getcoord(Y)@
 
-@c
 geom_export str wkbcreatepoint(wkb **out, dbl *x, dbl *y);
 
 str
@@ -1091,6 +1089,7 @@
        return MAL_SUCCEED;
 
 }
+@
 @c
 @:basic(Dimension,int,GEOSGeom_getDimensions,,0,1,*out=int_nil)@
 @:basic(GeometryTypeId,int,GEOSGeomTypeId,,-1,1,*out=int_nil)@
@@ -1162,6 +1161,7 @@
 
        return MAL_SUCCEED;
 }
+@
 @c
 @:spatial(Equals,GEOSEquals)@
 @:spatial(Disjoint,GEOSDisjoint)@
@@ -1265,6 +1265,7 @@
 
        throw(MAL, "geom.@1", "@2 failed");
 }
+@
 @c
 @:analysis(Area,GEOSArea)@
 @:analysis(Length,GEOSLength)@
@@ -1275,7 +1276,6 @@
 @:analysis3(Difference,GEOSDifference,geos2wkb)@
 @:analysis3(SymDifference,GEOSSymDifference,geos2wkb)@
 
-@c
 geom_export str wkbBuffer(wkb **out, wkb **geom, dbl *distance);
 
 str
diff --git a/monetdb5/extras/crackers/crackers.mx 
b/monetdb5/extras/crackers/crackers.mx
--- a/monetdb5/extras/crackers/crackers.mx
+++ b/monetdb5/extras/crackers/crackers.mx
@@ -17,103 +17,104 @@
 All Rights Reserved.
 @
 
-@f crackers
-@a Martin Kersten, Stratos Idreos, Stefan Manegold
-@d March 2006 - July 2007
-@* Cracker index
-
-Database cracking aims at enhancing query processing by adapting the way 
-data is stored to the query workload. Adaptation happens through physical 
-reorganization of data. Each query reorganized the data in such a way that 
-future queries can be answered faster.
-
-For each BAT b, the system creates on demand a cracker column c, i.e., 
-the first time a select operator is fired on b. From there on all queries that 
-perform a select operator on b will physically reorganize (i.e., crack) the 
cracker column c 
-such that the result set is in a consecutive area in c. 
-The result of the select operator is a non materialized view on c.
-
-For each cracker column c, there exists a cracker index, that maintains 
knowledge 
-on the various pieces created while cracking c. The cracker index is an 
AVLt-tree
-data structure. Each select operator uses the appropriate cracker index to 
determine 
-which pieces it should crack, and updates the index afterwards.
-
-The more queries are fired on a bat b the more the cracker column c of b is 
cracked.
-This way the cracker index on c, is continuously enriched with information 
that helps speed up 
-future queries. This way query processing improves as more queries are 
answered by adapting to 
-the query and data workload.
-
-Initial performance shows an exptected gain of at least on order of magnitude, 
-due to analysing less data during the select operator and also due to non 
materializing 
-the select operator result set.
-
-The cracker index can be created for the time being on
-tail columns of type {int,lng,dbl,flt,time,date,sht}. The head
-is always of type @sc{oid}.
-
-In addition, the cracker index can be used to feed a generator
-for pieces satisfying a range constraint.
-A cracker partition is indicated by the index in the cidx
-table. It can be used to initialize a BATview to represent
-the partition during processing.
-
-Updates support for cracker columns happens again in a self-organizing way.
-Incoming updates (inserts,deletes or updates) are kept in separate columns for 
-each BAT as pending updates. While processing queries, if the crackers select 
-operator detects that some of the pending updates should be taken into account,
-then updates are "merged" into the cracker column on the fly by also updating 
the 
-cracker index accordingly.
-
-Since the cracker columns are physically reorganized the result of a select 
operator is not 
-ordered in the same way as the rest of the bats of the same relation. This has 
side-effects 
-during projection operations. We cannot use ordered fetch joins anymore. 
-This decrades performance significantly
-when multiple projections exist in a query plan.
-
-This problem is solved by the second generation cracing, i.e., 
-sideways cracking that uses cracker maps instead of cracker columns.
-A cracker map contains the values of two attributes of the same relation. 
-These are attributes used together in query plans. The main idea is that 
-cracking based on one of these attributes (the head of the map) is forwarded 
on the tail. 
-This way projections can be performed by performing simple cracking operations 
on maps.
-Multiple maps can exist for each attribute A as head consisting the map set of 
A. All maps in the same 
-map set S are aligned before used in a query plan. Aligment happens by keeping 
a history of all cracking 
-operations in maps of S and ensuring that the same cracks have been applied to 
all necessary maps 
-and in the same order. Sideways cracking maintains the advantage of cracking 
even 
-in the presence of multiple projection operations.
-
-Ongoing work.
-Multitable cracking.
-
-The cracker module should be prepared to deal with any of the base types.
-For strings this becomes an issue.
-
-The current implementation uses an unprotected cracker
-index. This limits the interface at slightly more overhead
-of searching the cracker index upon each call.
-
-Currently crakcer structures are in memory. They need to be stored in disk and 
become persistent.
-
-Repeated cracking leads to an ever growing index. This process
-can be stopped by setting the granule size,
-i.e.  the minimum number of tuples in each piece,
-or the maximum number of pieces.
-
-The implementation is geared at early experimentation
-without all the details to make the code robust and
-ultra fast.
-
-
-@+ Interface
-
-
-@- Type expansion
-
+@c
+/*
+ * @f crackers
+ * @a Martin Kersten, Stratos Idreos, Stefan Manegold
+ * @d March 2006 - July 2007
+ * @* Cracker index
+ *
+ * Database cracking aims at enhancing query processing by adapting the way
+ * data is stored to the query workload. Adaptation happens through physical
+ * reorganization of data. Each query reorganized the data in such a way that
+ * future queries can be answered faster.
+ *
+ * For each BAT b, the system creates on demand a cracker column c, i.e.,
+ * the first time a select operator is fired on b. From there on all queries 
that
+ * perform a select operator on b will physically reorganize (i.e., crack) the 
cracker column c
+ * such that the result set is in a consecutive area in c.
+ * The result of the select operator is a non materialized view on c.
+ *
+ * For each cracker column c, there exists a cracker index, that maintains 
knowledge
+ * on the various pieces created while cracking c. The cracker index is an 
AVLt-tree
+ * data structure. Each select operator uses the appropriate cracker index to 
determine
+ * which pieces it should crack, and updates the index afterwards.
+ *
+ * The more queries are fired on a bat b the more the cracker column c of b is 
cracked.
+ * This way the cracker index on c, is continuously enriched with information 
that helps speed up
+ * future queries. This way query processing improves as more queries are 
answered by adapting to
+ * the query and data workload.
+ *
+ * Initial performance shows an exptected gain of at least on order of 
magnitude,
+ * due to analysing less data during the select operator and also due to non 
materializing
+ * the select operator result set.
+ *
+ * The cracker index can be created for the time being on
+ * tail columns of type {int,lng,dbl,flt,time,date,sht}. The head
+ * is always of type @sc{oid}.
+ *
+ * In addition, the cracker index can be used to feed a generator
+ * for pieces satisfying a range constraint.
+ * A cracker partition is indicated by the index in the cidx
+ * table. It can be used to initialize a BATview to represent
+ * the partition during processing.
+ *
+ * Updates support for cracker columns happens again in a self-organizing way.
+ * Incoming updates (inserts,deletes or updates) are kept in separate columns 
for
+ * each BAT as pending updates. While processing queries, if the crackers 
select
+ * operator detects that some of the pending updates should be taken into 
account,
+ * then updates are "merged" into the cracker column on the fly by also 
updating the
+ * cracker index accordingly.
+ *
+ * Since the cracker columns are physically reorganized the result of a select 
operator is not
+ * ordered in the same way as the rest of the bats of the same relation. This 
has side-effects
+ * during projection operations. We cannot use ordered fetch joins anymore.
+ * This decrades performance significantly
+ * when multiple projections exist in a query plan.
_______________________________________________
Checkin-list mailing list
[email protected]
http://mail.monetdb.org/mailman/listinfo/checkin-list

Reply via email to