Changeset: 9f8079511cf0 for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=9f8079511cf0
Modified Files:
monetdb5/mal/mal.mx
monetdb5/mal/mal_namespace.mx
monetdb5/mal/mal_parser.mx
monetdb5/mal/mal_profiler.mx
monetdb5/mal/mal_properties.mx
monetdb5/mal/mal_readline.mx
monetdb5/mal/mal_recycle.mx
monetdb5/mal/mal_resolve.mx
monetdb5/optimizer/Makefile.ag
monetdb5/optimizer/opt_constants.mx
Branch: headless
Log Message:
Half way monetdb5/mal directory
Mostly simple preparation for .c and .h split.
Type resolution aimed now for COL types.
diffs (truncated from 4806 to 300 lines):
diff --git a/monetdb5/mal/mal.mx b/monetdb5/mal/mal.mx
--- a/monetdb5/mal/mal.mx
+++ b/monetdb5/mal/mal.mx
@@ -17,164 +17,7 @@
All Rights Reserved.
@
-@f mal
-@-
-@node Design Considerations, Architecture Overview, Design Overview, Design
Overview
-@+ Design Considerations
-Redesign of the MonetDB software stack was driven by the need to
-reduce the effort to extend the system into novel directions
-and to reduce the Total Execution Cost (TEC).
-The TEC is what an end-user or application program will notice.
-The TEC is composed on several cost factors:
-@itemize
-@item A)
-API message handling
-@item P)
-Parsing and semantic analysis
-@item O)
-Optimization and plan generation
-@item D)
-Data access to the persistent store
-@item E)
-Execution of the query terms
-@item R)
-Result delivery to the application
-@end itemize
-
-Choosing an architecture for processing database operations pre-supposes an
-intuition on how the cost will be distributed. In an OLTP
-setting you expect most of the cost to be in (P,O), while in OLAP it will
-be (D,E,R). In a distributed setting the components (O,D,E) are dominant.
-Web-applications would focus on (A,E,R).
-
-Such a simple characterization ignores the wide-spread
-differences that can be experienced at each level. To illustrate,
-in D) and R) it makes a big difference whether the data is already in the
-cache or still on disk. With E) it makes a big difference whether you
-are comparing two integers, evaluation of a mathematical function,
-e.g., Gaussian, or a regular expression evaluation on a string.
-As a result, intense optimization in one area may become completely invisible
-due to being overshadowed by other cost factors.
-
-The Version 5 infrastructure is designed to ease addressing each
-of these cost factors in a well-defined way, while retaining the
-flexibility to combine the components needed for a particular situation.
-It results in an architecture where you assemble the components
-for a particular application domain and hardware platform.
-
-The primary interface to the database kernel is still based on
-the exchange of text in the form of queries and simply formatted results.
-This interface is designed for ease of interpretation, versatility and
-is flexible to accommodate system debugging and application tool development.
-Although a textual interface potentially leads to a performance degradation,
-our experience with earlier system versions
-showed that the overhead can be kept within acceptable bounds.
-Moreover, a textual interface reduces the programming
-effort otherwise needed to develop test and application programs.
-The XML trend as the language for tool interaction supports our decision.
-@-
-@node Architecture Overview, MAL Synopsis, Design Considerations, Design
Overview
-@+ Architecture Overview
-The architecture is built around a few independent components:
-the MonetDB server, the merovigian, and the client application.
-The MonetDB server is the heart of the system, it manages a single
-physical database on one machine for all (concurrent) applications.
-The merovigian program works along side a single server, keeping
-an eye on its behavior. If the server accidently crashes, it is this program
-that will attempt an automatic restart.
-
-The top layer consists of applications written in your favorite
-language.
-They provide both specific functionality
-for a particular product, e.g.,
@url{http://kdl.cs.umass.edu/software,Proximity},
-and generic functionality, e.g.,
-the @url{http://www.aquafold.com,Aquabrowser} or
@url{http://www.minq.se,Dbvisualizer}.
-The applications communicate with the server
-using de-facto standard interface packaged,
-i.e., JDBC, ODBC, Perl, PHP, etc.
-
-The middle layer consists of query language processors such as
-SQL and XQuery. The former supports the core functionality
-of SQL'99 and extends into SQL'03. The latter is based on
-the W3C standard and includes the XUpdate functionality.
-The query language processors each manage their own private catalog structure.
-Software bridges, e.g., import/export routines, are used to
-share data between language paradigms.
-
-@iftex
-@image{base00,,,,.pdf}
-@emph{Figure 2.1}
-@end iftex
-@-
-@node MAL Synopsis, Execution Engine, Architecture Overview, Design Overview
-@+ MonetDB Assembly Language (MAL)
-The target language for a query compiler is
-the MonetDB Assembly Language (MAL).
-It was designed to ease code generation
-and fast interpretation by the server.
-The compiler produces algebraic query plans, which
-are turned into physical execution
-plans by the MAL optimizers.
-
-The output of a compiler is either an @sc{ascii} representation
-of the MAL program or the compiler is tightly coupled with
-the server to save parsing and communication overhead.
-
-A snippet of the MAL code produced by the SQL compiler
-for the query @sc{select count(*) from tables}
-is shown below. It illustrates a sequences of relational
-operations against a table column and producing a
-partial result.
-@example
- ...
- _22:bat[:oid,:oid] := sql.bind_dbat("tmp","_tables",0);
- _23 := bat.reverse(_22);
- _24 := algebra.kdifference(_20,_23);
- _25 := algebra.markT(_24,0:oid);
- _26 := bat.reverse(_25);
- _27 := algebra.join(_26,_20);
- _28 := bat.setWriteMode(_19);
- bat.append(_28,_27,true);
- ...
-@end example
-
-MAL supports the full breadth of computational paradigms
-deployed in a database setting. It is language framework
-where the execution semantics is determined by the
-code transformations and the final engine choosen.
-
-The design and implementation of MAL takes the functionality offered
-previously a significant step further. To name a few:
-@itemize @bullet
-@item All instructions are strongly typed before being executed.
-@item It supports polymorphic functions.
-They act as templates that produce strongly typed instantiations when needed.
-@item Function style expressions where
-each assignment instruction can receive multiple target results;
-it forms a point in the dataflow graph.
-@item It supports co-routines (Factories) to build streaming applications.
-@item Properties are associated with the program code for
-ease of optimization and scheduling.
-@item It can be readily extended with user defined types and
-function modules.
-@end itemize
-
-@-
-@{
-@+ Critical sections and semaphores
-MonetDB Version 5 is implemented as a collection of threads.
-This calls for extreme
-care in coding. At several places locks and semaphores are necessary
-to achieve predictable results. In particular, after they are created
-and when they are inspected or being modified to take decisions.
-In the current implementation the following list of locks and semaphores
-is used in the Monet layer:
-
-@mal
-@+ Monet Basic Definitions
-Definitions that need to included in every file of the Monet system,
-as well as in user defined module implementations.
@h
#ifndef _MAL_H
#define _MAL_H
@@ -196,12 +39,6 @@
#define mal_export extern
#endif
-@+ Monet Calling Options
-The number of invocation arguments is kept to a minimum.
-See `man mserver5` or tools/mserver/mserver5.1
-for additional system variable settings.
-@
-@h
#define MAXSCRIPT 64
mal_export char monet_cwd[PATHLENGTH];
@@ -231,65 +68,12 @@
#define GRPperformance (JOINPROPMASK | DEADBEEFMASK)
#define GRPoptimizers (1<<27) /* == OPTMASK; cf., gdk/gdk.mx */
#define GRPforcemito (FORCEMITOMASK)
-@c
-#include <monetdb_config.h>
-#include <mal.h>
-
-char monet_cwd[PATHLENGTH] = { 0 };
-int monet_welcome = 1;
-str *monet_script;
-int monet_daemon=0;
-size_t monet_memory;
-@}
-@-
-@node Execution Engine, Session Scenarios, MAL Synopsis , Design Overview
-@+ Execution Engine
-The execution engine comes in several flavors. The default is a
-simple, sequential MAL interpreter. For each MAL function call it creates
-a stack frame, which is initialized with all constants found in the
-function body. During interpretation the garbage collector
-ensures freeing of space consumptive tables (BATs) and strings.
-Furthermore, all temporary structures are garbage collected before
-the funtion returns the result.
-
-This simple approach leads to an accumulation of temporary variables.
-They can be freed earlier in the process using an explicit garbage collection
-command, but the general intend is to leave such decisions to an optimizer
-or scheduler.
-
-The execution engine is only called when all MAL instructions
-can be resolved against the available libraries.
-Most modules are loaded when the server starts using a
-bootstrap script @sc{mal_init.mx}
-Failure to find the startup-file terminates the session.
-It most likely points to an error in the MonetDB configuration file.
-
-During the boot phase, the global symbol table is initialized
-with MAL function and factory definitions, and
-loading the pre-compiled commands and patterns.
-The libraries are dynamically loaded by default.
-Expect tens of modules and hundreds of operations to become readily available.
-
-Modules can not be dropped without restarting the server.
-The rational behind this design decision is that a dynamic load/drop feature
-is often hardly used and severely complicates the code base.
-In particular, upon each access to the global symbol table we have to be
-prepared that concurrent threads may be actively changing its structure.
-Especially, dropping modules may cause severe problems by not being
-able to detect all references kept around.
-This danger required all accesses to global information to be packaged
-in a critical section, which is known to be a severe performance hindrance.
-
-@{
-@h
-
mal_export MT_Lock mal_contextLock;
mal_export MT_Lock mal_remoteLock;
mal_export MT_Lock mal_profileLock ;
mal_export MT_Lock mal_copyLock ;
-
mal_export int mal_init(void);
mal_export void mal_exit(void);
mal_export int moreClients(int reruns);
@@ -300,7 +84,6 @@
* also moving the implementation there...
*/
-
/* Listing modes are globally known */
#define LIST_INPUT 1 /* echo original input */
#define LIST_MAL_STMT 2 /* show mal instruction */
@@ -326,6 +109,206 @@
#endif
#endif /* _MAL_H*/
+@c
+/* Author(s) M.L. Kersten
+ * @node Design Considerations, Architecture Overview, Design Overview,
Design Overview
+ * @+ Design Considerations
+ * Redesign of the MonetDB software stack was driven by the need to
+ * reduce the effort to extend the system into novel directions
+ * and to reduce the Total Execution Cost (TEC).
+ * The TEC is what an end-user or application program will notice.
+ * The TEC is composed on several cost factors:
+ * @itemize
+ * @item A)
+ * API message handling
+ * @item P)
+ * Parsing and semantic analysis
+ * @item O)
+ * Optimization and plan generation
+ * @item D)
+ * Data access to the persistent store
+ * @item E)
+ * Execution of the query terms
+ * @item R)
+ * Result delivery to the application
+ * @end itemize
+ *
+ * Choosing an architecture for processing database operations pre-supposes an
+ * intuition on how the cost will be distributed. In an OLTP
+ * setting you expect most of the cost to be in (P,O), while in OLAP it will
+ * be (D,E,R). In a distributed setting the components (O,D,E) are dominant.
+ * Web-applications would focus on (A,E,R).
+ *
+ * Such a simple characterization ignores the wide-spread
+ * differences that can be experienced at each level. To illustrate,
+ * in D) and R) it makes a big difference whether the data is already in the
+ * cache or still on disk. With E) it makes a big difference whether you
+ * are comparing two integers, evaluation of a mathematical function,
+ * e.g., Gaussian, or a regular expression evaluation on a string.
+ * As a result, intense optimization in one area may become completely
invisible
+ * due to being overshadowed by other cost factors.
+ *
+ * The Version 5 infrastructure is designed to ease addressing each
+ * of these cost factors in a well-defined way, while retaining the
_______________________________________________
Checkin-list mailing list
[email protected]
http://mail.monetdb.org/mailman/listinfo/checkin-list
