Bugs item #939842, was opened at 2004-04-22 00:41 Message generated for change (Comment added) made by sf-robot You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=939842&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: (zombie: MonetDB 4.3 CVS Head) >Status: Closed Resolution: Fixed Priority: 5 Private: No Submitted By: Fabian (mr-meltdown) Assigned to: Peter Boncz (boncz) Summary: SQL: Meltdown: cartesian product crashes server? Initial Comment: consider the following dialogue: Welcome to the MonetDB interactive JDBC terminal! Database: MonetDB 4.3.17 (4.3) Driver: MonetDB Native Driver 0.4 beta release (0.4) Type \q to quit, \h for a list of available commands auto commit mode: on monetdb-> select count(*) from voyages; +---------- | count_mediumint +---------- | 8141 +---------- 1 rows monetdb-> select count(*) from invoices; +---------- | count_mediumint +---------- | 3836 +---------- 1 rows monetdb-> select count(*) from craftsmen; +---------- | count_mediumint +---------- | 2372 +---------- 1 rows monetdb-> select count(*) from voyages, invoices, craftsmen; Error on line 4: java.io.IOException: End of stream reached Executed query: select count(*) from voyages, invoices, craftsmen; monetdb-> Mserver dies without a notice after a minute or two. top reports a memory usage of around 500MB, but my systems is swapping heavily making it unusable for the time that Mserver is alive. Dunno if this can make any difference, but I'm using blocked mode Mapi interface. Will try with line mode as well using MapiClient. ---------------------------------------------------------------------- >Comment By: SourceForge Robot (sf-robot) Date: 2007-11-29 19:20 Message: Logged In: YES user_id=1312539 Originator: NO This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 365 days (the time period specified by the administrator of this Tracker). ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2006-11-29 01:05 Message: Logged In: YES user_id=572415 Originator: NO over-ruling "sf-robot": re-set to "Pending" since adding a proper test has neither been done nor finally been discarded, yet. User (submitter) & developer (assignee), please check, again. ---------------------------------------------------------------------- Comment By: SourceForge Robot (sf-robot) Date: 2006-11-21 19:20 Message: Logged In: YES user_id=1312539 Originator: NO This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 365 days (the time period specified by the administrator of this Tracker). ---------------------------------------------------------------------- Comment By: SourceForge Robot (sf-robot) Date: 2005-11-20 19:20 Message: Logged In: YES user_id=1312539 This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 14 days (the time period specified by the administrator of this Tracker). ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2005-11-06 01:56 Message: Logged In: YES user_id=572415 BugDay_2005-10-06: "left-overs" are now marked as "pending" ---------------------------------------------------------------------- Comment By: Roberto Cornacchia (cornuz) Date: 2005-10-06 10:10 Message: Logged In: YES user_id=883451 BugDay_2005-10-06: CLAIMED BY cornuz BugDay_2005-10-06: NO TEST / POSTPONED The test also depends on the available amount of memory... ---------------------------------------------------------------------- Comment By: Fabian (mr-meltdown) Date: 2004-05-07 02:36 Message: Logged In: YES user_id=963970 seems to work: monetdb-> select count(*) from voyages, craftsmen, passengers; Error on line 2: ERROR: GDKload: cannot mmap(): name=01/102, ext=buns.priv OS: Cannot allocate memory ERROR: BATpropcheck: BAT parameter: BAT required. ERROR: CMDjoin: operation failed. Executed query: select count(*) from voyages, craftsmen, passengers; monetdb-> BUT.... something gets messed up, because issuing the same statement again does: monetdb-> select count(*) from voyages, craftsmen, passengers; Error on line 3: ERROR: interpret: no matching MIL operator to 'sql2()'. Executed query: select count(*) from voyages, craftsmen, passengers; So, life is better, but it looks like the SQL session gets affected after this error. (Simple) other select statements do work. ---------------------------------------------------------------------- Comment By: Martin Kersten (mlkersten) Date: 2004-05-06 10:51 Message: Logged In: YES user_id=490798 The source code has been patched to react to failures of this kind. An error is produced and the system continous working. ---------------------------------------------------------------------- Comment By: Martin Kersten (mlkersten) Date: 2004-04-24 00:13 Message: Logged In: YES user_id=490798 The original query would produced 8141*3836*2372 = 74074893872 results. It is unclear in which stage of the MIL execution the server crashes. I suspect after the first join upon allocation of the temporaries for the second halve. A first line of defense is to issue an emergency stop if a BAT request size goes beyond a highwatermark level. Possible responses: 1) GDKfatal, too crude 2) GDKerror is set and the actual size is trimmed to an acceptable level and making the BAT readonly to avoid any further update (tryout with a patch in gdk_bat.mx) ---------------------------------------------------------------------- Comment By: Fabian (mr-meltdown) Date: 2004-04-23 02:01 Message: Logged In: YES user_id=963970 solution b is dirty. It implicates that all sessions to the particular server are invalid which more or less requires a reconnect of the client(s) involved. ---------------------------------------------------------------------- Comment By: Peter Boncz (boncz) Date: 2004-04-23 01:53 Message: Logged In: YES user_id=591107 there are two approaches to this problem: (a) fix all GDK/module primitives that receive BATs directly from other GDK/module C primitives. They should properly collect their garbage and also return NULL (b) implement a Mguardian that at least keeps all Mapiclients open, restarts the Mserver, reconnects and just tells all clients that their transactions where aborted. I have no specific preference as of yet. ---------------------------------------------------------------------- Comment By: Niels Nes (nielsnes) Date: 2004-04-22 03:38 Message: Logged In: YES user_id=43556 The memory model of monet is fare from save. There is no way the sql frontend could return an error as the underlying BATnew calls will not return a error (NULL) but just crash! Adding checks for all returns of BATnew (and other BAT returning functions) is currently considered a to large job. ---------------------------------------------------------------------- Comment By: Fabian (mr-meltdown) Date: 2004-04-22 01:11 Message: Logged In: YES user_id=963970 Yup, I'm aware of that I'm generating a HUGE result set... but hey consider me to be an SQL starter, then why does the DB crash on me?!? What did I do wrong? In the worst case I expected to get three columns with rowcounts for each table! (LOL) I would be happy if Mserver terminated with an out of memory error. I would be ultimate happy if Mserver would cut off my client connection with an too much resources used error. I would be in the seventh heaven if Mserver terminated the current query execution and returned an error to the client saying 'current execution terminated: out of memory' and maintained the connection to my client. ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2004-04-22 01:03 Message: Logged In: YES user_id=572415 hm, be aware of the fact, that this creates a temp. BAT of 31,228,876 tuples, i.e., about 238 MB in case the BAT is [oid,oid], hence, there *might* be a memory ... It does work in a simple test, though: ======= mil>module(radix); mil>a:=uniform(8141).project(0); mil>a.slice(0,1).print; #-------------------------# # h tmp_303 # name # oid int # type #-------------------------# [ [EMAIL PROTECTED], 0 ] [ [EMAIL PROTECTED], 0 ] mil>b:=uniform(3836).project(0).reverse; mil>b.slice(0,1).print; #-------------------------# # t tmp_303 # name # int oid # type #-------------------------# [ 0, [EMAIL PROTECTED] ] [ 0, [EMAIL PROTECTED] ] mil>c:=join(a,b); mil>c.count.print; [ 31228876 ] mil> ======== ---------------------------------------------------------------------- Comment By: Fabian (mr-meltdown) Date: 2004-04-22 00:47 Message: Logged In: YES user_id=963970 MapiClient does exactly the same: sql>select count(*) from voyages, invoices, craftsmen; MAPI = [EMAIL PROTECTED]:45123 ACTION= mapi_fetch_line QUERY = select count(*) from voyages, invoices, craftsmen; ERROR = Battle control terminated ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=939842&group_id=56967 ------------------------------------------------------------------------- SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future. http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
