Bugs item #2822300, was opened at 2009-07-16 09:29 Message generated for change (Comment added) made by stmane You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2822300&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: SQL/Core Group: MonetDB5 "stable" Status: Open Resolution: None Priority: 3 Private: No Submitted By: Tomas (tomnvc) Assigned to: Niels Nes (nielsnes) >Summary: SQL: Server crashes silently Initial Comment: Database: # MonetDB server v5.12.2, based on kernel v1.30.2 # Serving database 'Test1', using 1 thread # Compiled for i686-pc-win32/32bit with 32bit OIDs dynamically linked # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see C:\MonetDB\engine\etc\monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ Schema and table: CREATE SCHEMA "test" AUTHORIZATION "monetdb"; SET SCHEMA "test"; CREATE TABLE "test"."tb1" ( "id" varchar(32), "pan" varchar(19), "dt_month" smallint, "dt_year" int, "amount" int, "merchant" varchar(128), "store" varchar(128), "dt" timestamp(7) ); To evaluate MonetDB potential performance we created simple table to simulate possible data. Given table was populated with random data where: ID is totally unique (Guid in HEX format), pan is truncated primary account number in format like this 411111******1111 dt_month and dt_year simulate month and year with 6 (2004 to 2009) different values for year amount is random value from 10 to 981256. Merchant is 100 different string values. Store is 10000 different string values. dt is time. As you can see we wanted to test different value variance on different columns and insert/select query performance. Problem is that server starts to crash after about 11-12 million records have been added. We tested it using sql insert and copy records into, both ways result in crash. Same problem occurred on two different machines: One is Windows XP with 2Gb ram single core cpu, Other is Windows XP with 1.5Gb ram dual core cpu. In both cases number of records for first crashes is about 11-12 millions. The more records we have the more likely crash to occur. Sometimes it crashes so that server does not accept any sql commands any more. It exits upon client connection... We execute our tests from .NET thru wrapped native mapi library. Please fix this… I liked its performance up to 10 mill records, but I want to test it with 500 mill records and more! If it works well... we might even develop native ADO.NET provider for it :) ---------------------------------------------------------------------- >Comment By: Stefan Manegold (stmane) Date: 2009-08-14 13:56 Message: tagged subject ---------------------------------------------------------------------- Comment By: Tomas (tomnvc) Date: 2009-07-21 07:58 Message: It seems that 64 bit version does not have this problem. 200M+ records did not crash server and data size on disk was over 25 Gigs. We will have to set up larger environment for our tests and most likely we’ll go for 1000M to 2000M records, not 500M initially intended. I guess it is ok, we would have to go for 64 bit version any way, but still… it would be nice if 32 bit version wouldn’t crash! We would have more confidence using this very nice product :) ---------------------------------------------------------------------- Comment By: Niels Nes (nielsnes) Date: 2009-07-19 11:46 Message: This is very likely a 32bit monetdb problem. Could you run a 64 bit version of monetdb on your 64 bit windows server? For you 500M test case we recommend using a 64 bit version. ---------------------------------------------------------------------- Comment By: Tomas (tomnvc) Date: 2009-07-16 15:09 Message: We just reproduced same problem on 64bit version of Windows 2003 Server , Computer has 3Gb of memory. It took 30mil records to crash it thought… ---------------------------------------------------------------------- Comment By: Tomas (tomnvc) Date: 2009-07-16 13:36 Message: I have attached c# project to reproduce the problem (MonetDbCrash.rar) Please add DBEngine folder near executable file and add this files to it: iconv.dll, libbat.dll, libeay32.dll, libmapi.dll, libmonetdb5.dll, libmutils.dll, libstream.dll, libxml2.dll, pcre.dll, pthreadvc2.dll, ssleay32.dll, zlib1.dll ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2822300&group_id=56967 ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
