Feature Requests item #2261241, was opened at 2008-11-11 09:07 Message generated for change (Settings changed) made by nielsnes You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482471&aid=2261241&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. >Category: None >Group: None Status: Open Priority: 5 Private: No Submitted By: Thomas Neumann (tneumann) Assigned to: Niels Nes (nielsnes) Summary: bulkloading a large string table kills mserver Initial Comment: Bulkloading a large string table of the form (id,value) kills mserver. I attached a data generator that generates 21GB of input to illustrate the problem. How to reproduce: - g++ -odatagen -O2 datagen.cpp - ./datagen > test.sql - monetdb create foo - monetdb start foo - mclient -l sql -d foo test.sql results in: ERROR = Connection terminated and the server crashed (on Linux x86_64). Note that there seem to be two problems in one: The primary crash seems to be due to the "primary key" constraints in the table. Dropping it changes the behavior: The bulkload succeeds on a machine with 32GB of main memory, but effectively stops (i.e., swap excessively) on a machine with 2GB of main memory. Adding the primary key constrants after successful bulkloading without it crashes the server, too. ---------------------------------------------------------------------- >Comment By: Niels Nes (nielsnes) Date: 2009-02-13 07:24 Message: indeed on a limited machine there still is a crash. Probably when the memory (or swap) run out. Changing into a feature request as its a long term goal to improve per client resource handling and make it more safe. ---------------------------------------------------------------------- Comment By: Niels Nes (nielsnes) Date: 2009-02-12 07:47 Message: I'll try on a 2G machine later. ---------------------------------------------------------------------- Comment By: Thomas Neumann (tneumann) Date: 2009-02-12 07:43 Message: Both machines were 64 bit systems. Note that I reported the bug before the latest release, so it might have been fixed in between (in particular since the changelog mentions changes to the data loader). I did not try the last release yet. ---------------------------------------------------------------------- Comment By: Niels Nes (nielsnes) Date: 2009-02-12 07:02 Message: The 2GB machine was a 32 or 64 bit system? Loading on a 64 bit machine with 8G seems to work okay (including primary key). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482471&aid=2261241&group_id=56967 ------------------------------------------------------------------------------ Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
