Bugs item #2771052, was opened at 2009-04-17 18:55 Message generated for change (Comment added) made by mlkersten You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2771052&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: SQL/Core Group: SQL "stable" >Status: Open >Resolution: Accepted Priority: 5 Private: No Submitted By: Stefan Manegold (stmane) >Assigned to: Martin Kersten (mlkersten) Summary: M5/SQL: parallel bulk-load (copy into) incorrect Initial Comment: With a simple CSV file containing 100000000 rows, each holding just a single '1', I get the following when running a (default) 4-threaded mserver5 on my 4-core machine: $ mserver5 --dbinit='include sql;' # MonetDB server v5.10.3, based on kernel v1.28.3 # Serving database 'demo', using 4 threads # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.28.3 loaded $ mclient -lsql sql>create table i0 (a integer); 0 tuples sql>create table i1 (a integer); 0 tuples sql>create table i2 (a integer); 0 tuples sql>create table i3 (a integer); 0 tuples sql>create table i4 (a integer); 0 tuples sql>create table i5 (a integer); 0 tuples sql>create table i6 (a integer); 0 tuples sql>create table i7 (a integer); 0 tuples sql>create table i8 (a integer); 0 tuples $ mclient -lsql sql>copy 1 records into i0 from '/tmp/1.csv'; Rows affected 1 sql>copy 10 records into i1 from '/tmp/1.csv'; Rows affected 10 sql>copy 100 records into i2 from '/tmp/1.csv'; Rows affected 100 sql>copy 1000 records into i3 from '/tmp/1.csv'; Rows affected 1000 sql>copy 10000 records into i4 from '/tmp/1.csv'; Rows affected 40000 sql>copy 100000 records into i5 from '/tmp/1.csv'; Rows affected 400000 sql>copy 1000000 records into i6 from '/tmp/1.csv'; Rows affected 4000000 sql>copy 10000000 records into i7 from '/tmp/1.csv'; Rows affected 40000000 sql>copy 100000000 records into i8 from '/tmp/1.csv'; Rows affected 100000000 (Mind the '4's where '1's are expected.) The problem is analogously reproducible with `mserver5 --set gdk_nr_threads=3` & `mserver5 --set gdk_nr_threads=2`; bulk-loading works fine with `mserver5 --set gdk_nr_threads=1`. ---------------------------------------------------------------------- >Comment By: Martin Kersten (mlkersten) Date: 2009-08-09 13:15 Message: The bug should have been left open, as it is a BUG part of at least two releases and should be fixed. ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-07-31 06:54 Message: Clean-up: This one is no longer relevant, as parallel bulk loading has been disabled due to inherent design flaws: =================================================================== 2009/05/25 - sjoerd: MonetDB5/src/modules/mal/tablet.mx,1.194.2.3 Set nway to 1 since parallel import *cannot* work with the current implementation: there is an inherent bug when the number of records to be imported is smaller than what is available in the file. =================================================================== ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-04-17 19:13 Message: added test in sql/src/test/BugTracker-2009/Tests/parallel_bulk-load.SF-2771052.{unpack_csv,test}.* (Feb2009 release branch) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2771052&group_id=56967 ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
