Bugs item #2779702, was opened at 2009-04-23 19:09
Message generated for change (Comment added) made by stmane
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2779702&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: SQL/Core
Group: SQL CVS Head
>Status: Open
Resolution: Accepted
Priority: 5
Private: No
Submitted By: Roberto Cornacchia (cornuz)
>Assigned to: Roberto Cornacchia (cornuz)
Summary: SQL COPY INTO, data corrupted after shutdown
Initial Comment:
I'm loading a table from disk. The data seems OK until I shut down the db.
If I do that, then the data is corrupted.
It shouldn't be related to the bug about parallel bulk load, because I'm using
gdk_threads=1
# create a clean database
$ monetdb create test
successfully created database 'test'
$ monetdb start test
starting database 'test'... done
# Load a table from file 'ne_doc' (140377 tuples)
# This table is too large to be an attachment to this bug report.
# It can be found here: /ufs/roberto/ne_doc
$ echo "CREATE TABLE ne_doc (neID int, predicate VARCHAR(30), docID int);" |
mclient -lsql -d test
$ echo "COPY 140377 RECORDS INTO ne_doc FROM '/ufs/roberto/ne_doc';" | mclient
-lsql -d test
[ 140377 ]
# Ask for the unique 'predicates'
$ echo "select predicate from ne_doc group by predicate;" | mclient -lsql -d
test
% sys.ne_doc # table_name
% predicate # name
% varchar # type
% 21 # length
[ "ep-contracting-states" ]
[ "inventor_of" ]
[ "assignee_of" ]
# Stop and start the database
$ monetdb stop test
stopping database 'test'... done
$ monetdb start test
starting database 'test'... done
# Ask for the unique 'predicates'
$ echo "select predicate from ne_doc group by predicate;" | mclient -lsql -d
test
% sys.ne_doc # table_name
% predicate # name
% varchar # type
% 21 # length
[ "ep-contracting-states" ]
[ "inventor_of" ]
[ "" ]
It seems corrupted after the sutdown
----------------------------------------------------------------------
>Comment By: Stefan Manegold (stmane)
Date: 2009-08-11 11:54
Message:
Roberto,
could I use your data to add a test to CVS?
Or is that data not meant to be public?
----------------------------------------------------------------------
Comment By: Martin Kersten (mlkersten)
Date: 2009-08-06 16:33
Message:
no additional information to disagree with the implicit solution provided.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2009-06-05 17:00
Message:
I just tried this with the May2009 release, and it appears to work fine
now, both using 4 threads on my quad core desktop as well as using only one
thread (ghk_nr_threads=1) --- note, though, that parallel bulk loading has
been disabled recently as the code appeared to be incorrect for many cases
---.
Roberto,
could I use your data to add a test to CVS?
Or is that data not meant to be public?
Stefan
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2009-04-27 19:02
Message:
Strangely, after having lunch and after cleaning (revoming) all legacy
databases and while Roberto was watching, I could (and still can) reproduce
the problem with the Feb2009-SP2 code base (didn't try any other, yet)
In fact, when loading the data multi-threaded (default on multi-CPU/-core
machines), it works fine, and I get the following both directly after
loading as well as after restarting the server:
$ echo "select predicate, count(*) from ne_doc group by predicate;" |
mclient -lsql -d test
% sys.ne_doc, sys.ne_doc # table_name
% predicate, L1 # name
% varchar, wrd # type
% 21, 5 # length
[ "ep-contracting-states", 67386 ]
[ "inventor_of", 51596 ]
[ "assignee_of", 21395 ]
However, when loading the data single-threaded(!) (gdk_nr_threads=1), I
get the above only in the very mserver5 that loaded the data; after
restarting the server, I get:
$ echo "select predicate, count(*) from ne_doc group by predicate;" |
mclient -lsql -d test
% sys.ne_doc, sys.ne_doc # table_name
% predicate, L1 # name
% varchar, wrd # type
% 21, 5 # length
[ "ep-contracting-states", 67386 ]
[ "inventor_of", 51596 ]
[ "", 21395 ]
... very strange --- further analysis pending ...
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2009-04-27 11:21
Message:
On my 64-bit Fedora 10 desktop, I cannot reproduce the problem, neither
with the Feb2009 SP2 code base, nor with this mornings CVS HEAD code base
...
What I see, though, with the Feb2009 SP2 code base, when starting
`mserver5 --dbinit='include sql;'` by hand and then bulk-loading the data
with `COPY INTO ...` is the following error on the mserver5 console:
>!ERROR: GDKsave: failed name=10/1073, ext=tail, mode 0
I don't see this when loading single threaded nor with the CVS HEAD code
base.
----------------------------------------------------------------------
Comment By: Roberto Cornacchia (cornuz)
Date: 2009-04-27 10:56
Message:
Indeed, I do use the correct keyword,in monetdb5.conf:
gdk_nr_threads=1
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2009-04-26 10:38
Message:
Just to be sure:
I suppose you are using
--set gdk_nr_threads=1
^^^^
not
--set gdk_threads=1
^
right?
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2779702&group_id=56967
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
