Bugs item #2316185, was opened at 2008-11-20 01:33 Message generated for change (Settings changed) made by stmane You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2316185&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core Group: MonetDB5 CVS Head Status: Open Resolution: Fixed >Priority: 8 Private: No Submitted By: Stefan de Konink (skinkie) Assigned to: Nobody/Anonymous (nobody) Summary: Data corruption bug after copy into Initial Comment: Before anyone starts about 'I cannot reproduce this bug'; you can request (or already have) an account to this machine. What I do; START TRANSACTION; CREATE TABLE nodes_legacy (id serial, long integer, lat integer, username varchar(255), timestamp timestamptz); CREATE TABLE node_tags (node integer, k varchar(255), v varchar(1024), primary key (node, k), foreign key(node) references nodes_legacy); COPY 6330269 RECORDS INTO nodes_legacy from '/tmp/import/nodes.csv' USING DELIMITERS ',', '\n', ''''; COPY 5768152 RECORDS INTO node_tags from '/tmp/import/node_tags.csv' USING DELIMITERS ',', '\n', ''''; COMMIT; Sample of nodes.csv: 123350, 608491291, 51499150, 'raskas', 2007-05-31T19:44:09Z 123351, 608610167, 51501482, 'raskas', 2007-05-31T19:44:08Z 123352, 608639476, 51508729, 'raskas', 2007-05-31T19:44:09Z Sample of node_tags.csv 123379, 'created_by', 'YahooApplet 1.0' 123382, 'created_by', 'YahooApplet 1.0' 123383, 'created_by', 'YahooApplet 1.0' gdb --args /opt/monetdb/bin/mserver5 --daemon=yes --dbinit="include sql;" GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i686-pc-linux-gnu"... (gdb) run Starting program: /opt/monetdb/bin/mserver5 --daemon=yes --dbinit=include\ sql\; [Thread debugging using libthread_db enabled] [New Thread 0xb6f726c0 (LWP 18157)] [New Thread 0xb3771b90 (LWP 18160)] # MonetDB server v5.9.0, based on kernel v1.27.0 # Serving database 'demo', using 1 thread # Compiled for i686-pc-linux-gnu/32bit with 32bit OIDs dynamically linked # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008- MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /opt/monetdb/etc/monetdb5.conf) [New Thread 0xb27a3b90 (LWP 18162)] # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.27.0 loaded mserver5: ../../../src/gdk/gdk_bat.mx:1746: void_inplace: Assertion `BATcount(b) > (id -b->H->seq)' failed. Program received signal SIGABRT, Aborted. [Switching to Thread 0xb27a3b90 (LWP 18162)] 0xb800d424 in __kernel_vsyscall () (gdb) bt #0 0xb800d424 in __kernel_vsyscall () #1 0xb6fe5670 in raise () from /lib/libc.so.6 #2 0xb6fe6eb8 in abort () from /lib/libc.so.6 #3 0xb6fde64e in __assert_fail () from /lib/libc.so.6 #4 0xb7b38c24 in void_inplace (b=0x87dc100, id=164, val=0x8e9ad6c, force=1 '\001') at ../../../src/gdk/gdk_bat.mx:1746 #5 0xb7b38f44 in void_replace_bat (b=0x87dc100, u=0x87e3ad8, force=1 '\001') at ../../../src/gdk/gdk_bat.mx:1771 #6 0xb234213d in load_delta (bat=0x8e99458, bid=147, ubid=284, type=13) at ../../../../src/storage/bat/bat_storage.mx:423 #7 0xb23421f4 in load_bat (bat=0x8e99458, type=13) at ../../../../src/storage/bat/bat_storage.mx:440 #8 0xb23426c1 in create_col (tr=0x8e532a8, c=0x8e993c0) at ../../../../src/storage/bat/bat_storage.mx:543 #9 0xb23346fb in bootstrap_create_column (tr=0x8e532a8, t=0x8e920b8, name=0xb2379881 "default", sqltype=0xb2379b0e "varchar", digits=2048) at ../../../src/storage/store.mx:1082 #10 0xb2335a1f in store_init (debug=0, store=store_bat, logdir=0x87ee3bc "/opt/monetdb/var/MonetDB5/sql_logs", dbname=0x87ee2cc "demo", stk=0) at ../../../src/storage/store.mx:1304 #11 0xb2308c1a in mvc_init (debug=0, store=store_bat, stk=0) at ../../../src/server/sql_mvc.mx:268 #12 0xb22aa109 in SQLinit (c=0x804d3fc) at ../../../../src/backends/monet5/sql_scenario.mx:235 #13 0xb7f927f4 in initScenario (c=0x804d3fc, s=0xb7fa9888) at ../../../src/mal/mal_scenario.mx:260 #14 0xb7f932d4 in setScenario (c=0x804d3fc, nme=0x8d9b323 "sql") at ../../../src/mal/mal_scenario.mx:504 #15 0xb7f5b38f in MSscheduleClient (command=0x8d9b2e8 "LIT", challenge=0xb27a32a0 "9S9SSENV", fin=0x87cb0a8, fout=0x8d98d18) at ../../../src/mal/mal_session.mx:342 #16 0xb2ca64a6 in doChallenge (cmd=0x0, in=0x8d1b258, out=0x8d1b808) at ../../../../src/modules/mal/mal_mapi.mx:479 #17 0xb2ca671f in SERVERlistenThread (Sock=0x8d1a708) at ../../../../src/modules/mal/mal_mapi.mx:562 #18 0xb731f170 in start_thread () from /lib/libpthread.so.0 #19 0xb7086dfe in clone () from /lib/libc.so.6 In order to get in to this 'situation' I had an issue with a previous (full-)import because of a duplicate primary key. I expected this did a proper rollback. Then I tried this (partial-)import again, but removed the tables that broke my import. ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-02-16 10:47 Message: re-opened to remind us that we should consider adding a test for this one. ---------------------------------------------------------------------- Comment By: Niels Nes (nielsnes) Date: 2009-02-11 21:05 Message: fixed. We now update keys. This prevents unaligned bats. ---------------------------------------------------------------------- Comment By: Romulo Goncalves (romulog) Date: 2008-11-20 16:43 Message: "I did the insertion and I did get any crash except complains about foreign key violation because the ids values from the nodes.csv do not match the node values on node_tags.csv" I did not get any crash.... Sorry I forgot the "not" ---------------------------------------------------------------------- Comment By: Romulo Goncalves (romulog) Date: 2008-11-20 16:41 Message: I did the insertion and I did get any crash except complains about foreign key violation because the ids values from the nodes.csv do not match the node values on node_tags.csv I decided to make them equal and I used the latest stable version of M5-SQL with normal compilation and no changes related with number of threads.... I select the content of the tables: % sys.node_tags, sys.node_tags, sys.node_tags # table_name % node, k, v # name % int, varchar, varchar # type % 6, 10, 15 # length [ 123350, "created_by", "YahooApplet 1.0" ] [ 123351, "created_by", "YahooApplet 1.0" ] [ 123352, "created_by", "YahooApplet 1.0" ] % sys.nodes_legacy, sys.nodes_legacy, sys.nodes_legacy, sys.nodes_legacy, sys.nodes_legacy # table_name % id, long, lat, username, timestamp # name % int, int, int, varchar, timestamptz # type % 6, 9, 8, 6, 26 # length [ 123350, 608491291, 51499150, "raskas", 2007-05-31 19:44:09.000000+00:00 ] [ 123351, 608610167, 51501482, "raskas", 2007-05-31 19:44:08.000000+00:00 ] [ 123352, 608639476, 51508729, "raskas", 2007-05-31 19:44:09.000000+00:00 ] I need more content if you do not mind ;) PS: The same was done for M5-SQL current version.... ---------------------------------------------------------------------- Comment By: Stefan de Konink (skinkie) Date: 2008-11-20 16:17 Message: I can reproduce this bug with --set gdk_nr_threads=1. Might be unrelated? ---------------------------------------------------------------------- Comment By: Martin Kersten (mlkersten) Date: 2008-11-20 12:00 Message: The bug reported seems to be related to the parallel data loader bug #2261241 The easiest way to confirm this is to load the database using the setting --set gdk_nr_threads=1 which should not cause this problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2316185&group_id=56967 ------------------------------------------------------------------------------ Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
