Bugs item #1988776, was opened at 2008-06-09 10:25
Message generated for change (Comment added) made by nielsnes
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1988776&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: SQL/Core
Group: (zombie: MonetDB5 5.4)
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jorge Limon (geolemo)
>Assigned to: Niels Nes (nielsnes)
Summary: Data are corrupted
Initial Comment:
I'm evaluating Monetdb to store large tables. To test the monetdb perfomance I
decided to load a large table of radius logs into a single table "radlog_logs":
CREATE TABLE "radius"."radlog_logs" (
"cpi" varchar(20),
"login" varchar(40),
"segundos" int,
"connections" int,
"nodoacceso" int,
"telefono" int,
"idprovincia" smallint,
"idred" smallint,
"idfecha" int,
"mmbb" varchar(11),
"codproducto" varchar(10),
"es_suscriptor" char(5)
I use a sql dump file and mclient to bulk the data into my test table like this:
mclient -lsql -uradius -Pvoc < final2.sql
where "final2.sql" has 1,3 millons of inserts commands like:
INSERT INTO "radlog_logs" VALUES ('arrakis', -1, -1, 1, 4495, 'igmap', '1411',
'false');;
I repeated this step several times until I have a table with more that 20
millons of records (I spent 2 days in this tasks because server crashed several
times in this process).
After this, I started to review data by launching diferents querys and I saw
that information was corrupted.
For example, the last column has only "true" or "false" string values, but I
found registers with bad information (it seems like some columns was stored
data of others columns!!!)
For example in the following query ALLt records must contain the following:
cpi = arrakis
login = arkadsl954789006
es_suscriptor = false
but...columns data are mixed!
sql>select cpi,login, es_suscriptor from radius.radlog_logs where
login='arkadsl954789006';
+------------------+-----------------+--------------+
| cpi |login |es_suscriptor |
+==================+=================+==============+
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arrakis |arkadsl954789006 |false |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arrakis |arkadsl954789006 |false |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arrakis |arkadsl954789006 |false |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arrakis |arkadsl954789006 |false |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arrakis |arkadsl954789006 |false |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arkadsl954789006 |arkadsl954789006 |arkadsl954789 |
: | |006 |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
| arrakis |arkadsl954789006 |false |
+------------------+-----------------+--------------+
I use:
Monetdb v5.4.0
Windows 2003 Standard
I use a Proliant DL380 server with 1 gb ram
----------------------------------------------------------------------
>Comment By: Niels Nes (nielsnes)
Date: 2008-06-09 11:04
Message:
Logged In: YES
user_id=43556
Originator: NO
I would like to know if it correctly works on the soon to be released
MonetDB (5.6)/SQL (2.24.0). Also loading will be much faster if you change
to using copy into instead of insert statements.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1988776&group_id=56967
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
