This message was forwarded from [email protected]. The MonetDB
mailing lists have moved to monetdb.org. Please subscribe to
[email protected], and unsubscribe from this list.
See: http://mail.monetdb.org/mailman/listinfo/developers-list
Send developers-list mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
http://mail.monetdb.org/mailman/listinfo/developers-list
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of developers-list digest..."
Today's Topics:
1. Unique key constraint violation violated (Tapomay Dey)
2. fast bit integration into monetdb (Stanislav Seltser)
3. Re: fast bit integration into monetdb (Stefan de Konink)
4. Re: fast bit integration into monetdb (Stanislav Seltser)
5. Re: fast bit integration into monetdb (Stefan Manegold)
6. Re: fast bit integration into monetdb (Stanislav Seltser)
7. Re: Unique key constraint violation violated (Tapomay Dey)
----------------------------------------------------------------------
Message: 1
Date: Thu, 27 Dec 2012 07:45:41 -0800 (PST)
From: Tapomay Dey <[email protected]>
To: "[email protected]" <[email protected]>
Subject: Unique key constraint violation violated
Message-ID:
<[email protected]>
Content-Type: text/plain; charset="us-ascii"
Is this a known issue that you are working on? If so please give me the bug id
where I would know when its fixed.
The case is as follows:
I have a table with 32 columns with unique key on 3 of them.
I am running live ETLs for putting data into monetdb. These ETLs are generating
sql insert scripts.(inserts/updates/ddls)
Using JDBC and c3p0 connection pooling.
Have developed an external wrapper module to achieve master-master replication.
One common consistency check would be a count*.
Observation:
Queries are run in batches of 100 inserts clubbed into a single insert.
I do a select count(*) after each batch of 100.
After running 9100 queries the count* on one DB was 9100 but in the other was
9200.
The second DB had 100 records that were duplicates of 8901-9000.
It means when I inserted 9001 to 9100 the previous batch went into the store
again with it and somehow surpassed the unique key constraint check.
I have also observed another case where count(*) gave less results than actual
no. of rows in the DB.
REQUEST:
Just want to confirm if such kind of issues are part of active development.
Otherwise it would be really difficult for me to push monetdb usage into
production.
Will try again with MonetDataSource instead of c3p0. But I doubt if the issues
are client side or connection-related.
Plz note that the keys for which this occurs is random and thus I concluded
it's not a data issue.
Regards,
Tapomay Dey.
Software Design Engg.
Sokrati Inc.
Email: tapomayATsokratiDOTcom
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.monetdb.org/pipermail/developers-list/attachments/20121227/534caf5c/attachment.html>
------------------------------
Message: 2
Date: Thu, 27 Dec 2012 14:34:22 -0500
From: Stanislav Seltser <[email protected]>
To: [email protected]
Subject: fast bit integration into monetdb
Message-ID: <[email protected]>
Content-Type: text/plain; charset=us-ascii
does anybody know if fastbit integration into scidb was ever completed?
------------------------------
Message: 3
Date: Thu, 27 Dec 2012 20:35:13 +0100
From: Stefan de Konink <[email protected]>
To: [email protected]
Subject: Re: fast bit integration into monetdb
Message-ID: <[email protected]>
Content-Type: text/plain; charset=ISO-8859-1
On 12/27/12 20:34, Stanislav Seltser wrote:
> does anybody know if fastbit integration into scidb was ever completed?
Yes, but it targeted MonetDB4 (not 5, the version we now all use).
Stefan
------------------------------
Message: 4
Date: Thu, 27 Dec 2012 14:38:25 -0500
From: Stanislav Seltser <[email protected]>
To: "Communication channel for developers of the MonetDB suite."
<[email protected]>
Subject: Re: fast bit integration into monetdb
Message-ID: <[email protected]>
Content-Type: text/plain; charset=us-ascii
Stefan, do you know if that code in monetdb source archive for monetdb4 and how
hard would it be to integrate with monetdb 5?
On Dec 27, 2012, at 2:35 PM, Stefan de Konink <[email protected]> wrote:
> On 12/27/12 20:34, Stanislav Seltser wrote:
>> does anybody know if fastbit integration into scidb was ever completed?
>
> Yes, but it targeted MonetDB4 (not 5, the version we now all use).
>
> Stefan
>
> _______________________________________________
> developers-list mailing list
> [email protected]
> http://mail.monetdb.org/mailman/listinfo/developers-list
------------------------------
Message: 5
Date: Thu, 27 Dec 2012 22:47:00 +0100 (CET)
From: Stefan Manegold <[email protected]>
To: "Communication channel for developers of the MonetDB suite."
<[email protected]>
Subject: Re: fast bit integration into monetdb
Message-ID: <[email protected]>
Content-Type: text/plain; charset=utf-8
This is the *MonetDB* (http://www.monetdb.org/) mailing list, so I'm afraid we
cannot say much about *SciDB* (http://www.scidb.org/) ... ;-)
As for MonetDB & FastBit, as far as I am aware, if there ever was a complete
integration into MonetDB, it was not done by or provided to the (core) MonetDB
team, at least not in a state or under a license that would have allowed us to
include it into our open-source code base.
Those how know more/better are more than welcome to correct me, if I'm wrong ...
Best,
Stefan
----- Original Message -----
> does anybody know if fastbit integration into scidb was ever
> completed?
> _______________________________________________
> developers-list mailing list
> [email protected]
> http://mail.monetdb.org/mailman/listinfo/developers-list
>
------------------------------
Message: 6
Date: Thu, 27 Dec 2012 17:00:23 -0500
From: Stanislav Seltser <[email protected]>
To: "Communication channel for developers of the MonetDB suite."
<[email protected]>
Subject: Re: fast bit integration into monetdb
Message-ID: <[email protected]>
Content-Type: text/plain; charset=us-ascii
sorry i made a typo. i had meant integration into monetdb
the integration work fastbit team done with monetdb is described in 2006
white paper http://crd-legacy.lbl.gov/~kewu/ps/LBNL-61768.html . in the paper
they make references to be working with core monetdb team
in changing sql parser to accomodate new index and query executor to figure out
which nodes in query plan are answerable by fastbit. they do show in the paper
combined monetdb/fastbit outperforms mysql more than 500 times on text searhing
and filtering and 8 times faster than original monetdb by itself. most likely
monetdb 4 was the current released system in 2006..
On Dec 27, 2012, at 4:47 PM, Stefan Manegold <[email protected]> wrote:
> This is the *MonetDB* (http://www.monetdb.org/) mailing list, so I'm afraid
> we cannot say much about *SciDB* (http://www.scidb.org/) ... ;-)
>
> As for MonetDB & FastBit, as far as I am aware, if there ever was a complete
> integration into MonetDB, it was not done by or provided to the (core)
> MonetDB team, at least not in a state or under a license that would have
> allowed us to include it into our open-source code base.
>
> Those how know more/better are more than welcome to correct me, if I'm wrong
> ...
>
> Best,
> Stefan
>
> ----- Original Message -----
>> does anybody know if fastbit integration into scidb was ever
>> completed?
>> _______________________________________________
>> developers-list mailing list
>> [email protected]
>> http://mail.monetdb.org/mailman/listinfo/developers-list
>>
> _______________________________________________
> developers-list mailing list
> [email protected]
> http://mail.monetdb.org/mailman/listinfo/developers-list
------------------------------
Message: 7
Date: Fri, 28 Dec 2012 02:04:52 -0800 (PST)
From: Tapomay Dey <[email protected]>
To: "Communication channel for developers of the MonetDB suite."
<[email protected]>
Subject: Re: Unique key constraint violation violated
Message-ID:
<[email protected]>
Content-Type: text/plain; charset="us-ascii"
Confirmed that it occurs even with MonetDataSource.
Also tried switching to July12 SP2.
Need pointers to diagnosing it further.
Any ideas?
________________________________
From: Tapomay Dey <[email protected]>
To: "[email protected]" <[email protected]>
Sent: Thursday, December 27, 2012 9:15 PM
Subject: Unique key constraint violation violated
Is this a known issue that you are working on? If so please give me the bug id
where I would know when its fixed.
The case is as follows:
I have a table with 32 columns with unique key on 3 of them.
I am running live ETLs for putting data into monetdb. These ETLs are generating
sql insert scripts.(inserts/updates/ddls)
Using JDBC and c3p0 connection pooling.
Have developed an external wrapper module to achieve master-master replication.
One common consistency check would be a count*.
Observation:
Queries are run in batches of 100 inserts clubbed into a single insert.
I do a select count(*) after each batch of 100.
After running 9100 queries the count* on one DB was 9100 but in the other was
9200.
The second DB had 100 records that were duplicates of 8901-9000.
It means when I inserted 9001 to 9100 the previous batch went into the store
again with it and somehow surpassed the unique key constraint check.
I have also observed another case where count(*) gave less results than actual
no. of rows in the DB.
REQUEST:
Just want to confirm if such kind of issues are part of active development.
Otherwise it would be really difficult for me to push monetdb usage into
production.
Will try again with MonetDataSource instead of c3p0. But I doubt if the issues
are client side or connection-related.
Plz note that the keys for which this occurs is random and thus I concluded
it's not a data issue.
Regards,
Tapomay Dey.
Software Design Engg.
Sokrati Inc.
Email: tapomayATsokratiDOTcom
_______________________________________________
developers-list mailing list
[email protected]
http://mail.monetdb.org/mailman/listinfo/developers-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.monetdb.org/pipermail/developers-list/attachments/20121228/80c0c405/attachment.html>
------------------------------
_______________________________________________
developers-list mailing list
[email protected]
http://mail.monetdb.org/mailman/listinfo/developers-list
End of developers-list Digest, Vol 4, Issue 19
**********************************************
------------------------------------------------------------------------------
Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and
much more. Get web development skills now with LearnDevNow -
350+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122812
_______________________________________________
Monetdb-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-developers
