Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-27 Thread Jeff Davis
On Mon, 2009-10-26 at 17:23 +, Dean Rasheed wrote: > If it's of any relevance, I'm currently using an optimised build, with > assert checking off. > [Linux x86_64, 2 core Intel Core2] Ok, I'm able to reproduce it now. Thanks for looking into it! Regards, Jeff Davis -- Sent via pgsq

Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-26 Thread Robert Haas
On Mon, Oct 26, 2009 at 9:46 AM, Simon Riggs wrote: > On Mon, 2009-10-26 at 13:28 +, Dean Rasheed wrote: > >> It works for all kinds of trigger events, >> and is intended as a complete drop-in replacement for the after >> triggers queue. > >> > All of those seem false in the general case. What

Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-26 Thread Dean Rasheed
2009/10/26 Jeff Davis : > On Mon, 2009-10-26 at 13:41 +, Dean Rasheed wrote: >> I did a quick bit of testing, and I think that there is a >> locking/concurrency problem :-( > > Unfortunately I can't reproduce the problem on my machine; it always > passes. > That's odd. It happens every time on

Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-26 Thread Dean Rasheed
2009/10/26 Simon Riggs : > On Mon, 2009-10-26 at 13:28 +, Dean Rasheed wrote: > >> It works for all kinds of trigger events, >> and is intended as a complete drop-in replacement for the after >> triggers queue. > >> > All of those seem false in the general case. What will you do? >> >> At this

Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-26 Thread Jeff Davis
On Mon, 2009-10-26 at 13:41 +, Dean Rasheed wrote: > I did a quick bit of testing, and I think that there is a > locking/concurrency problem :-( Unfortunately I can't reproduce the problem on my machine; it always passes. If you have a minute, can you try to determine if the problem can happe

Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-26 Thread Simon Riggs
On Mon, 2009-10-26 at 13:28 +, Dean Rasheed wrote: > It works for all kinds of trigger events, > and is intended as a complete drop-in replacement for the after > triggers queue. > > All of those seem false in the general case. What will you do? > > At this point I'm looking for more feedba

Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-26 Thread Dean Rasheed
2009/10/25 Jeff Davis : > On Mon, 2009-10-19 at 17:48 +0100, Dean Rasheed wrote: >> This is a WIP patch to replace the after-trigger queues with TID bitmaps >> to prevent them from using excessive amounts of memory. Each round of >> trigger executions is a modified bitmap heap scan. > > Can you ple

Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-26 Thread Dean Rasheed
2009/10/25 Simon Riggs : > On Mon, 2009-10-19 at 17:48 +0100, Dean Rasheed wrote: > >> This is a WIP patch to replace the after-trigger queues with TID bitmaps >> to prevent them from using excessive amounts of memory. Each round of >> trigger executions is a modified bitmap heap scan. > > This is

Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-25 Thread Jeff Davis
On Mon, 2009-10-19 at 17:48 +0100, Dean Rasheed wrote: > This is a WIP patch to replace the after-trigger queues with TID bitmaps > to prevent them from using excessive amounts of memory. Each round of > trigger executions is a modified bitmap heap scan. Can you please take a look at my patch here

Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-25 Thread Simon Riggs
On Mon, 2009-10-19 at 17:48 +0100, Dean Rasheed wrote: > This is a WIP patch to replace the after-trigger queues with TID bitmaps > to prevent them from using excessive amounts of memory. Each round of > trigger executions is a modified bitmap heap scan. This is an interesting patch. The justific

Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-19 Thread Dean Rasheed
2009/10/19 Robert Haas : > On Mon, Oct 19, 2009 at 12:48 PM, Dean Rasheed > wrote: >> This is a WIP patch to replace the after-trigger queues with TID bitmaps >> to prevent them from using excessive amounts of memory. Each round of >> trigger executions is a modified bitmap heap scan. > > If the b

Re: [HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-19 Thread Robert Haas
On Mon, Oct 19, 2009 at 12:48 PM, Dean Rasheed wrote: > This is a WIP patch to replace the after-trigger queues with TID bitmaps > to prevent them from using excessive amounts of memory. Each round of > trigger executions is a modified bitmap heap scan. If the bitmap becomes lossy, how do you pre

[HACKERS] Scaling up deferred unique checks and the after trigger queue

2009-10-07 Thread Dean Rasheed
I've started looking at the following TODO item: "Improve deferrable unique constraints for cases with many conflicts" and Tom's suggestion that the rows to be checked can be stored in a bitmap, which would become lossy when the number of rows becomes large enough. There is also another TODO it