On Tue, 2007-07-03 at 12:33 -0400, Christopher Browne wrote:
> I'll see about doing an experiment on this to see if, for the DELETE
> case, it seems to actually help. It may be that the performance
> effects are small to none, so that the added code complication isn't
> worthwhile.
>
In a simple test I ran, DELETE of the entire 5M record table using
sequential scan was MUCH faster (9.41s) than 5M individual DELETE
statements in a single transaction (552.49s).
5M records is small enough to fit into memory. I expect the difference
would be even greater when the index and table can't both fit into
memory and the deletes are distributed randomly over the table.
I think it is worth exploring ways of solving this problem. Right now
slony is great for small inserts, updates, and deletes. But any large
update/delete on the origin can cause the subscribers to fall way
behind.
> Remember, the thought we started with was:
> "What if we could do something that would make mass operations less
> expensive?"
>
> I don't want to introduce anything that can materially increase
> processing costs.
>
> The more intelligent we try to get, the more expensive the
> logtrigger() function gets, and if the price is high enough, then we
> gain nothing.
>
> The only "win" I see is if we can opportunistically join some
> statements together. If we have to make the log trigger function
> universally *WAY* more expensive, well, that's a performance loss :-(.
Is there any way that we could detect (even without 100% confidence)
that a transaction is "big" and we should spend more effort trying to
optimize it?
I know that's a big project (in the general case), but there might be
some simple things that would work.
Regards,
Jeff Davis
_______________________________________________
Slony1-general mailing list
[email protected]
http://lists.slony.info/mailman/listinfo/slony1-general
