I agree - making chunk size configurable would be best.
-ar
On 12/14/2012 03:37 PM, Rosser Schwarz wrote:
I can't speak to your proposed algorithmic changes, but I do think
chunk size should maybe be configurable, to help deal with low-memory
environments like the OP's, and to allow DBAs to set larger chunk
sizes, when they've also set max_stack_depth and the relevant ulimits
accordingly.
On Fri, Dec 14, 2012 at 12:41 PM, Armin Resch <[email protected]
<mailto:[email protected]>> wrote:
Bucardoans,
we had a situation in which our memory-constrained device would
pick bucardo_ctl or a postgres backend as a candidate to be killed
when more than 3,000 deltas would have to be copied to a slave in
one transaction. I attached a patch to correct the situation and
was wondering whether it can be considered to be applied upstream
- at least for B4; it applies to the PUSHDELTA section. In
addition to the smaller chunk size (which admittedly might not be
ideal for larger systems), it contains the following changes:
1 - Allow tables to be replicated which contain unique keys where
one or more columns are NULL
2 - performance improvement by refactoring processing of the delta
record set
With regard to (1), we found out that Bucardo would not replicate
unique keys containing NULLs because the Postgres syntax does not
produce output for the following WHERE clause:
WHERE (col1,col2,col3,col4) IN
(('A','B',NULL,'D'),('A',NULL,'C',NULL))
The algorithm would split those COPY operations into two separate
queries as such:
A) <blah> WHERE (col1,col2,col4) IN (('A','B','D')) AND col3 IS NULL
B) <blah> WHERE (col1,col3) IN (('A','C') AND col2 IS NULL AND
col4 IS NULL
Best,
-ar
_______________________________________________
Bucardo-general mailing list
[email protected] <mailto:[email protected]>
https://mail.endcrypt.com/mailman/listinfo/bucardo-general
--
:wq
_______________________________________________
Bucardo-general mailing list
[email protected]
https://mail.endcrypt.com/mailman/listinfo/bucardo-general
