Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-22 Thread Kevin Grittner
Simon Riggs wrote: > Greg Stark wrote: >> Tom Lane wrote: >>> Isn't there an even more serious problem, namely that this >>> assumes *all* transactions are serializable? Do you mean in terms of the serializable transaction isolation level, or something else? I haven't read the patches, but I've

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-15 Thread Tatsuo Ishii
> The design Andres and Simon have advanced already eliminates a lot of > the common failure cases (now(), random(), nextval()) suffered by pgPool > and similar tools. But remember, this feature doesn't have to be Well, pgpool-II already solved the now() case by using query rewriting technique. T

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Steve Singer
On 12-10-11 06:27 PM, Josh Berkus wrote: On 10/10/12 7:26 PM, Bruce Momjian wrote: How does Slony write its changes without causing serialization replay conflicts? Since nobody from the Slony team answered this: a) Slony replicates *rows*, not *statements* True, but the proposed logical repl

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Simon Riggs
On 11 October 2012 03:16, Greg Stark wrote: > On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane wrote: >>> I think I've mentioned it before, but in the interest of not being >>> seen to critique the bikeshed only after it's been painted: this >>> design gives up something very important that exists in ou

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Christopher Browne
On Wed, Oct 10, 2012 at 10:26 PM, Bruce Momjian wrote: > How does Slony write its changes without causing serialization replay > conflicts? It uses a sequence to break any ordering conflicts at the time that data is inserted into its log tables. If there are two transactions, A and B, that were

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Josh Berkus
On 10/10/12 7:26 PM, Bruce Momjian wrote: > How does Slony write its changes without causing serialization replay > conflicts? Since nobody from the Slony team answered this: a) Slony replicates *rows*, not *statements* b) Slony uses serializable mode internally for row replication c) it's possib

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Hannu Krosing
On 10/11/2012 03:10 AM, Robert Haas wrote: On Wed, Oct 10, 2012 at 7:02 PM, Peter Geoghegan wrote: The purpose of ApplyCache/transaction reassembly is to reassemble interlaced records, and organise them by XID, so that the consumer client code sees only streams (well, lists) of records split by

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Hannu Krosing
On 10/11/2012 04:31 AM, Tom Lane wrote: Greg Stark writes: On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane wrote: Isn't there an even more serious problem, namely that this assumes *all* transactions are serializable? What happens when they aren't? Or even just that the effective commit order is n

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Andres Freund
On Thursday, October 11, 2012 04:49:20 AM Andres Freund wrote: > On Thursday, October 11, 2012 04:31:21 AM Tom Lane wrote: > > Greg Stark writes: > > > On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane wrote: > > >> Isn't there an even more serious problem, namely that this assumes > > >> all transaction

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Andres Freund
On Thursday, October 11, 2012 04:31:21 AM Tom Lane wrote: > Greg Stark writes: > > On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane wrote: > >> Isn't there an even more serious problem, namely that this assumes > >> *all* transactions are serializable? What happens when they aren't? > >> Or even just t

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Andres Freund
On Thursday, October 11, 2012 04:16:39 AM Greg Stark wrote: > On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane wrote: > >> I think I've mentioned it before, but in the interest of not being > >> seen to critique the bikeshed only after it's been painted: this > >> design gives up something very important

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Andres Freund
On Thursday, October 11, 2012 03:10:48 AM Robert Haas wrote: > On Wed, Oct 10, 2012 at 7:02 PM, Peter Geoghegan wrote: > > The purpose of ApplyCache/transaction reassembly is to reassemble > > interlaced records, and organise them by XID, so that the consumer > > client code sees only streams (we

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Tom Lane
Greg Stark writes: > On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane wrote: >> Isn't there an even more serious problem, namely that this assumes >> *all* transactions are serializable? What happens when they aren't? >> Or even just that the effective commit order is not XID order? > I don't think it

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Bruce Momjian
On Thu, Oct 11, 2012 at 03:16:39AM +0100, Greg Stark wrote: > On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane wrote: > >> I think I've mentioned it before, but in the interest of not being > >> seen to critique the bikeshed only after it's been painted: this > >> design gives up something very important

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Andres Freund
Hi, First of: Awesome review. On Thursday, October 11, 2012 01:02:26 AM Peter Geoghegan wrote: > What follows is an initial overview of the patch (or at least my > understanding of the patch, which you may need to correct), and some > points of concern. > > > * applycache module which reassemble

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Greg Stark
On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane wrote: >> I think I've mentioned it before, but in the interest of not being >> seen to critique the bikeshed only after it's been painted: this >> design gives up something very important that exists in our current >> built-in replication solution, namely

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Tom Lane
Robert Haas writes: > On Wed, Oct 10, 2012 at 7:02 PM, Peter Geoghegan > wrote: >> The purpose of ApplyCache/transaction reassembly is to reassemble >> interlaced records, and organise them by XID, so that the consumer >> client code sees only streams (well, lists) of records split by XID. > I

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Bruce Momjian
On Wed, Oct 10, 2012 at 09:10:48PM -0400, Robert Haas wrote: > > You consider this to be a throw-away function that won't ever be > > committed. However, I strongly feel that you should move it into > > /contrib, so that it can serve as a sort of reference implementation > > for authors of decoder

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Robert Haas
On Wed, Oct 10, 2012 at 7:02 PM, Peter Geoghegan wrote: > The purpose of ApplyCache/transaction reassembly is to reassemble > interlaced records, and organise them by XID, so that the consumer > client code sees only streams (well, lists) of records split by XID. I think I've mentioned it before,

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Bruce Momjian
On Thu, Oct 11, 2012 at 01:34:58AM +0200, anara...@anarazel.de wrote: > > > Bruce Momjian schrieb: > > >On Thu, Oct 11, 2012 at 12:02:26AM +0100, Peter Geoghegan wrote: > >> On 15 September 2012 01:39, Andres Freund > >wrote: > >> > (0008-Introduce-wal-decoding-via-catalog-timetravel.patch) >

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread anara...@anarazel.de
Bruce Momjian schrieb: >On Thu, Oct 11, 2012 at 12:02:26AM +0100, Peter Geoghegan wrote: >> On 15 September 2012 01:39, Andres Freund >wrote: >> > (0008-Introduce-wal-decoding-via-catalog-timetravel.patch) >> >> This patch is the 8th of 8 in a patch series that covers different >> aspects of

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Josh Berkus
> Does this design allow replication/communcation between clusters running > different major versions of Postgres? Just catching up on your email, hmmm? Yes, that's part of the design 2Q presented. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Bruce Momjian
On Thu, Oct 11, 2012 at 12:02:26AM +0100, Peter Geoghegan wrote: > On 15 September 2012 01:39, Andres Freund wrote: > > (0008-Introduce-wal-decoding-via-catalog-timetravel.patch) > > This patch is the 8th of 8 in a patch series that covers different > aspects of the bi-directional replication fea

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Peter Geoghegan
On 15 September 2012 01:39, Andres Freund wrote: > (0008-Introduce-wal-decoding-via-catalog-timetravel.patch) This patch is the 8th of 8 in a patch series that covers different aspects of the bi-directional replication feature planned for PostgreSQL 9.3. For those that are unfamiliar with the BDR