Re: [HACKERS] Design for In-Core Logical Replication

On 29 July 2016 at 16:53, Robert Haas wrote: > I think that to really understand exactly what you and Petr have in > mind, we'd need a description of where publication and subscription > data is stored within the server, and exactly what gets stored. > Perhaps that will

Re: [HACKERS] Design for In-Core Logical Replication

Robert Haas wrote: > One minor comment is that this document makes extensive use of Terms > With Initial Capitals, which seems stylistically odd, although I'm not > precisely sure what would be better. We use publication on the first use only, which is turned into italics. -- Álvaro Herrera

Re: [HACKERS] Design for In-Core Logical Replication

On Wed, Jul 20, 2016 at 4:08 AM, Simon Riggs wrote: > In this post, Petr and I present a joint view on a design for how this > should work in-core, based upon our implementation experiences with physical > replication, pglogical and various comments so far. > > Note that

Re: [HACKERS] Design for In-Core Logical Replication

On 26/07/16 00:05, Hannu Krosing wrote: CREATE PUBLICATION mypub; ALTER PUBLICATION mypub ADD TABLE users, departments; Would a subscription just be a logical grouping or would it be something stronger like meaning atomic subscriptions and/or a dedicated replication slot ? Not sure what

Re: [HACKERS] Design for In-Core Logical Replication

On 07/20/2016 10:08 AM, Simon Riggs wrote: > > Monitoring > > pg_stat_replication > > > pg_stat_subscription > > and probably also `pg_stat_publication` or some other way to see, what tables are currently in a PUBLICATION, who has subscribed etc. > > CREATE PUBLICATION

Re: [HACKERS] Design for In-Core Logical Replication

On 7/20/16 8:35 PM, Craig Ringer wrote: I'm not sure how Petr's current design for in-core replication addresses this, if it does, or whether it's presently focused only on point-to-point replication like pglogical. As far as I'm concerned so long as it does direct point-to-point replication

Re: [HACKERS] Design for In-Core Logical Replication

ISTM pg_dump should alter publication/subscription when doing partial dumps, similar to how it handles foreign keys. Subscribed tables should be read-only by default. If you need something more advanced than that adding an extra option to the subscription (or wherever else) is going to be the

Re: [HACKERS] Design for In-Core Logical Replication

On 22 July 2016 at 22:55, Simon Riggs wrote: > > Filtering data at both the sender and receiver has been requested, so > I guess it makes sense to have a WHERE clause on both the Publication > and the Subscription, but I will think more on that. > Yeah, output plugin

Re: [HACKERS] Design for In-Core Logical Replication

On 22 July 2016 at 09:54, Dmitriy Sarafannikov wrote: > >> >>CREATE PUBLICATION mypub; >>ALTER PUBLICATION mypub ADD TABLE users, departments; > >>CREATE SUBSCRIPTION mysub WITH CONNECTION dbname=foo host=bar >>user=repuser PUBLICATION mypub; > >>The above will start

[HACKERS] Re: [HACKERS] Design for In-Core Logical Replication

> >CREATE PUBLICATION mypub; >ALTER PUBLICATION mypub ADD TABLE users, departments; >CREATE SUBSCRIPTION mysub WITH CONNECTION dbname=foo host=bar >user=repuser PUBLICATION mypub; >    The above will start the replication process which synchronizes the >    initial table contents of users and

Re: [HACKERS] Design for In-Core Logical Replication

On 21 July 2016 at 11:05, Joshua D. Drake wrote: > On 07/20/2016 06:35 PM, Craig Ringer wrote: > > First, I'd like to emphasise that logical replication has been stalled >> for ages now because we can no longer make forward progress on core >> features needed until we

Re: [HACKERS] Design for In-Core Logical Replication

On 07/20/2016 06:35 PM, Craig Ringer wrote: First, I'd like to emphasise that logical replication has been stalled for ages now because we can no longer make forward progress on core features needed until we have in-core logical replication (they're dismissed as irrelevant, no in core users,

Re: [HACKERS] Design for In-Core Logical Replication

On 21 July 2016 at 01:20, Simon Riggs wrote: > On 20 July 2016 at 17:52, Rod Taylor wrote: > > >> I think it's important for communication channels to be defined >> separately from the subscriptions. >> > > I agree and believe it will be that way. >

Re: [HACKERS] Design for In-Core Logical Replication

On 20/07/16 19:07, Simon Riggs wrote: On 20 July 2016 at 16:39, Joshua D. Drake > wrote: Logical Replication uses a Publish and Subscribe model with one or more Subscribers subscribing to one

Re: [HACKERS] Design for In-Core Logical Replication

On 20 July 2016 at 17:52, Rod Taylor wrote: > I think it's important for communication channels to be defined separately > from the subscriptions. > I agree and believe it will be that way. Craig is working on allowing Replication Slots to failover between nodes, to

Re: [HACKERS] Design for In-Core Logical Replication

On 20 July 2016 at 16:39, Joshua D. Drake wrote: > Logical Replication is a method of replicating data objects and their >> changes, based upon their Primary Keys (or Replication Identity). We >> > > Do we want a limitation based on Primary Key, or would it be

Re: [HACKERS] Design for In-Core Logical Replication

On Wed, Jul 20, 2016 at 4:08 AM, Simon Riggs wrote: > > > And on Subscriber database: > > CREATE SUBSCRIPTION mysub WITH CONNECTION dbname=foo host=bar > user=repuser PUBLICATION mypub; > > > > The above will start the replication process which

Re: [HACKERS] Design for In-Core Logical Replication

On 07/20/2016 01:08 AM, Simon Riggs wrote: Logical Replication Logical Replication is a method of replicating data objects and their changes, based upon their Primary Keys (or Replication Identity). We Do we want a limitation based on Primary Key, or would it be possible

[HACKERS] Design for In-Core Logical Replication

At PgCon we discussed that Petr Jelinek would be working on the code for an in-core logical replication implementation, while I would work on user interface/security models. Petr has been actively working on the code and will post patch in a few weeks, as discussed and agreed. Craig Ringer is also