Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-14 Thread Andres Freund
On 2012-12-13 21:40:43 +0100, Andres Freund wrote: > On 2012-12-13 11:02:06 -0500, Steve Singer wrote: > > On 12-12-12 06:20 AM, Andres Freund wrote: > > >>Possible solutions: > > >>1) INIT_LOGICAL_REPLICATION waits for an answer from the client that > > >>confirms that logical replication initiali

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-13 Thread Robert Haas
On Thu, Dec 13, 2012 at 5:55 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Dec 13, 2012 at 5:25 PM, Andres Freund >> wrote: >>> That would solve the consistency problem, yes. Would we need a special >>> kind of permission for this? I would say superuser/database owner only? > >> Yeah, I

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-13 Thread Tom Lane
Robert Haas writes: > On Thu, Dec 13, 2012 at 5:25 PM, Andres Freund wrote: >> That would solve the consistency problem, yes. Would we need a special >> kind of permission for this? I would say superuser/database owner only? > Yeah, I doubt we would need a whole new permission for it, but it > w

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-13 Thread Robert Haas
On Thu, Dec 13, 2012 at 5:25 PM, Andres Freund wrote: > That would solve the consistency problem, yes. Would we need a special > kind of permission for this? I would say superuser/database owner only? Yeah, I doubt we would need a whole new permission for it, but it would certainly have to be dis

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-13 Thread Andres Freund
On 2012-12-13 17:11:31 -0500, Robert Haas wrote: > On Tue, Dec 11, 2012 at 10:20 PM, Tom Lane wrote: > >> This is why the pg_dump master process executes a > >> lock table in access share mode > >> for every table, so your commands would all block. > > > > A lock doesn't protect against schema ch

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-13 Thread Robert Haas
On Tue, Dec 11, 2012 at 10:20 PM, Tom Lane wrote: >> This is why the pg_dump master process executes a >> lock table in access share mode >> for every table, so your commands would all block. > > A lock doesn't protect against schema changes made before the lock was > taken. The reason that the

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-13 Thread Andres Freund
On 2012-12-13 11:02:06 -0500, Steve Singer wrote: > On 12-12-12 06:20 AM, Andres Freund wrote: > >>Possible solutions: > >>1) INIT_LOGICAL_REPLICATION waits for an answer from the client that > >>confirms that logical replication initialization is finished. Before > >>that the walsender connection

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-13 Thread Steve Singer
On 12-12-12 06:20 AM, Andres Freund wrote: Possible solutions: 1) INIT_LOGICAL_REPLICATION waits for an answer from the client that confirms that logical replication initialization is finished. Before that the walsender connection cannot be used for anything else. 2) we remove the snapshot as so

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-12 Thread Andres Freund
On 2012-12-12 18:52:33 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2012-12-12 12:13:44 +0100, Andres Freund wrote: > >> This morning I wondered whether we couldn't protect against that by > >> acquiring share locks on the catalog rows pg_dump reads, that would > >> result in "could not s

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-12 Thread Tom Lane
Andres Freund writes: > On 2012-12-12 12:13:44 +0100, Andres Freund wrote: >> This morning I wondered whether we couldn't protect against that by >> acquiring share locks on the catalog rows pg_dump reads, that would >> result in "could not serialize access due to concurrent update" type of >> err

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-12 Thread Andres Freund
On 2012-12-12 12:13:44 +0100, Andres Freund wrote: > On 2012-12-11 22:20:18 -0500, Tom Lane wrote: > > Joachim Wieland writes: > > > On Tue, Dec 11, 2012 at 6:52 PM, Andres Freund > > > wrote: > > >> One problem I see is that while exporting a snapshot solves the > > >> visibility issues of the

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-12 Thread Andres Freund
On 2012-12-11 22:39:14 -0500, Steve Singer wrote: > On 12-12-11 06:52 PM, Andres Freund wrote: > >Hi, > > > > >Problem 1: > > > >One problem I see is that while exporting a snapshot solves the > >visibility issues of the table's contents it does not protect against > >schema changes. I am not sure

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-12 Thread Andres Freund
On 2012-12-11 22:20:18 -0500, Tom Lane wrote: > Joachim Wieland writes: > > On Tue, Dec 11, 2012 at 6:52 PM, Andres Freund > > wrote: > >> One problem I see is that while exporting a snapshot solves the > >> visibility issues of the table's contents it does not protect against > >> schema change

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-12 Thread Andres Freund
On 2012-12-11 21:05:51 -0500, Joachim Wieland wrote: > On Tue, Dec 11, 2012 at 6:52 PM, Andres Freund wrote: > > One problem I see is that while exporting a snapshot solves the > > visibility issues of the table's contents it does not protect against > > schema changes. I am not sure whether thats

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-11 Thread Steve Singer
On 12-12-11 06:52 PM, Andres Freund wrote: Hi, Problem 1: One problem I see is that while exporting a snapshot solves the visibility issues of the table's contents it does not protect against schema changes. I am not sure whether thats a problem. If somebody runs a CLUSTER or something like

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-11 Thread Tom Lane
Joachim Wieland writes: > On Tue, Dec 11, 2012 at 6:52 PM, Andres Freund wrote: >> One problem I see is that while exporting a snapshot solves the >> visibility issues of the table's contents it does not protect against >> schema changes. I am not sure whether thats a problem. >> >> If somebody

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-11 Thread Joachim Wieland
On Tue, Dec 11, 2012 at 6:52 PM, Andres Freund wrote: > One problem I see is that while exporting a snapshot solves the > visibility issues of the table's contents it does not protect against > schema changes. I am not sure whether thats a problem. > > If somebody runs a CLUSTER or something like

[HACKERS] Logical decoding & exported base snapshot

2012-12-11 Thread Andres Freund
Hi, When initiating a new logical replication "slot" I want to provide a 'SET TRANSACTION SNAPSHOT'able snapshot which can be used to setup a new replica. I have some questions arround this where I could use some input on. First, some basics around how this currently works: Test the other side: