Re: [HACKERS] Logical replication existing data copy

(At the moment using these patches for tests:) 0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch + 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+ 0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch +

Re: [HACKERS] Logical replication existing data copy

On 29/03/17 10:14, Erik Rijkers wrote: > On 2017-03-09 11:06, Erik Rijkers wrote: I use three different machines (2 desktop, 1 server) to test logical replication, and all three have now at least once failed to correctly synchronise a pgbench session (amidst many succesful

Re: [HACKERS] Logical replication existing data copy

On 2017-03-09 11:06, Erik Rijkers wrote: I use three different machines (2 desktop, 1 server) to test logical replication, and all three have now at least once failed to correctly synchronise a pgbench session (amidst many succesful runs, of course) (At the moment using tese patches for

Re: [HACKERS] Logical replication existing data copy

On 3/25/17 00:45, Mark Kirkwood wrote: > Minor niggle: > > bench=# DROP PUBLICATION pgbench; > DROP STATISTICS <=== > > I'm guessing that notification is wrong. Fixed. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] Logical replication existing data copy

On 25/03/17 07:52, Peter Eisentraut wrote: On 3/24/17 10:09, Petr Jelinek wrote: On 24/03/17 15:05, Peter Eisentraut wrote: On 3/23/17 19:32, Petr Jelinek wrote: Yes, I also forgot to check if the table actually exists on subscriber when fetching them in CREATE SUBSCRIPTION (we have check

Re: [HACKERS] Logical replication existing data copy

Am Freitag, den 24.03.2017, 14:57 -0400 schrieb Peter Eisentraut: > On 3/24/17 05:22, Michael Banck wrote: > > However, replication also seems to not work, I'm using the following > > script right now: > > The problem is that your publication does not include any tables. Oops, of course. That

Re: [HACKERS] Logical replication existing data copy

On 3/24/17 05:22, Michael Banck wrote: > However, replication also seems to not work, I'm using the following > script right now: The problem is that your publication does not include any tables. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

Re: [HACKERS] Logical replication existing data copy

On 3/24/17 10:09, Petr Jelinek wrote: > On 24/03/17 15:05, Peter Eisentraut wrote: >> On 3/23/17 19:32, Petr Jelinek wrote: >>> Yes, I also forgot to check if the table actually exists on subscriber >>> when fetching them in CREATE SUBSCRIPTION (we have check during >>> replication but not there).

Re: [HACKERS] Logical replication existing data copy

On 24/03/17 15:05, Peter Eisentraut wrote: > On 3/23/17 19:32, Petr Jelinek wrote: >> Yes, I also forgot to check if the table actually exists on subscriber >> when fetching them in CREATE SUBSCRIPTION (we have check during >> replication but not there). > > I think for this we can probably just

Re: [HACKERS] Logical replication existing data copy

On 3/23/17 19:32, Petr Jelinek wrote: > Yes, I also forgot to check if the table actually exists on subscriber > when fetching them in CREATE SUBSCRIPTION (we have check during > replication but not there). I think for this we can probably just change the missing_ok argument of RangeVarGetRelid()

Re: [HACKERS] Logical replication existing data copy

On 24/03/17 11:23, Erik Rijkers wrote: > On 2017-03-24 10:45, Mark Kirkwood wrote: >> >> However one minor observation - as Michael Banck noted - the elapsed >> time for slave to catch up after running: >> >> $ pgbench -c8 -T600 bench >> >> on the master was (subjectively) much longer than for

Re: [HACKERS] Logical replication existing data copy

On 2017-03-24 10:45, Mark Kirkwood wrote: However one minor observation - as Michael Banck noted - the elapsed time for slave to catch up after running: $ pgbench -c8 -T600 bench on the master was (subjectively) much longer than for physical streaming replication. Is this expected? I think

Re: [HACKERS] Logical replication existing data copy

On 24/03/17 12:32, Petr Jelinek wrote: On 24/03/17 00:14, Mark Kirkwood wrote: On 24/03/17 02:00, Peter Eisentraut wrote: On 3/21/17 21:38, Peter Eisentraut wrote: This patch is looking pretty good to me, modulo the failing pg_dump tests. Attached is a fixup patch. I have mainly updated

Re: [HACKERS] Logical replication existing data copy

Hi, On Fri, Mar 24, 2017 at 12:32:28AM +0100, Petr Jelinek wrote: > Yes, I also forgot to check if the table actually exists on subscriber > when fetching them in CREATE SUBSCRIPTION (we have check during > replication but not there). > > Attached patches should fix both issues. I no longer get

Re: [HACKERS] Logical replication existing data copy

On 24/03/17 00:14, Mark Kirkwood wrote: > On 24/03/17 02:00, Peter Eisentraut wrote: >> On 3/21/17 21:38, Peter Eisentraut wrote: >>> This patch is looking pretty good to me, modulo the failing pg_dump >>> tests. >>> >>> Attached is a fixup patch. I have mainly updated some comments and >>>

Re: [HACKERS] Logical replication existing data copy

On 24/03/17 02:00, Peter Eisentraut wrote: On 3/21/17 21:38, Peter Eisentraut wrote: This patch is looking pretty good to me, modulo the failing pg_dump tests. Attached is a fixup patch. I have mainly updated some comments and variable naming for (my) clarity. No functional changes.

Re: [HACKERS] Logical replication existing data copy

Hi, On 23/03/17 23:06, Michael Banck wrote: > Hi, > > On Thu, Mar 23, 2017 at 09:00:16AM -0400, Peter Eisentraut wrote: >> On 3/21/17 21:38, Peter Eisentraut wrote: >>> This patch is looking pretty good to me, modulo the failing pg_dump tests. >>> >>> Attached is a fixup patch. I have mainly

Re: [HACKERS] Logical replication existing data copy

Hi, On Thu, Mar 23, 2017 at 09:00:16AM -0400, Peter Eisentraut wrote: > On 3/21/17 21:38, Peter Eisentraut wrote: > > This patch is looking pretty good to me, modulo the failing pg_dump tests. > > > > Attached is a fixup patch. I have mainly updated some comments and > > variable naming for

Re: [HACKERS] Logical replication existing data copy

On 3/21/17 21:38, Peter Eisentraut wrote: > This patch is looking pretty good to me, modulo the failing pg_dump tests. > > Attached is a fixup patch. I have mainly updated some comments and > variable naming for (my) clarity. No functional changes. Committed all that. -- Peter Eisentraut

Re: [HACKERS] Logical replication existing data copy

On 22/03/17 13:36, Peter Eisentraut wrote: > On 3/20/17 16:08, Peter Eisentraut wrote: >> The current patch causes a failure in the pg_dump tests, because the >> generated CREATE SUBSCRIPTION commands make connection attempts that >> don't work. We have the pg_dump option

Re: [HACKERS] Logical replication existing data copy

On 3/20/17 16:08, Peter Eisentraut wrote: > The current patch causes a failure in the pg_dump tests, because the > generated CREATE SUBSCRIPTION commands make connection attempts that > don't work. We have the pg_dump option --no-create-subscription-slots > for this, but I suppose we should

Re: [HACKERS] Logical replication existing data copy

This patch is looking pretty good to me, modulo the failing pg_dump tests. Attached is a fixup patch. I have mainly updated some comments and variable naming for (my) clarity. No functional changes. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] Logical replication existing data copy

On 3/20/17 19:54, Petr Jelinek wrote: > Here is fixed version, also rebased on top of all the changes to pg_dump > tests. Subscriptions are dumped unless --no-subscriptions is specified. The problem with that is that pg_dump will now fail for unprivileged users. That's a separate problem to

Re: [HACKERS] Logical replication existing data copy

Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > The current patch causes a failure in the pg_dump tests, because the > generated CREATE SUBSCRIPTION commands make connection attempts that > don't work. We have the pg_dump option --no-create-subscription-slots > for this,

Re: [HACKERS] Logical replication existing data copy

The current patch causes a failure in the pg_dump tests, because the generated CREATE SUBSCRIPTION commands make connection attempts that don't work. We have the pg_dump option --no-create-subscription-slots for this, but I suppose we should expand that to --no-subscription-connect and use the

Re: [HACKERS] Logical replication existing data copy

On 3/14/17 12:12, Petr Jelinek wrote: >> Committing this in chunks makes sense anyway. >> >> I've attached a slightly version that makes pg_recvlogical skip slot >> export. The second patch is unchanged, use the copy from the >> immediately prior mail. >> > > Okay, I merged your changes in. > >

Re: [HACKERS] Logical replication existing data copy

On 11 March 2017 at 00:33, Petr Jelinek wrote: > On 09/03/17 18:48, Peter Eisentraut wrote: >> On 3/6/17 05:27, Petr Jelinek wrote: >>> And lastly I changed the automagic around exporting, not exporting and >>> using the snapshot produced by CREATE_REPLICATION_SLOT

Re: [HACKERS] Logical replication existing data copy

On 09/03/17 18:48, Peter Eisentraut wrote: > On 3/6/17 05:27, Petr Jelinek wrote: >> And lastly I changed the automagic around exporting, not exporting and >> using the snapshot produced by CREATE_REPLICATION_SLOT into explicit >> parameter for the CREATE_REPLICATION_SLOT command (and added simple

Re: [HACKERS] Logical replication existing data copy

On 09/03/17 18:46, Peter Eisentraut wrote: > On 3/6/17 05:27, Petr Jelinek wrote: >> updated and rebased version of the patch attached. > > Some comments on this patch: > > Can we have a better explanation of different snapshot options in > CREATE_REPLICATION_SLOT. We use all these variants in

Re: [HACKERS] Logical replication existing data copy

On 3/6/17 05:27, Petr Jelinek wrote: > And lastly I changed the automagic around exporting, not exporting and > using the snapshot produced by CREATE_REPLICATION_SLOT into explicit > parameter for the CREATE_REPLICATION_SLOT command (and added simple > framework for adding more of those if needed

Re: [HACKERS] Logical replication existing data copy

On 3/6/17 05:27, Petr Jelinek wrote: > updated and rebased version of the patch attached. Some comments on this patch: Can we have a better explanation of different snapshot options in CREATE_REPLICATION_SLOT. We use all these variants in different places, but it's not fully documented why.

Re: [HACKERS] Logical replication existing data copy

On 2017-03-09 11:06, Erik Rijkers wrote: On 2017-03-08 10:36, Petr Jelinek wrote: On 07/03/17 23:30, Erik Rijkers wrote: On 2017-03-06 11:27, Petr Jelinek wrote: 0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch + 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+

Re: [HACKERS] Logical replication existing data copy

On 2017-03-09 11:06, Erik Rijkers wrote: file Name: logrep.20170309_1021.1.1043.scale_25.clients_64.NOK.log 20170309_1021 is the start-time of the script 1 is master (2 is replica) 1043 is the time, 10:43, just before the pg_waldump call Sorry, that might be confusing. That 10:43 is the

Re: [HACKERS] Logical replication existing data copy

On 07/03/17 23:30, Erik Rijkers wrote: > On 2017-03-06 11:27, Petr Jelinek wrote: > >> 0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch + >> 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+ >> 0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch + >>

Re: [HACKERS] Logical replication existing data copy

On 2017-03-06 11:27, Petr Jelinek wrote: 0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch + 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+ 0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch + 0004-Fix-xl_running_xacts-usage-in-snapshot-builder.patch

Re: [HACKERS] Logical replication existing data copy

On Tue, Mar 7, 2017 at 11:39 AM, Peter Eisentraut wrote: > On 3/4/17 01:46, Robert Haas wrote: >>> So I think we should do it, but it needs to be configurable, my original >>> patch added GUC for it, Peter wanted it to be configurable per >>> subscription. I

Re: [HACKERS] Logical replication existing data copy

On 3/4/17 01:46, Robert Haas wrote: >> So I think we should do it, but it needs to be configurable, my original >> patch added GUC for it, Peter wanted it to be configurable per >> subscription. I guess we could add it as another option to the list of >> WITH (...) options for CREATE and ALTER

Re: [HACKERS] Logical replication existing data copy

On 06/03/17 23:40, Erik Rijkers wrote: > On 2017-03-06 16:10, Erik Rijkers wrote: >> On 2017-03-06 11:27, Petr Jelinek wrote: >>> Hi, >>> >>> updated and rebased version of the patch attached. >>> >> >> I compiled with /only/ this one latest patch: >>

Re: [HACKERS] Logical replication existing data copy

On 2017-03-06 16:10, Erik Rijkers wrote: On 2017-03-06 11:27, Petr Jelinek wrote: Hi, updated and rebased version of the patch attached. I compiled with /only/ this one latest patch: 0001-Logical-replication-support-for-initial-data-copy-v6.patch Is that correct, or are other patches

Re: [HACKERS] Logical replication existing data copy

On 2017-03-06 11:27, Petr Jelinek wrote: Hi, updated and rebased version of the patch attached. I compiled with /only/ this one latest patch: 0001-Logical-replication-support-for-initial-data-copy-v6.patch Is that correct, or are other patches still needed on top, or underneath?

Re: [HACKERS] Logical replication existing data copy

On Sat, Mar 4, 2017 at 11:41 AM, Petr Jelinek wrote: > On 04/03/17 06:46, Robert Haas wrote: >> On Tue, Feb 28, 2017 at 12:08 PM, Erik Rijkers wrote: >>> Would you remind me why synchronous_commit = on was deemed a better default? >> I'm wondering

Re: [HACKERS] Logical replication existing data copy

On 04/03/17 06:46, Robert Haas wrote: > On Tue, Feb 28, 2017 at 12:08 PM, Erik Rijkers wrote: >> Would you remind me why synchronous_commit = on was deemed a better default? > > I'm wondering about that, too. If you're trying to do logical > synchronous replication, then maybe

Re: [HACKERS] Logical replication existing data copy

On Tue, Feb 28, 2017 at 12:08 PM, Erik Rijkers wrote: > Would you remind me why synchronous_commit = on was deemed a better default? I'm wondering about that, too. If you're trying to do logical synchronous replication, then maybe there's some argument there, although even in

Re: [HACKERS] Logical replication existing data copy

On 28/02/17 20:36, Erik Rijkers wrote: > This is the most frequent error that happens while doing pgbench-runs > over logical replication: I run it continuously all day, and every few > hours an error occurs of the kind seen below: a table (pgbench_history, > mostly) ends up 1 row short (673466

Re: [HACKERS] Logical replication existing data copy

On 2017-02-28 07:38, Erik Rijkers wrote: On 2017-02-27 15:08, Petr Jelinek wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch + 0002-Fix-after-trigger-execution-in-logical-replication.patch +

Re: [HACKERS] Logical replication existing data copy

On 2017-02-27 15:08, Petr Jelinek wrote: The performance was why in original patch I wanted the apply process to default to synchronous_commit = off as without it the apply performance (due to applying transactions individually and in sequences) is quite lackluster. It can be worked around

Re: [HACKERS] Logical replication existing data copy

On 27/02/17 11:07, Erik Rijkers wrote: > With these patches: > > -- 0416d87c-09a5-182e-4901-236aec103...@2ndquadrant.com >Subject: Re: Logical Replication WIP > 48. > https://www.postgresql.org/message-id/attachment/49886/0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch > >

Re: [HACKERS] Logical replication existing data copy

With these patches: -- 0416d87c-09a5-182e-4901-236aec103...@2ndquadrant.com Subject: Re: Logical Replication WIP 48. https://www.postgresql.org/message-id/attachment/49886/0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch 49.

Re: [HACKERS] Logical replication existing data copy

On 2017-02-26 10:53, Erik Rijkers wrote: Not yet perfect, but we're getting there... Sorry, I made a mistake: I was running the newest patches on master but the older versions on replica (or more precise: I didn't properly shutdown the replica so the older version remained up and running

Re: [HACKERS] Logical replication existing data copy

On 2017-02-26 01:45, Petr Jelinek wrote: Again, much better... : -- out_20170226_0724.txt 25 -- pgbench -c 1 -j 8 -T 10 -P 5 -n 25 -- All is well. -- out_20170226_0751.txt 25 -- pgbench -c 4 -j 8 -T 10 -P 5 -n 25 -- All is well. -- out_20170226_0819.txt 25 -- pgbench -c

Re: [HACKERS] Logical replication existing data copy

On 25/02/17 09:40, Erik Rijkers wrote: > On 2017-02-25 00:40, Petr Jelinek wrote: > >> 0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch >> 0002-Fix-after-trigger-execution-in-logical-replication.patch >> 0003-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION.patch >>

Re: [HACKERS] Logical replication existing data copy

On 2017-02-25 00:40, Petr Jelinek wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch 0002-Fix-after-trigger-execution-in-logical-replication.patch 0003-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION.patch

Re: [HACKERS] Logical replication existing data copy

On 25/02/17 00:39, Erik Rijkers wrote: > On 2017-02-25 00:08, Petr Jelinek wrote: >> >> There is now a lot of fixes for existing code that this patch depends >> on. Hopefully some of the fixes get committed soonish. > > Indeed - could you look over the below list of 8 patches; is it correct > and

Re: [HACKERS] Logical replication existing data copy

On 2017-02-25 00:08, Petr Jelinek wrote: There is now a lot of fixes for existing code that this patch depends on. Hopefully some of the fixes get committed soonish. Indeed - could you look over the below list of 8 patches; is it correct and in the right (apply) order?

Re: [HACKERS] Logical replication existing data copy

On 25/02/17 00:05, Erik Rijkers wrote: > On 2017-02-24 22:58, Petr Jelinek wrote: >> On 23/02/17 01:41, Petr Jelinek wrote: >>> On 23/02/17 01:02, Erik Rijkers wrote: On 2017-02-22 18:13, Erik Rijkers wrote: > On 2017-02-22 14:48, Erik Rijkers wrote: >> On 2017-02-22 13:03, Petr

Re: [HACKERS] Logical replication existing data copy

On 2017-02-24 22:58, Petr Jelinek wrote: On 23/02/17 01:41, Petr Jelinek wrote: On 23/02/17 01:02, Erik Rijkers wrote: On 2017-02-22 18:13, Erik Rijkers wrote: On 2017-02-22 14:48, Erik Rijkers wrote: On 2017-02-22 13:03, Petr Jelinek wrote: 0001-Skip-unnecessary-snapshot-builds.patch

Re: [HACKERS] Logical replication existing data copy

On 23/02/17 01:02, Erik Rijkers wrote: > On 2017-02-22 18:13, Erik Rijkers wrote: >> On 2017-02-22 14:48, Erik Rijkers wrote: >>> On 2017-02-22 13:03, Petr Jelinek wrote: >>> 0001-Skip-unnecessary-snapshot-builds.patch 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch

Re: [HACKERS] Logical replication existing data copy

On 2017-02-22 18:13, Erik Rijkers wrote: On 2017-02-22 14:48, Erik Rijkers wrote: On 2017-02-22 13:03, Petr Jelinek wrote: 0001-Skip-unnecessary-snapshot-builds.patch 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch 0003-Fix-xl_running_xacts-usage-in-snapshot-builder.patch

Re: [HACKERS] Logical replication existing data copy

On 2017-02-22 13:03, Petr Jelinek wrote: 0001-Skip-unnecessary-snapshot-builds.patch 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch 0003-Fix-xl_running_xacts-usage-in-snapshot-builder.patch 0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch

Re: [HACKERS] Logical replication existing data copy - comments snapbuild.c

On 2017-02-19 23:24, Erik Rijkers wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch

Re: [HACKERS] Logical replication existing data copy - comments snapbuild.c

0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch 0004-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION-v2.patch

Re: [HACKERS] Logical replication existing data copy - comments origin.c

On 2017-02-19 17:21, Erik Rijkers wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch

Re: [HACKERS] Logical replication existing data copy - comments origin.c

0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch 0004-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION-v2.patch

Re: [HACKERS] Logical replication existing data copy

On 18/02/17 14:17, Erik Rijkers wrote: > On 2017-02-16 00:43, Petr Jelinek wrote: >> On 13/02/17 14:51, Erik Rijkers wrote: >>> On 2017-02-11 11:16, Erik Rijkers wrote: On 2017-02-08 23:25, Petr Jelinek wrote: > 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch >

Re: [HACKERS] Logical replication existing data copy

On 18/02/17 14:24, Erik Rijkers wrote: >> >> Maybe add this to the 10 Open Items list? >> https://wiki.postgresql.org/wiki/PostgreSQL_10_Open_Items >> >> It may garner a bit more attention. >> > > Ah sorry, it's there already. Hmm that's interesting given that it's not even committed yet :)

Re: [HACKERS] Logical replication existing data copy

On 2017-02-11 11:16, Erik Rijkers wrote: On 2017-02-08 23:25, Petr Jelinek wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch

Re: [HACKERS] Logical replication existing data copy

Maybe add this to the 10 Open Items list? https://wiki.postgresql.org/wiki/PostgreSQL_10_Open_Items It may garner a bit more attention. Ah sorry, it's there already. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Logical replication existing data copy

On 2017-02-16 00:43, Petr Jelinek wrote: On 13/02/17 14:51, Erik Rijkers wrote: On 2017-02-11 11:16, Erik Rijkers wrote: On 2017-02-08 23:25, Petr Jelinek wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch

Re: [HACKERS] Logical replication existing data copy

On 2017-02-16 00:43, Petr Jelinek wrote: On 13/02/17 14:51, Erik Rijkers wrote: On 2017-02-11 11:16, Erik Rijkers wrote: On 2017-02-08 23:25, Petr Jelinek wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch

Re: [HACKERS] Logical replication existing data copy

On 13/02/17 14:51, Erik Rijkers wrote: > On 2017-02-11 11:16, Erik Rijkers wrote: >> On 2017-02-08 23:25, Petr Jelinek wrote: >> >>> 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch >>> 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch >>>

Re: [HACKERS] Logical replication existing data copy

On 2017-02-11 11:16, Erik Rijkers wrote: On 2017-02-08 23:25, Petr Jelinek wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch

Re: [HACKERS] Logical replication existing data copy - sgml fixes

On 2017-02-09 02:25, Erik Rijkers wrote: On 2017-02-08 23:25, Petr Jelinek wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch

Re: [HACKERS] Logical replication existing data copy

On 2017-02-08 23:25, Petr Jelinek wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch

Re: [HACKERS] Logical replication existing data copy

On 2017-02-08 23:25, Petr Jelinek wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch

Re: [HACKERS] Logical replication existing data copy

Hi, here is updated patch. Note that it's rebased on top of logical replication improvements patches [1] (which still apply fine to my surprise). It will probably need another rebase once patches from Masahiko Sawada and Fujii Masao get in. [1]

Re: [HACKERS] Logical replication existing data copy

On Fri, Jan 20, 2017 at 3:03 AM, Petr Jelinek wrote: > Okay, here is v3 with some small fixes and rebased on top of rename. > Also it's rebased without the > 0005-Add-separate-synchronous-commit-control-for-logical--v18.patch as I > don't expect that one to go

Re: [HACKERS] Logical replication existing data copy

On 2017-01-18 20:45, Petr Jelinek wrote: AFAICS you should always get error from that test after you enable the Ah tes, you were right, of course; I had assumed the earlier mentioned CREATE SUBSCRIPTION ... [ WITH (COPY DATA | NOCOPY DATA) ] but that syntax wasn't implemented, I now

Re: [HACKERS] Logical replication existing data copy

On 18/01/17 17:35, Erik Rijkers wrote: > On 2017-01-18 14:46, Petr Jelinek wrote: > >> 0001-Logical-replication-support-for-initial-data-copy-v2.patch > > Applies and builds fine on top of the previous 5 patches. > > Two problems: > > 1. alter_subscription.sgml has an unpaired -tag, which

Re: [HACKERS] Logical replication existing data copy

On 2017-01-18 14:46, Petr Jelinek wrote: 0001-Logical-replication-support-for-initial-data-copy-v2.patch Applies and builds fine on top of the previous 5 patches. Two problems: 1. alter_subscription.sgml has an unpaired -tag, which breaks the doc-build: This is the offending patch-line:

Re: [HACKERS] Logical replication existing data copy

On 12/19/16 4:30 AM, Petr Jelinek wrote: > This patch implements data synchronization for the logical replication. > It works both for initial setup of subscription as well as for tables > added later to replication when the subscription is already active. First detailed read-through. General

Re: [HACKERS] Logical replication existing data copy

On 19/12/16 18:25, Peter Eisentraut wrote: > On 12/19/16 4:30 AM, Petr Jelinek wrote: >> So existing table data can be copied once subscription is created, but >> also new tables can be added and their data copied. This is where the >> REFRESH PUBLICATION comes into play. Adding table to

Re: [HACKERS] Logical replication existing data copy

On 20 Dec. 2016 1:27 am, "Peter Eisentraut" < peter.eisentr...@2ndquadrant.com> wrote: On 12/19/16 4:30 AM, Petr Jelinek wrote: > So existing table data can be copied once subscription is created, but > also new tables can be added and their data copied. This is where the > REFRESH PUBLICATION

Re: [HACKERS] Logical replication existing data copy

On 12/19/16 4:30 AM, Petr Jelinek wrote: > So existing table data can be copied once subscription is created, but > also new tables can be added and their data copied. This is where the > REFRESH PUBLICATION comes into play. Adding table to publication does > not make it automatically replicated