Re: [HACKERS] snapbuild woes

On 2017-06-09 09:25:34 +0200, Antonin Houska wrote: > Andres Freund wrote: > > > Looking at 0001: > > - GetOldestSafeDecodingTransactionId() only guarantees to return an xid > > safe for decoding (note how procArray->replication_slot_catalog_xmin > > is checked), not one

Re: [HACKERS] snapbuild woes

Andres Freund wrote: > Looking at 0001: > - GetOldestSafeDecodingTransactionId() only guarantees to return an xid > safe for decoding (note how procArray->replication_slot_catalog_xmin > is checked), not one for the initial snapshot - so afaics this whole > exercise

Re: [HACKERS] snapbuild woes

On 2017-05-14 14:54:37 +0200, Petr Jelinek wrote: > On 13/05/17 22:23, Andres Freund wrote: > >> And wait for session 3 to finish slot creation, takes about 20 mins on > >> my laptop without patches, minute and half with your patches for 0004 > >> and 0005 (or with your 0004 and my 0005) and about

Re: [HACKERS] snapbuild woes

On 13/05/17 22:23, Andres Freund wrote: > On 2017-05-12 10:57:55 +0200, Petr Jelinek wrote: >> Hmm, well it helps but actually now that we don't track individual >> running transactions anymore it got much less effective (my version of >> 0005 does as well). >> >> The example workload I test with

Re: [HACKERS] snapbuild woes

On 2017-05-12 10:57:55 +0200, Petr Jelinek wrote: > Hmm, well it helps but actually now that we don't track individual > running transactions anymore it got much less effective (my version of > 0005 does as well). > > The example workload I test with is: > session 1: open transaction, do a write,

Re: [HACKERS] snapbuild woes

On 12/05/17 10:57, Petr Jelinek wrote: > On 12/05/17 03:31, Andres Freund wrote: >> On 2017-05-11 14:54:26 -0700, Andres Freund wrote: >>> On 2017-05-11 14:51:55 -0700, wrote: On 2017-05-08 00:10:12 -0700, Andres Freund wrote: > I plan to commit the next pending patch after the back

Re: [HACKERS] snapbuild woes

On 12/05/17 03:31, Andres Freund wrote: > On 2017-05-11 14:54:26 -0700, Andres Freund wrote: >> On 2017-05-11 14:51:55 -0700, wrote: >>> On 2017-05-08 00:10:12 -0700, Andres Freund wrote: I plan to commit the next pending patch after the back branch releases are cut - it's an invasive

Re: [HACKERS] snapbuild woes

On 2017-05-11 14:54:26 -0700, Andres Freund wrote: > On 2017-05-11 14:51:55 -0700, wrote: > > On 2017-05-08 00:10:12 -0700, Andres Freund wrote: > > > I plan to commit the next pending patch after the back branch releases > > > are cut - it's an invasive fix and the issue doesn't cause corruption

Re: [HACKERS] snapbuild woes

On 2017-04-15 05:18:49 +0200, Petr Jelinek wrote: > >>> From 3318a929e691870f3c1ca665bec3bfa8ea2af2a8 Mon Sep 17 00:00:00 2001 > >>> From: Petr Jelinek > >>> Date: Sun, 26 Feb 2017 01:07:33 +0100 > >>> Subject: [PATCH 3/5] Prevent snapshot builder xmin from going backwards >

Re: [HACKERS] snapbuild woes

On 2017-05-11 14:51:55 -0700, wrote: > On 2017-05-08 00:10:12 -0700, Andres Freund wrote: > > I plan to commit the next pending patch after the back branch releases > > are cut - it's an invasive fix and the issue doesn't cause corruption > > "just" slow slot creation. So it seems better to wait

Re: [HACKERS] snapbuild woes

On Thu, May 11, 2017 at 2:51 PM, Andres Freund wrote: > Now that that's done, here's an updated version of that patch. Note the > new logic to trigger xl_running_xact's to be logged at the right spot. > Works well in my testing. You forgot the patch. :-) -- Peter

Re: [HACKERS] snapbuild woes

On 2017-05-08 00:10:12 -0700, Andres Freund wrote: > I plan to commit the next pending patch after the back branch releases > are cut - it's an invasive fix and the issue doesn't cause corruption > "just" slow slot creation. So it seems better to wait for a few days, > rather than hurry it into

Re: [HACKERS] snapbuild woes

On 11/05/17 16:33, Stas Kelvich wrote: > >> On 10 May 2017, at 11:43, Petr Jelinek wrote: >> >> On 09/05/17 22:11, Erik Rijkers wrote: >>> On 2017-05-09 21:00, Petr Jelinek wrote: On 09/05/17 19:54, Erik Rijkers wrote: > On 2017-05-09 11:50, Petr Jelinek

Re: [HACKERS] snapbuild woes

> On 10 May 2017, at 11:43, Petr Jelinek wrote: > > On 09/05/17 22:11, Erik Rijkers wrote: >> On 2017-05-09 21:00, Petr Jelinek wrote: >>> On 09/05/17 19:54, Erik Rijkers wrote: On 2017-05-09 11:50, Petr Jelinek wrote: >>> >>> Ah okay, so this is same

Re: [HACKERS] snapbuild woes

On 09/05/17 22:11, Erik Rijkers wrote: > On 2017-05-09 21:00, Petr Jelinek wrote: >> On 09/05/17 19:54, Erik Rijkers wrote: >>> On 2017-05-09 11:50, Petr Jelinek wrote: >>> >> >> Ah okay, so this is same issue that's reported by both Masahiko Sawada >> [1] and Jeff Janes [2]. >> >> [1] >>

Re: [HACKERS] snapbuild woes

On 2017-05-09 21:00, Petr Jelinek wrote: On 09/05/17 19:54, Erik Rijkers wrote: On 2017-05-09 11:50, Petr Jelinek wrote: Ah okay, so this is same issue that's reported by both Masahiko Sawada [1] and Jeff Janes [2]. [1]

Re: [HACKERS] snapbuild woes

On 09/05/17 19:54, Erik Rijkers wrote: > On 2017-05-09 11:50, Petr Jelinek wrote: > >> I rebased the above mentioned patch to apply to the patches Andres sent, >> if you could try to add it on top of what you have and check if it still >> fails, that would be helpful. > > It still fails. > >

Re: [HACKERS] snapbuild woes

On 2017-05-09 11:50, Petr Jelinek wrote: I rebased the above mentioned patch to apply to the patches Andres sent, if you could try to add it on top of what you have and check if it still fails, that would be helpful. It still fails. With these patches -

Re: [HACKERS] snapbuild woes

On 2017-05-09 11:50, Petr Jelinek wrote: On 09/05/17 10:59, Erik Rijkers wrote: On 2017-05-09 10:50, Petr Jelinek wrote: On 09/05/17 00:03, Erik Rijkers wrote: On 2017-05-05 02:00, Andres Freund wrote: Could you have a look? [...] I rebased the above mentioned patch to apply to the

Re: [HACKERS] snapbuild woes

On 09/05/17 10:59, Erik Rijkers wrote: > On 2017-05-09 10:50, Petr Jelinek wrote: >> On 09/05/17 00:03, Erik Rijkers wrote: >>> On 2017-05-05 02:00, Andres Freund wrote: Could you have a look? >>> >>> Running tests with these three patches: >>>

Re: [HACKERS] snapbuild woes

On 2017-05-09 10:50, Petr Jelinek wrote: On 09/05/17 00:03, Erik Rijkers wrote: On 2017-05-05 02:00, Andres Freund wrote: Could you have a look? Running tests with these three patches: 0001-WIP-Fix-off-by-one-around-GetLastImportantRecPtr.patch+

Re: [HACKERS] snapbuild woes

On 09/05/17 00:03, Erik Rijkers wrote: > On 2017-05-05 02:00, Andres Freund wrote: >> >> Could you have a look? > > Running tests with these three patches: > >> 0001-WIP-Fix-off-by-one-around-GetLastImportantRecPtr.patch+ >> 0002-WIP-Possibly-more-robust-snapbuild-approach.patch + >>

Re: [HACKERS] snapbuild woes

On 2017-05-05 02:00, Andres Freund wrote: Could you have a look? Running tests with these three patches: 0001-WIP-Fix-off-by-one-around-GetLastImportantRecPtr.patch+ 0002-WIP-Possibly-more-robust-snapbuild-approach.patch + fix-statistics-reporting-in-logical-replication-work.patch

Re: [HACKERS] snapbuild woes

On May 3, 2017 10:45:16 PM PDT, Noah Misch wrote: >On Thu, Apr 27, 2017 at 09:42:58PM -0700, Andres Freund wrote: >> >> >> On April 27, 2017 9:34:44 PM PDT, Noah Misch >wrote: >> >On Fri, Apr 21, 2017 at 10:36:21PM -0700, Andres Freund wrote: >> >> On

Re: [HACKERS] snapbuild woes

On 05/05/17 18:18, Andres Freund wrote: > On 2017-05-05 13:53:16 +0200, Petr Jelinek wrote: >> On 05/05/17 02:42, Andres Freund wrote: >>> On 2017-05-04 17:00:04 -0700, Andres Freund wrote: Attached is a prototype patch for that. >>> >> >> I am not sure I understand the ABI comment for

Re: [HACKERS] snapbuild woes

On 2017-05-05 13:53:16 +0200, Petr Jelinek wrote: > On 05/05/17 02:42, Andres Freund wrote: > > On 2017-05-04 17:00:04 -0700, Andres Freund wrote: > >> Attached is a prototype patch for that. > > > > I am not sure I understand the ABI comment for started_collection_at. > What's the ABI issue?

Re: [HACKERS] snapbuild woes

On 05/05/17 02:42, Andres Freund wrote: > On 2017-05-04 17:00:04 -0700, Andres Freund wrote: >> Attached is a prototype patch for that. > I am not sure I understand the ABI comment for started_collection_at. What's the ABI issue? The struct is private to snapbuild.c module. Or you want to store

Re: [HACKERS] snapbuild woes

On 05/05/17 02:00, Andres Freund wrote: > Hi, > > On 2017-05-02 08:55:53 +0200, Petr Jelinek wrote: >> Aah, now I understand we talked about slightly different things, I >> considered the running thing to be first step towards tracking aborted >> txes everywhere. >> I think >> we'll have to

Re: [HACKERS] snapbuild woes

On 2017-05-04 17:00:04 -0700, Andres Freund wrote: > Attached is a prototype patch for that. Oops. Andres >From b6eb46e376e40f3e2e9a55d16b1b37b27904564b Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Thu, 4 May 2017 16:40:52 -0700 Subject: [PATCH 1/2] WIP: Fix off-by-one

Re: [HACKERS] snapbuild woes

Hi, On 2017-05-02 08:55:53 +0200, Petr Jelinek wrote: > Aah, now I understand we talked about slightly different things, I > considered the running thing to be first step towards tracking aborted > txes everywhere. > I think > we'll have to revisit tracking of aborted transactions in PG11 then >

Re: [HACKERS] snapbuild woes

On 04/05/17 07:45, Noah Misch wrote: > On Thu, Apr 27, 2017 at 09:42:58PM -0700, Andres Freund wrote: >> >> >> On April 27, 2017 9:34:44 PM PDT, Noah Misch wrote: >>> On Fri, Apr 21, 2017 at 10:36:21PM -0700, Andres Freund wrote: On 2017-04-17 21:16:57 -0700, Andres Freund

Re: [HACKERS] snapbuild woes

On Thu, Apr 27, 2017 at 09:42:58PM -0700, Andres Freund wrote: > > > On April 27, 2017 9:34:44 PM PDT, Noah Misch wrote: > >On Fri, Apr 21, 2017 at 10:36:21PM -0700, Andres Freund wrote: > >> On 2017-04-17 21:16:57 -0700, Andres Freund wrote: > >> > I've since the previous

Re: [HACKERS] snapbuild woes

On 01/05/17 21:14, Andres Freund wrote: > On 2017-05-01 11:09:44 +0200, Petr Jelinek wrote: >> On 01/05/17 10:03, Andres Freund wrote: >>> On 2017-05-01 03:54:49 +0200, Petr Jelinek wrote: > But, I still think we need to restart the tracking after new xl_running_xacts. Reason for that

Re: [HACKERS] snapbuild woes

On Mon, May 01, 2017 at 12:32:07PM -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-05-01 08:46:47 -0400, Tom Lane wrote: > >> 30sec is kind of a big lump from a buildfarm standpoint, especially if > >> you mean "it runs for 30s on my honkin' fast workstation". I'm

Re: [HACKERS] snapbuild woes

On 5/1/17 13:02, Tom Lane wrote: > Andres Freund writes: >> On 2017-05-01 12:32:07 -0400, Tom Lane wrote: >>> But quite aside from the question of whether we can afford the cycles, >>> it seems like the wrong approach. IMO the buildfarm is mainly for >>> verifying

Re: [HACKERS] snapbuild woes

On 2017-05-01 11:09:44 +0200, Petr Jelinek wrote: > On 01/05/17 10:03, Andres Freund wrote: > > On 2017-05-01 03:54:49 +0200, Petr Jelinek wrote: > >> But, I still think we need to restart the tracking after new > >> xl_running_xacts. Reason for that is afaics any of the catalog snapshots > >>

Re: [HACKERS] snapbuild woes

Andres Freund writes: > On 2017-05-01 12:32:07 -0400, Tom Lane wrote: >> But quite aside from the question of whether we can afford the cycles, >> it seems like the wrong approach. IMO the buildfarm is mainly for >> verifying portability, not for trying to prove that

Re: [HACKERS] snapbuild woes

On 2017-05-01 12:32:07 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-05-01 08:46:47 -0400, Tom Lane wrote: > >> 30sec is kind of a big lump from a buildfarm standpoint, especially if > >> you mean "it runs for 30s on my honkin' fast workstation". I'm fine > >>

Re: [HACKERS] snapbuild woes

Andres Freund writes: > On 2017-05-01 08:46:47 -0400, Tom Lane wrote: >> 30sec is kind of a big lump from a buildfarm standpoint, especially if >> you mean "it runs for 30s on my honkin' fast workstation". I'm fine >> with individual tests that run for ~ 1sec. > I was more

Re: [HACKERS] snapbuild woes

On 2017-05-01 08:46:47 -0400, Tom Lane wrote: > Noah Misch writes: > > On Fri, Apr 21, 2017 at 10:34:58PM -0700, Andres Freund wrote: > >> ... I was wondering about adding > >> a loop that simply runs for like 30s and then quits or such, but who > >> knows. > > > If the

Re: [HACKERS] snapbuild woes

On 05/01/2017 08:46 AM, Tom Lane wrote: > Noah Misch writes: >> On Fri, Apr 21, 2017 at 10:34:58PM -0700, Andres Freund wrote: >>> ... I was wondering about adding >>> a loop that simply runs for like 30s and then quits or such, but who >>> knows. >> If the probabilistic test

Re: [HACKERS] snapbuild woes

Noah Misch writes: > On Fri, Apr 21, 2017 at 10:34:58PM -0700, Andres Freund wrote: >> ... I was wondering about adding >> a loop that simply runs for like 30s and then quits or such, but who >> knows. > If the probabilistic test catches the bug even 5% of the time in typical

Re: [HACKERS] snapbuild woes

On 01/05/17 10:03, Andres Freund wrote: > On 2017-05-01 03:54:49 +0200, Petr Jelinek wrote: >> I agree with adding running, I think that's good thing even for the per >> transaction tracking and snapshot exports - we could use the newly added >> field to get rid of the issue we have with 'snapshot

Re: [HACKERS] snapbuild woes

On 2017-05-01 03:54:49 +0200, Petr Jelinek wrote: > I agree with adding running, I think that's good thing even for the per > transaction tracking and snapshot exports - we could use the newly added > field to get rid of the issue we have with 'snapshot too large' when > there were many aborted

Re: [HACKERS] snapbuild woes

On 01/05/17 04:29, Craig Ringer wrote: > On 1 May 2017 at 09:54, Petr Jelinek wrote: > >> But, I still think we need to restart the tracking after new >> xl_running_xacts. Reason for that is afaics any of the catalog snapshots >> that we assigned to transactions at

Re: [HACKERS] snapbuild woes

On 1 May 2017 at 09:54, Petr Jelinek wrote: > But, I still think we need to restart the tracking after new > xl_running_xacts. Reason for that is afaics any of the catalog snapshots > that we assigned to transactions at the end of SnapBuildCommitTxn might > be

Re: [HACKERS] snapbuild woes

On 01/05/17 03:35, Andres Freund wrote: > On 2017-04-30 17:59:21 -0700, Andres Freund wrote: >> ISTM, we need a xip_status array in SnapBuild->running. Then, whenever >> a xl_running_xacts is encountered while builder->running.xcnt > 0, loop >> for i in 0 .. xcnt_space, and end SnapBuildEndTxn()

Re: [HACKERS] snapbuild woes

On 2017-04-30 17:59:21 -0700, Andres Freund wrote: > ISTM, we need a xip_status array in SnapBuild->running. Then, whenever > a xl_running_xacts is encountered while builder->running.xcnt > 0, loop > for i in 0 .. xcnt_space, and end SnapBuildEndTxn() if xip_status[i] but > the xid is not

Re: [HACKERS] snapbuild woes

Hi, On 2017-04-15 05:18:49 +0200, Petr Jelinek wrote: > + /* > + * c) we already seen the xl_running_xacts and tried to do the above. > + * However because of race condition in LogStandbySnapshot() there might > + * have been transaction reported as running but in reality has

Re: [HACKERS] snapbuild woes

On Fri, Apr 21, 2017 at 10:34:58PM -0700, Andres Freund wrote: > I've a bunch of tests, but I don't quite know whether we can expose all > of them via classical tests. There are several easy ones that I > definitely want to add (import "empty" snapshot; import snapshot with > running xacts;

Re: [HACKERS] snapbuild woes

On April 27, 2017 9:34:44 PM PDT, Noah Misch wrote: >On Fri, Apr 21, 2017 at 10:36:21PM -0700, Andres Freund wrote: >> On 2017-04-17 21:16:57 -0700, Andres Freund wrote: >> > I've since the previous update reviewed Petr's patch, which he >since has >> > updated over the

Re: [HACKERS] snapbuild woes

On Fri, Apr 21, 2017 at 10:36:21PM -0700, Andres Freund wrote: > On 2017-04-17 21:16:57 -0700, Andres Freund wrote: > > I've since the previous update reviewed Petr's patch, which he since has > > updated over the weekend. I'll do another round tomorrow, and will see > > how it looks. I think we

Re: [HACKERS] snapbuild woes

On 25/04/17 00:59, Andres Freund wrote: > Hi, > > On 2017-04-15 05:18:49 +0200, Petr Jelinek wrote: >> Hi, here is updated patch (details inline). > > I'm not yet all that happy, sorry: > > Looking at 0001: > - GetOldestSafeDecodingTransactionId() only guarantees to return an xid > safe for

Re: [HACKERS] snapbuild woes

Hi, On 2017-04-15 05:18:49 +0200, Petr Jelinek wrote: > Hi, here is updated patch (details inline). I'm not yet all that happy, sorry: Looking at 0001: - GetOldestSafeDecodingTransactionId() only guarantees to return an xid safe for decoding (note how procArray->replication_slot_catalog_xmin

Re: [HACKERS] snapbuild woes

On 2017-04-17 21:16:57 -0700, Andres Freund wrote: > I've since the previous update reviewed Petr's patch, which he since has > updated over the weekend. I'll do another round tomorrow, and will see > how it looks. I think we might need some more tests for this to be > committable, so it might

Re: [HACKERS] snapbuild woes

On 2017-04-20 13:32:10 +0200, Petr Jelinek wrote: > On 20/04/17 02:09, Andres Freund wrote: > > On 2017-04-17 21:16:57 -0700, Andres Freund wrote: > > I'm working on some infrastructure around this. Not sure if it needs to > > be committed, but it's certainly useful for evaluation. Basically

Re: [HACKERS] snapbuild woes

On 20/04/17 02:09, Andres Freund wrote: > On 2017-04-17 21:16:57 -0700, Andres Freund wrote: >> I think we might need some more tests for this to be committable, so >> it might not become committable tomorrow. > > I'm working on some infrastructure around this. Not sure if it needs to > be

Re: [HACKERS] snapbuild woes

On 2017-04-17 21:16:57 -0700, Andres Freund wrote: > I think we might need some more tests for this to be committable, so > it might not become committable tomorrow. I'm working on some infrastructure around this. Not sure if it needs to be committed, but it's certainly useful for evaluation.

Re: [HACKERS] snapbuild woes

On 2017-04-16 22:04:04 -0400, Noah Misch wrote: > On Thu, Apr 13, 2017 at 12:58:12AM -0400, Noah Misch wrote: > > On Wed, Apr 12, 2017 at 10:21:51AM -0700, Andres Freund wrote: > > > On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: > > > > On 4/12/17 02:31, Noah Misch wrote: > > > > >>> But I

Re: [HACKERS] snapbuild woes

On Thu, Apr 13, 2017 at 12:58:12AM -0400, Noah Misch wrote: > On Wed, Apr 12, 2017 at 10:21:51AM -0700, Andres Freund wrote: > > On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: > > > On 4/12/17 02:31, Noah Misch wrote: > > > >>> But I hope you mean to commit these snapbuild patches before

Re: [HACKERS] snapbuild woes

Hi, here is updated patch (details inline). On 13/04/17 20:00, Petr Jelinek wrote: > Thanks for looking at this! > > On 13/04/17 02:29, Andres Freund wrote: >> Hi, >> On 2017-03-03 01:30:11 +0100, Petr Jelinek wrote: >> >>> From 7d5b48c8cb80e7c867b2096c999d08feda50b197 Mon Sep 17 00:00:00 2001

Re: [HACKERS] snapbuild woes

On 13/04/17 07:02, Andres Freund wrote: > > On April 12, 2017 9:58:12 PM PDT, Noah Misch wrote: >> On Wed, Apr 12, 2017 at 10:21:51AM -0700, Andres Freund wrote: >>> On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: On 4/12/17 02:31, Noah Misch wrote: >>> But I

Re: [HACKERS] snapbuild woes

Thanks for looking at this! On 13/04/17 02:29, Andres Freund wrote: > Hi, > On 2017-03-03 01:30:11 +0100, Petr Jelinek wrote: > >> From 7d5b48c8cb80e7c867b2096c999d08feda50b197 Mon Sep 17 00:00:00 2001 >> From: Petr Jelinek >> Date: Fri, 24 Feb 2017 21:39:03 +0100 >>

Re: [HACKERS] snapbuild woes

On April 12, 2017 9:58:12 PM PDT, Noah Misch wrote: >On Wed, Apr 12, 2017 at 10:21:51AM -0700, Andres Freund wrote: >> On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: >> > On 4/12/17 02:31, Noah Misch wrote: >> > >>> But I hope you mean to commit these snapbuild patches

Re: [HACKERS] snapbuild woes

On Wed, Apr 12, 2017 at 10:21:51AM -0700, Andres Freund wrote: > On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: > > On 4/12/17 02:31, Noah Misch wrote: > > >>> But I hope you mean to commit these snapbuild patches before the > > >>> postgres 10 > > >>> release? As far as I know, logical

Re: [HACKERS] snapbuild woes

Hi, On 2017-03-03 01:30:11 +0100, Petr Jelinek wrote: > From 7d5b48c8cb80e7c867b2096c999d08feda50b197 Mon Sep 17 00:00:00 2001 > From: Petr Jelinek > Date: Fri, 24 Feb 2017 21:39:03 +0100 > Subject: [PATCH 1/5] Reserve global xmin for create slot snasphot export > >

Re: [HACKERS] snapbuild woes

On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: > On 4/12/17 02:31, Noah Misch wrote: > >>> But I hope you mean to commit these snapbuild patches before the postgres > >>> 10 > >>> release? As far as I know, logical replication is still very broken > >>> without > >>> them (or at least

Re: [HACKERS] snapbuild woes

On 3 March 2017 at 00:30, Petr Jelinek wrote: >> 0004 - Changes handling of the xl_running_xacts in initial snapshot >> build to what I wrote above and removes the extra locking from >> LogStandbySnapshot introduced by logical decoding. This seems OK and unlikely

Re: [HACKERS] snapbuild woes

On 4/12/17 02:31, Noah Misch wrote: >>> But I hope you mean to commit these snapbuild patches before the postgres 10 >>> release? As far as I know, logical replication is still very broken without >>> them (or at least some of that set of 5 patches - I don't know which ones >>> are essential and

Re: [HACKERS] snapbuild woes

On Sat, Apr 08, 2017 at 07:30:59AM -0700, Andres Freund wrote: > On 2017-04-08 16:29:10 +0200, Erik Rijkers wrote: > > On 2017-04-08 15:56, Andres Freund wrote: > > > On 2017-04-08 09:51:39 -0400, David Steele wrote: > > > > On 3/2/17 7:54 PM, Petr Jelinek wrote: > > > > > > > > > > Yes the copy

Re: [HACKERS] snapbuild woes

On 4/8/17 10:29 AM, Erik Rijkers wrote: > On 2017-04-08 15:56, Andres Freund wrote: >> On 2017-04-08 09:51:39 -0400, David Steele wrote: >>> On 3/2/17 7:54 PM, Petr Jelinek wrote: >>> > >>> > Yes the copy patch needs rebase as well. But these ones are fine. >>> >>> This bug has been moved to CF

Re: [HACKERS] snapbuild woes

On 2017-04-08 16:29:10 +0200, Erik Rijkers wrote: > On 2017-04-08 15:56, Andres Freund wrote: > > On 2017-04-08 09:51:39 -0400, David Steele wrote: > > > On 3/2/17 7:54 PM, Petr Jelinek wrote: > > > > > > > > Yes the copy patch needs rebase as well. But these ones are fine. > > > > > > This bug

Re: [HACKERS] snapbuild woes

On 2017-04-08 15:56, Andres Freund wrote: On 2017-04-08 09:51:39 -0400, David Steele wrote: On 3/2/17 7:54 PM, Petr Jelinek wrote: > > Yes the copy patch needs rebase as well. But these ones are fine. This bug has been moved to CF 2017-07. FWIW, as these are bug-fixes that need to be

Re: [HACKERS] snapbuild woes

On 2017-04-08 09:51:39 -0400, David Steele wrote: > On 3/2/17 7:54 PM, Petr Jelinek wrote: > > > > Yes the copy patch needs rebase as well. But these ones are fine. > > This bug has been moved to CF 2017-07. FWIW, as these are bug-fixes that need to be backpatched, I do plan to work on them

Re: [HACKERS] snapbuild woes

On 3/2/17 7:54 PM, Petr Jelinek wrote: > > Yes the copy patch needs rebase as well. But these ones are fine. This bug has been moved to CF 2017-07. -- -David da...@pgmasters.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] snapbuild woes

On 03/03/17 01:53, Erik Rijkers wrote: > On 2017-03-03 01:30, Petr Jelinek wrote: > > With these patches: > > 0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch > 0002-Fix-after-trigger-execution-in-logical-replication.patch >

Re: [HACKERS] snapbuild woes

On 2017-03-03 01:30, Petr Jelinek wrote: With these patches: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch 0002-Fix-after-trigger-execution-in-logical-replication.patch 0003-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION.patch

Re: [HACKERS] snapbuild woes

On 26/02/17 01:43, Petr Jelinek wrote: > On 24/02/17 22:56, Petr Jelinek wrote: >> On 22/02/17 03:05, Petr Jelinek wrote: >>> On 13/12/16 00:38, Petr Jelinek wrote: On 12/12/16 23:33, Andres Freund wrote: > On 2016-12-12 23:27:30 +0100, Petr Jelinek wrote: >> On 12/12/16 22:42, Andres

Re: [HACKERS] snapbuild woes

On 24/02/17 22:56, Petr Jelinek wrote: > On 22/02/17 03:05, Petr Jelinek wrote: >> On 13/12/16 00:38, Petr Jelinek wrote: >>> On 12/12/16 23:33, Andres Freund wrote: On 2016-12-12 23:27:30 +0100, Petr Jelinek wrote: > On 12/12/16 22:42, Andres Freund wrote: >> Hi, >> >> On

Re: [HACKERS] snapbuild woes

On 22/02/17 03:05, Petr Jelinek wrote: > On 13/12/16 00:38, Petr Jelinek wrote: >> On 12/12/16 23:33, Andres Freund wrote: >>> On 2016-12-12 23:27:30 +0100, Petr Jelinek wrote: On 12/12/16 22:42, Andres Freund wrote: > Hi, > > On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote:

Re: [HACKERS] snapbuild woes

On 22/02/17 11:29, Erik Rijkers wrote: > On 2017-02-22 03:05, Petr Jelinek wrote: >> >> So to summarize attached patches: >> 0001 - Fixes performance issue where we build tons of snapshots that we >> don't need which kills CPU. >> >> 0002 - Disables the use of ondisk historical snapshots for

Re: [HACKERS] snapbuild woes

On 2017-02-22 03:05, Petr Jelinek wrote: So to summarize attached patches: 0001 - Fixes performance issue where we build tons of snapshots that we don't need which kills CPU. 0002 - Disables the use of ondisk historical snapshots for initial consistent snapshot export as it may result in

Re: [HACKERS] snapbuild woes

On 13/12/16 00:38, Petr Jelinek wrote: > On 12/12/16 23:33, Andres Freund wrote: >> On 2016-12-12 23:27:30 +0100, Petr Jelinek wrote: >>> On 12/12/16 22:42, Andres Freund wrote: Hi, On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: > Hi, > First one is outright bug, which

Re: [HACKERS] snapbuild woes

On 12/12/16 23:33, Andres Freund wrote: > On 2016-12-12 23:27:30 +0100, Petr Jelinek wrote: >> On 12/12/16 22:42, Andres Freund wrote: >>> Hi, >>> >>> On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: Hi, First one is outright bug, which has to do with how we track running

Re: [HACKERS] snapbuild woes

On 2016-12-12 23:27:30 +0100, Petr Jelinek wrote: > On 12/12/16 22:42, Andres Freund wrote: > > Hi, > > > > On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: > >> Hi, > >> First one is outright bug, which has to do with how we track running > >> transactions. What snapbuild basically does while

Re: [HACKERS] snapbuild woes

On 12/12/16 22:42, Andres Freund wrote: > Hi, > > On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: >> Hi, >> First one is outright bug, which has to do with how we track running >> transactions. What snapbuild basically does while doing initial snapshot >> is read the xl_running_xacts record,

Re: [HACKERS] snapbuild woes

Hi, On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: > Hi, > First one is outright bug, which has to do with how we track running > transactions. What snapbuild basically does while doing initial snapshot > is read the xl_running_xacts record, store the list of running txes and > then wait until

Re: [HACKERS] snapbuild woes

On 12 December 2016 at 00:36, Kevin Grittner wrote: > On Sun, Dec 11, 2016 at 1:17 AM, Craig Ringer > wrote: >> On 11 Dec. 2016 06:50, "Petr Jelinek" wrote: >>> On 10/12/16 23:10, Petr Jelinek wrote: >>> The

Re: [HACKERS] snapbuild woes

On Sun, Dec 11, 2016 at 1:17 AM, Craig Ringer wrote: > On 11 Dec. 2016 06:50, "Petr Jelinek" wrote: >> On 10/12/16 23:10, Petr Jelinek wrote: >> >>> The attached 0002-Skip-unnecessary-snapshot-builds.patch changes this >>> behavior so

Re: [HACKERS] snapbuild woes

On 11 Dec. 2016 06:50, "Petr Jelinek" wrote: On 10/12/16 23:10, Petr Jelinek wrote: > > The attached 0002-Skip-unnecessary-snapshot-builds.patch changes this > behavior so that we don't make snapshots for transactions that we seen > wholly and know that they didn't

Re: [HACKERS] snapbuild woes

On 10/12/16 23:10, Petr Jelinek wrote: > > The attached 0002-Skip-unnecessary-snapshot-builds.patch changes this > behavior so that we don't make snapshots for transactions that we seen > wholly and know that they didn't make catalog changes even if we didn't > reach consistency yet. > Eh,

[HACKERS] snapbuild woes

Hi, I recently found couple of issues with the way initial logical decoding snapshot is made. First one is outright bug, which has to do with how we track running transactions. What snapbuild basically does while doing initial snapshot is read the xl_running_xacts record, store the list of