Re: Possible bug in logical replication.

2018-07-19 Thread Alvaro Herrera
On 2018-Jul-19, Michael Paquier wrote: > On Thu, Jul 19, 2018 at 12:38:53AM -0400, Alvaro Herrera wrote: > > On 2018-Jul-19, Michael Paquier wrote: > >> I am fine either way if you want to have the last call. So please feel > >> free to choose what you prefer here. That's no big deal. > > > > O

Re: Possible bug in logical replication.

2018-07-18 Thread Erik Rijkers
On 2018-07-19 03:42, Michael Paquier wrote: On Wed, Jul 18, 2018 at 02:30:53PM -0400, Alvaro Herrera wrote: In the immortal words of Julian Bream: "yeah, I didn't like any of that". One wikipedia lookup later, I still don't know where this quote comes from, but at least I understand who the ma

Re: Possible bug in logical replication.

2018-07-18 Thread Michael Paquier
On Thu, Jul 19, 2018 at 12:38:53AM -0400, Alvaro Herrera wrote: > On 2018-Jul-19, Michael Paquier wrote: >> I am fine either way if you want to have the last call. So please feel >> free to choose what you prefer here. That's no big deal. > > Okay. You want to push it, or shall I? It seems to

Re: Possible bug in logical replication.

2018-07-18 Thread Alvaro Herrera
On 2018-Jul-19, Michael Paquier wrote: > I am fine either way if you want to have the last call. So please feel > free to choose what you prefer here. That's no big deal. Okay. You want to push it, or shall I? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Developme

Re: Possible bug in logical replication.

2018-07-18 Thread Michael Paquier
On Wed, Jul 18, 2018 at 10:45:27PM -0400, Alvaro Herrera wrote: > On 2018-Jul-19, Michael Paquier wrote: > >> On Wed, Jul 18, 2018 at 02:30:53PM -0400, Alvaro Herrera wrote: >> > In the immortal words of Julian Bream: "yeah, I didn't like any of >> > that". >> >> One wikipedia lookup later, I sti

Re: Possible bug in logical replication.

2018-07-18 Thread Alvaro Herrera
On 2018-Jul-19, Michael Paquier wrote: > On Wed, Jul 18, 2018 at 02:30:53PM -0400, Alvaro Herrera wrote: > > In the immortal words of Julian Bream: "yeah, I didn't like any of > > that". > > One wikipedia lookup later, I still don't know where this quote comes > from, but at least I understand wh

Re: Possible bug in logical replication.

2018-07-18 Thread Michael Paquier
On Wed, Jul 18, 2018 at 02:30:53PM -0400, Alvaro Herrera wrote: > In the immortal words of Julian Bream: "yeah, I didn't like any of > that". One wikipedia lookup later, I still don't know where this quote comes from, but at least I understand who the man is. I may be missing something, but I can

Re: Possible bug in logical replication.

2018-07-18 Thread Alvaro Herrera
On 2018-Jul-12, Michael Paquier wrote: > On Wed, Jul 04, 2018 at 10:50:28AM +0900, Michael Paquier wrote: > > On Tue, Jul 03, 2018 at 01:17:48AM -0400, Alvaro Herrera wrote: > > > Let me review tomorrow. > > > > Of course, please feel free. > > Alvaro, are you planning to look at that to close t

Re: Possible bug in logical replication.

2018-07-12 Thread Michael Paquier
On Wed, Jul 04, 2018 at 10:50:28AM +0900, Michael Paquier wrote: > On Tue, Jul 03, 2018 at 01:17:48AM -0400, Alvaro Herrera wrote: > > Let me review tomorrow. > > Of course, please feel free. Alvaro, are you planning to look at that to close the loop? The latest version is here: https://postgr.e

Re: Possible bug in logical replication.

2018-07-09 Thread Arseny Sher
Michael Paquier writes: > Okay, let's do as you suggest then. Do you find the attached adapted? Yes, thanks! -- Arseny Sher Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Possible bug in logical replication.

2018-07-09 Thread Michael Paquier
On Tue, Jul 03, 2018 at 09:16:42AM +0300, Arseny Sher wrote: > Michael Paquier writes: >> On Thu, Jun 21, 2018 at 07:31:20PM +0900, Michael Paquier wrote: >>> Could it be possible to get a patch from all the feedback and exchange >>> gathered here? Petr, I think that it would not hurt if you use

Re: Possible bug in logical replication.

2018-07-03 Thread Michael Paquier
On Tue, Jul 03, 2018 at 01:17:48AM -0400, Alvaro Herrera wrote: > Let me review tomorrow. Of course, please feel free. -- Michael signature.asc Description: PGP signature

Re: Possible bug in logical replication.

2018-07-02 Thread Arseny Sher
Michael Paquier writes: > On Thu, Jun 21, 2018 at 07:31:20PM +0900, Michael Paquier wrote: >> Could it be possible to get a patch from all the feedback and exchange >> gathered here? Petr, I think that it would not hurt if you use the set >> of words and comments you think is most adapted as t

Re: Possible bug in logical replication.

2018-07-02 Thread Alvaro Herrera
On 2018-Jul-03, Michael Paquier wrote: > On Thu, Jun 21, 2018 at 07:31:20PM +0900, Michael Paquier wrote: > > Could it be possible to get a patch from all the feedback and exchange > > gathered here? Petr, I think that it would not hurt if you use the set > > of words and comments you think is mo

Re: Possible bug in logical replication.

2018-07-02 Thread Michael Paquier
On Thu, Jun 21, 2018 at 07:31:20PM +0900, Michael Paquier wrote: > Could it be possible to get a patch from all the feedback and exchange > gathered here? Petr, I think that it would not hurt if you use the set > of words and comments you think is most adapted as the primary author of > the featur

Re: Possible bug in logical replication.

2018-06-22 Thread Michael Paquier
On Fri, Jun 22, 2018 at 04:33:12PM +0900, Kyotaro HORIGUCHI wrote: > pg_advance_replication_slots can advance uninitialized physical > slots and that might not be good. (Logical slots always have > initial invalid values in thw two lsn columns.) The current logic is careful that users willing to m

Re: Possible bug in logical replication.

2018-06-22 Thread Kyotaro HORIGUCHI
Hello. At Thu, 14 Jun 2018 16:06:43 -0400, Alvaro Herrera wrote in <20180614200643.3my362zmfiwitrni@alvherre.pgsql> > Can somebody (Arseny, Konstantin, horiguti, Sawada) please confirm that > Michaël's commit fixes the reported bug? pg_advance_replication_slots can advance uninitialized physica

Re: Possible bug in logical replication.

2018-06-21 Thread Michael Paquier
On Thu, Jun 21, 2018 at 12:18:44PM +0200, Petr Jelinek wrote: > On 20/06/18 09:59, Arseny Sher wrote: >> Michael Paquier writes: >>> It seems to me that we still want to have the slot forwarding finish in >>> this case even if this is interrupted. Petr, isn't that the intention >>> here? > > Wel

Re: Possible bug in logical replication.

2018-06-21 Thread Petr Jelinek
On 20/06/18 09:59, Arseny Sher wrote: > > Michael Paquier writes: > >> On Mon, Jun 18, 2018 at 09:42:36PM +0900, Michael Paquier wrote: >>> On Fri, Jun 15, 2018 at 06:27:56PM +0300, Arseny Sher wrote: >>> It seems to me that we still want to have the slot forwarding finish in >>> this case even

Re: Possible bug in logical replication.

2018-06-20 Thread Arseny Sher
Michael Paquier writes: > On Mon, Jun 18, 2018 at 09:42:36PM +0900, Michael Paquier wrote: >> On Fri, Jun 15, 2018 at 06:27:56PM +0300, Arseny Sher wrote: >> It seems to me that we still want to have the slot forwarding finish in >> this case even if this is interrupted. Petr, isn't that the in

Re: Possible bug in logical replication.

2018-06-19 Thread Michael Paquier
On Mon, Jun 18, 2018 at 09:42:36PM +0900, Michael Paquier wrote: > On Fri, Jun 15, 2018 at 06:27:56PM +0300, Arseny Sher wrote: > It seems to me that we still want to have the slot forwarding finish in > this case even if this is interrupted. Petr, isn't that the intention > here? I have been che

Re: Possible bug in logical replication.

2018-06-18 Thread Michael Paquier
On Fri, Jun 15, 2018 at 06:27:56PM +0300, Arseny Sher wrote: > I confirm that starting reading WAL since restart_lsn as implemented in > f731cfa fixes this issue, as well as the second issue tushar mentioned > at [1]. Thanks! +/* + * Start reading WAL at restart_lsn, which certainly point

Re: Possible bug in logical replication.

2018-06-15 Thread Arseny Sher
Alvaro Herrera writes: > Can somebody (Arseny, Konstantin, horiguti, Sawada) please confirm that > Michaël's commit fixes the reported bug? I confirm that starting reading WAL since restart_lsn as implemented in f731cfa fixes this issue, as well as the second issue tushar mentioned at [1]. I th

Re: Possible bug in logical replication.

2018-06-15 Thread Masahiko Sawada
On Fri, Jun 15, 2018 at 5:06 AM, Alvaro Herrera wrote: > Hello > > Can somebody (Arseny, Konstantin, horiguti, Sawada) please confirm that > Michaël's commit fixes the reported bug? > I don't confirm that commit deeply yet but I have confirmed that the reported bug is fixed using the following te

Re: Possible bug in logical replication.

2018-06-14 Thread Alvaro Herrera
Hello Can somebody (Arseny, Konstantin, horiguti, Sawada) please confirm that Michaël's commit fixes the reported bug? Thanks, -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Possible bug in logical replication.

2018-06-10 Thread Michael Paquier
On Thu, Jun 07, 2018 at 08:32:10AM +0100, Simon Riggs wrote: > On 6 June 2018 at 17:22, Alvaro Herrera wrote: >> This thread seems to have died down without any fix being proposed. >> Simon, you own this open item. > > Thanks, will look. Petr and I have found a couple of issues about the slot ad

Re: Possible bug in logical replication.

2018-06-07 Thread Simon Riggs
On 6 June 2018 at 17:22, Alvaro Herrera wrote: > This thread seems to have died down without any fix being proposed. > Simon, you own this open item. Thanks, will look. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Service

Re: Possible bug in logical replication.

2018-06-06 Thread Alvaro Herrera
This thread seems to have died down without any fix being proposed. Simon, you own this open item. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Possible bug in logical replication.

2018-05-24 Thread Arseny Sher
Michael Paquier writes: > Maybe I am being too naive, but wouldn't it be just enough to update the > confirmed flush LSN to ctx->reader->ReadRecPtr? This way, the slot > advances up to the beginning of the last record where user wants to > advance, and not the beginning of the next record: Sam

Re: Possible bug in logical replication.

2018-05-24 Thread Arseny Sher
Hello, Kyotaro HORIGUCHI writes: > restart_lsn stays at the beginning of a transaction until the > transaction ends so just using restart_lsn allows repeated > decoding of a transaction, in short, rewinding occurs. The > function works only for inactive slot so the current code works > fine on t

Re: Possible bug in logical replication.

2018-05-24 Thread Michael Paquier
On Thu, May 24, 2018 at 10:14:01AM +0900, Kyotaro HORIGUCHI wrote: > At Wed, 23 May 2018 15:56:22 +0900, Masahiko Sawada > wrote in >> Another possible way might be to make XLogFindNextRecord valid in >> backend code and move startlsn to the first valid record with an lsn >> >= startlsn by using

Re: Possible bug in logical replication.

2018-05-23 Thread Kyotaro HORIGUCHI
Hello. At Wed, 23 May 2018 15:56:22 +0900, Masahiko Sawada wrote in > > So directly set ctx->reader->EndRecPtr by startlsn fixes the > > problem, but I found another problem here. > > I confirmed that the attached patch fixes this problem as well as the > same problem reported on another threa

Re: Possible bug in logical replication.

2018-05-22 Thread Masahiko Sawada
On Fri, May 18, 2018 at 2:37 PM, Kyotaro HORIGUCHI wrote: > At Thu, 17 May 2018 13:54:07 +0300, Arseny Sher wrote > in <87in7md034.fsf@ars-thinkpad> >> >> Konstantin Knizhnik writes: >> >> > I think that using restart_lsn instead of confirmed_flush is not right >> > approach. >> > If restart_l

Re: Possible bug in logical replication.

2018-05-17 Thread Kyotaro HORIGUCHI
At Thu, 17 May 2018 13:54:07 +0300, Arseny Sher wrote in <87in7md034.fsf@ars-thinkpad> > > Konstantin Knizhnik writes: > > > I think that using restart_lsn instead of confirmed_flush is not right > > approach. > > If restart_lsn is not available and confirmed_flush is pointing to page > > bou

Re: Possible bug in logical replication.

2018-05-17 Thread Michael Paquier
On Thu, May 17, 2018 at 12:31:09PM +0300, Konstantin Knizhnik wrote: > I wonder who is now maintaining logical replication stuff and whether this > bug is going to be fixed? I haven't looked at your problem in details so I cannot give a sure conclusion, but when it comes to pg_replication_slot_adv

Re: Possible bug in logical replication.

2018-05-17 Thread Arseny Sher
Konstantin Knizhnik writes: > I think that using restart_lsn instead of confirmed_flush is not right > approach. > If restart_lsn is not available and confirmed_flush is pointing to page > boundary, then in any case we should somehow handle this case and adjust > startlsn to point on the valid

Re: Possible bug in logical replication.

2018-05-17 Thread Konstantin Knizhnik
On 17.05.2018 10:45, Konstantin Knizhnik wrote: We got the following assertion failure at our buildfarm of master branch of Postgres in contrib/test_decoding regression tests: 2018-05-07 19:50:07.241 MSK [5af083bf.54ae:49] DETAIL: Streaming transactions committing after 0/2A0, reading W

Possible bug in logical replication.

2018-05-17 Thread Konstantin Knizhnik
We got the following assertion failure at our buildfarm of master branch of Postgres in contrib/test_decoding regression tests: 2018-05-07 19:50:07.241 MSK [5af083bf.54ae:49] DETAIL: Streaming transactions committing after 0/2A0, reading WAL from 0/29FFF1C. 2018-05-07 19:50:07.241 MSK [5af