Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Wed, Apr 27, 2016 at 12:03 PM, Andres Freund wrote: > On 2016-04-27 11:59:39 -0400, Robert Haas wrote: >> Masahiko Sawada posted a patch that fixes the problem for him, which >> does not involve any new WAL record type. It also seems to be fixing >> the problem in a way

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-27 11:59:39 -0400, Robert Haas wrote: > Masahiko Sawada posted a patch that fixes the problem for him, which > does not involve any new WAL record type. It also seems to be fixing > the problem in a way that is clean and consistent with what we've done > elsewhere. It only fixes one

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Wed, Apr 27, 2016 at 11:51 AM, Andres Freund wrote: > On 2016-04-27 14:27:33 +0900, Michael Paquier wrote: >> On Wed, Apr 27, 2016 at 12:25 AM, Robert Haas wrote: >> > In other words, I think Masahiko Sawada's patch in the original post >> > is

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-27 14:27:33 +0900, Michael Paquier wrote: > On Wed, Apr 27, 2016 at 12:25 AM, Robert Haas wrote: > > In other words, I think Masahiko Sawada's patch in the original post > > is pretty much right on target, except that we don't need to do that > > always, but

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Wed, Apr 27, 2016 at 12:25 AM, Robert Haas wrote: > In other words, I think Masahiko Sawada's patch in the original post > is pretty much right on target, except that we don't need to do that > always, but rather only in the FPI case when the call to >

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Mon, Apr 25, 2016 at 2:10 PM, Andres Freund wrote: > Afaics the problem described below was introduced in b4e07417, do you > have a different/better proposal than > s/CacheInvalidateSmgr/CacheInvalidateRelcache/? Doing that doesn't feel > quite right either, because it only

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 26 April 2016 at 04:48, Andres Freund wrote: > No, I think we got to do this in all branches. I was just wondering > about how to fix vm_extend(). Which I do think we got to fix, even in > the back-branches. > I think replacing CacheInvalidateSmgr() with

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-26 12:39:37 +0900, Michael Paquier wrote: > Thinking about the logging of smgr invalidations, this is quite > interesting. But what would we actually gain in doing that? Do you > foresee any advantages in doing so? The only case where this would be > useful now is for vm_extend by

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Tue, Apr 26, 2016 at 12:39 PM, Michael Paquier wrote: > On Tue, Apr 26, 2016 at 11:47 AM, Andres Freund wrote: >> Thanks for looking into this. >> >> On 2016-04-26 11:43:06 +0900, Michael Paquier wrote: >>> On Sun, Apr 24, 2016 at 11:51 AM,

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Tue, Apr 26, 2016 at 11:47 AM, Andres Freund wrote: > Thanks for looking into this. > > On 2016-04-26 11:43:06 +0900, Michael Paquier wrote: >> On Sun, Apr 24, 2016 at 11:51 AM, Andres Freund wrote: >> > ISTM we should additionally replace the

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

Hi, Thanks for looking into this. On 2016-04-26 11:43:06 +0900, Michael Paquier wrote: > On Sun, Apr 24, 2016 at 11:51 AM, Andres Freund wrote: > > ISTM we should additionally replace the CacheInvalidateSmgr() with a > > CacheInvalidateRelcache() and document that that

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Sun, Apr 24, 2016 at 11:51 AM, Andres Freund wrote: > Here's a patch doing so. Note that, after putting the record into RM_XACT_ID > first, I've decided to make it a RM_STANDBY_ID type record. I think it's > likely that this is going to be needed beyond transaction

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Tue, Apr 26, 2016 at 1:13 AM, Andres Freund wrote: > I think my approach of a separate record is going to be easier to > backpatch - the new commit record format you took advantage of is > new. Sorry for the late reply. After reading both patches yesterday, I found your

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

Dean, Robert, Afaics the problem described below was introduced in b4e07417, do you have a different/better proposal than s/CacheInvalidateSmgr/CacheInvalidateRelcache/? Doing that doesn't feel quite right either, because it only makes the file extension visible at end-of-xact - which is mostly

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-24 22:10:19 +0100, Simon Riggs wrote: > On 18 April 2016 at 13:15, Simon Riggs wrote: > > (non-reply just because of travel) > > > > OK, I'll write up a patch today to fix, with a view to backpatching. > > > > Patch from Tuesday. On various planes. I posted a

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 18 April 2016 at 13:15, Simon Riggs wrote: > On 18 April 2016 at 12:43, Michael Paquier > wrote: > >> On Sat, Apr 16, 2016 at 5:37 AM, Alvaro Herrera >> wrote: >> > Andres Freund wrote: >> >> On 2016-04-15 15:26:17

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-15 16:53:37 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-04-15 15:26:17 -0400, Tom Lane wrote: > >> I think the bottom line is that we misdesigned the WAL representation > >> by assuming that this sort of info could always be piggybacked on a > >>

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-18 13:15:19 +0100, Simon Riggs wrote: > OK, I'll write up a patch today to fix, with a view to backpatching. Cool. I've spent a couple minutes on this yesterday, and have the start of a patch - do you want that? - Andres -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-18 20:43:30 +0900, Michael Paquier wrote: > Yeah, introducing a new WAL record to address this issue in > back-branches would not be an issue, and that's what we should do. For > HEAD, let's add that in the commit record. I'm not sure why/how you'd do it that way in HEAD? I mean the

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-15 17:37:03 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2016-04-15 15:26:17 -0400, Tom Lane wrote: > > > I think the bottom line is that we misdesigned the WAL representation > > > by assuming that this sort of info could always be piggybacked on a > > > transaction

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 18 April 2016 at 12:43, Michael Paquier wrote: > On Sat, Apr 16, 2016 at 5:37 AM, Alvaro Herrera > wrote: > > Andres Freund wrote: > >> On 2016-04-15 15:26:17 -0400, Tom Lane wrote: > >> > I think the bottom line is that we misdesigned the

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Sat, Apr 16, 2016 at 5:37 AM, Alvaro Herrera wrote: > Andres Freund wrote: >> On 2016-04-15 15:26:17 -0400, Tom Lane wrote: >> > I think the bottom line is that we misdesigned the WAL representation >> > by assuming that this sort of info could always be piggybacked

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

Andres Freund writes: > On 2016-04-15 15:26:17 -0400, Tom Lane wrote: >> I think the bottom line is that we misdesigned the WAL representation >> by assuming that this sort of info could always be piggybacked on a >> transaction commit record. It's time to fix that. > I

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

Andres Freund wrote: > On 2016-04-15 15:26:17 -0400, Tom Lane wrote: > > I think the bottom line is that we misdesigned the WAL representation > > by assuming that this sort of info could always be piggybacked on a > > transaction commit record. It's time to fix that. > > I think we got to

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-15 15:26:17 -0400, Tom Lane wrote: > I think the bottom line is that we misdesigned the WAL representation > by assuming that this sort of info could always be piggybacked on a > transaction commit record. It's time to fix that. I think we got to piggyback it onto a commit record, as

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

Simon Riggs writes: > In my understanding we have two choices for this bug > 1) assign an xid so it forces sending a message (message plus xid) > 2) send a message without assigning an xid (message only) > (1) seems like it is worse for backpatching, IMHO, though I am

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 15 April 2016 at 20:01, Andres Freund wrote: > On 2016-04-15 19:59:06 +0100, Simon Riggs wrote: > > For me, the issue is that we need to do something to catch bugs. The > > existing code does not have any test at all to check whether we are doing > > the wrong thing - it

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-15 19:59:06 +0100, Simon Riggs wrote: > For me, the issue is that we need to do something to catch bugs. The > existing code does not have any test at all to check whether we are doing > the wrong thing - it just lets the wrong thing happen. But sending the message, without assigning

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 15 April 2016 at 18:44, Robert Haas wrote: > On Fri, Apr 15, 2016 at 1:12 PM, Tom Lane wrote: > > Robert Haas writes: > >> On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund > wrote: > >>> The easiest way to

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-15 13:44:27 -0400, Robert Haas wrote: > On Fri, Apr 15, 2016 at 1:12 PM, Tom Lane wrote: > > Robert Haas writes: > >> On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund wrote: > >>> The easiest way to achieve that seems to

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Fri, Apr 15, 2016 at 1:12 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund wrote: >>> The easiest way to achieve that seems to be to just assign an xid if >>> that's the case; while

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

Robert Haas writes: > On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund wrote: >> The easiest way to achieve that seems to be to just assign an xid if >> that's the case; while it's not necessarily safe/efficient to do so at >> the point the invalidation

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-15 13:07:19 -0400, Robert Haas wrote: > On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund wrote: > > On 2016-04-14 11:50:58 -0400, Robert Haas wrote: > >> On Wed, Apr 13, 2016 at 9:58 PM, Andres Freund wrote: > >> > We've recently discussed a very

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund wrote: > On 2016-04-14 11:50:58 -0400, Robert Haas wrote: >> On Wed, Apr 13, 2016 at 9:58 PM, Andres Freund wrote: >> > We've recently discussed a very similar issue around >> >

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-04-14 11:50:58 -0400, Robert Haas wrote: > On Wed, Apr 13, 2016 at 9:58 PM, Andres Freund wrote: > > We've recently discussed a very similar issue around > > http://www.postgresql.org/message-id/20160227002958.peftvmcx4dxwe...@alap3.anarazel.de > > > > Unfortunately

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Wed, Apr 13, 2016 at 9:58 PM, Andres Freund wrote: > We've recently discussed a very similar issue around > http://www.postgresql.org/message-id/20160227002958.peftvmcx4dxwe...@alap3.anarazel.de > > Unfortunately Simon over in that thread disagreed there about fixing > this

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On 2016-03-31 01:02:06 -0400, Noah Misch wrote: > On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote: > > As a result of looked into code around the recvoery, ISTM that the > > cause is related to relation cache clear. > > In heap_xlog_visible, if the standby server receives WAL

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Fri, Apr 1, 2016 at 9:10 AM, Noah Misch wrote: > On Thu, Mar 31, 2016 at 04:48:26PM +0900, Masahiko Sawada wrote: >> On Thu, Mar 31, 2016 at 2:02 PM, Noah Misch wrote: >> > On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote: >> >> As a result

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Thu, Mar 31, 2016 at 04:48:26PM +0900, Masahiko Sawada wrote: > On Thu, Mar 31, 2016 at 2:02 PM, Noah Misch wrote: > > On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote: > >> As a result of looked into code around the recvoery, ISTM that the > >> cause is

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Thu, Mar 31, 2016 at 2:02 PM, Noah Misch wrote: > On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote: >> As a result of looked into code around the recvoery, ISTM that the >> cause is related to relation cache clear. >> In heap_xlog_visible, if the standby

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote: > As a result of looked into code around the recvoery, ISTM that the > cause is related to relation cache clear. > In heap_xlog_visible, if the standby server receives WAL record then > relation cache is eventually cleared in

[HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

Hi all, I faced suspicious behaviour on hot standby server related to visibility map. The scenario is, 1. Create table and check internal of visibility map on master server. postgres(1)=# create table hoge (col int); CREATE TABLE postgres(1)=# insert into hoge select generate_series(1,10);