Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Thu, Nov 9, 2017 at 4:53 PM, Andres Freund wrote: > Primarily because it's not an anti-corruption tool. I'd be surprised if > there weren't ways to corrupt the page using these corruptions that > aren't detected by it. It's very hard to assess the risk of missing something

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On 2017-11-09 16:45:07 -0800, Peter Geoghegan wrote: > On Thu, Nov 9, 2017 at 4:17 PM, Andres Freund wrote: > >> Actually, on second thought, I take that back -- I don't think that > >> REINDEXing will even finish once a HOT chain is broken by the bug. > >>

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Thu, Nov 9, 2017 at 4:17 PM, Andres Freund wrote: >> Actually, on second thought, I take that back -- I don't think that >> REINDEXing will even finish once a HOT chain is broken by the bug. >> IndexBuildHeapScan() actually does quite a good job of making sure >> that HOT

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Thu, Nov 9, 2017 at 4:17 PM, Andres Freund wrote: >> I don't follow you here. Why would REINDEXing make the rows that >> should be dead disappear again, even for a short period of time? > > It's not the REINDEX that makes them reappear. Of course. I was just trying to make

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On 2017-11-09 16:02:17 -0800, Peter Geoghegan wrote: > > What I'm currently wondering about is how much we need to harden > > postgres against such existing corruption. If e.g. the hot chains are > > broken somebody might have reindexed thinking the problem is fixed - but > > if they then later

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Thu, Nov 9, 2017 at 2:24 PM, Andres Freund wrote: > Attached is a version of the already existing regression test that both > reproduces the broken hot chain (and thus failing index lookups) and > then also the tuple reviving. I don't see any need for letting this run >

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On 2017-11-04 06:15:00 -0700, Andres Freund wrote: > The reason for that is that I hadn't yet quite figured out how the bug I > described in the commit message (and the previously committed testcase) > would cause that. I was planning to diagnose / experiment more with this > and write an email if

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On 2017-11-04 06:15:00 -0700, Andres Freund wrote: > The current testcase, and I think the descriptions in the relevant > threads, all actually fail to point out the precise way the bug is > triggered. As e.g. evidenced that the freeze-the-dead case appears to > not cause any failures in

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On 2017-11-03 12:36:59 -0700, Peter Geoghegan wrote: > Andres Freund wrote: > > Here's that patch. I've stared at this some, and Robert did too. Robert > > mentioned that the commit message might need some polish and I'm not > > 100% sure about the error message texts yet. >

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On November 4, 2017 1:22:04 AM GMT+05:30, Alvaro Herrera wrote: >Peter Geoghegan wrote: >> Andres Freund wrote: > >> > Staring at the vacuumlazy hunk I think I might have found a related >bug: >> > heap_update_tuple() just copies the old xmax to

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Alvaro Herrera wrote: He means that the tuple that heap_update moves to page 1 (which will no longer be processed by vacuum) will contain a multixact that's older than relminmxid -- because it is copied unchanged by heap_update instead of properly checking against age

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Peter Geoghegan wrote: > Andres Freund wrote: > > Staring at the vacuumlazy hunk I think I might have found a related bug: > > heap_update_tuple() just copies the old xmax to the new tuple's xmax if > > a multixact and still running. It does so without verifying liveliness >

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Andres Freund wrote: Here's that patch. I've stared at this some, and Robert did too. Robert mentioned that the commit message might need some polish and I'm not 100% sure about the error message texts yet. The commit message should probably say that the bug involves the

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On 2017-11-02 06:05:51 -0700, Andres Freund wrote: > Hi, > > On 2017-11-02 13:49:47 +0100, Alvaro Herrera wrote: > > Andres Freund wrote: > > > I think the problem is on the pruning, rather than the freezing side. We > > > can't freeze a tuple if it has an alive predecessor - rather than > > >

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Robert Haas writes: > Well, my thought was that delaying this release for a week would be > better than either (a) doing an extra minor release just to get this > fix out or (b) waiting another three months to release this fix. The > former seems like fairly unnecessary

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Thu, Nov 2, 2017 at 10:38 PM, Tom Lane wrote: >> Personally, I think it would be best to push the release out a week. > > I would only be in favor of that if there were some reason to think that > the bug is worse now than it's been in the four years since 9.3 was >

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Thu, Nov 2, 2017 at 10:26 PM, Peter Geoghegan wrote: > On Thu, Nov 2, 2017 at 9:44 AM, Robert Haas wrote: >> The second commit (22576734b805fb0952f9be841ca8f643694ee868) is where >> I think things get a lot more dangerous. The problem (as Andres >>

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Robert Haas writes: > Just to be clear, it looks like "Fix freezing of a dead HOT-updated > tuple" (46c35116ae1acc8826705ef2a7b5d9110f9d6e84) went in before 10.0 > was stamped, but "Fix traversal of half-frozen update chains" > (22576734b805fb0952f9be841ca8f643694ee868)

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Thu, Nov 2, 2017 at 9:44 AM, Robert Haas wrote: > The second commit (22576734b805fb0952f9be841ca8f643694ee868) is where > I think things get a lot more dangerous. The problem (as Andres > pointed out to me this afternoon) is that it seems possible to end up > with a

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Thu, Nov 2, 2017 at 8:25 PM, Alvaro Herrera wrote: > Pushed the reverts. > > I noticed while doing so that REL_10_STABLE contains the bogus commits. > Does that change our opinion regarding what to do for people upgrading > to a version containing the broken commits?

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Andres Freund writes: > > > Do we care about people upgrading to unreleased versions? We could do > > > nothing, document it in the release notes, or ??? > > > > Do nothing. > > Agreed. Not much we can do

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

* Tom Lane (t...@sss.pgh.pa.us) wrote: > Andres Freund writes: > > Do we care about people upgrading to unreleased versions? We could do > > nothing, document it in the release notes, or ??? > > Do nothing. Agreed. Not much we can do there. Thanks! Stephen signature.asc

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Andres Freund writes: > Do we care about people upgrading to unreleased versions? We could do > nothing, document it in the release notes, or ??? Do nothing. regards, tom lane -- Sent via pgsql-committers mailing list

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Hi, On 2017-11-02 13:49:47 +0100, Alvaro Herrera wrote: > Andres Freund wrote: > > I think the problem is on the pruning, rather than the freezing side. We > > can't freeze a tuple if it has an alive predecessor - rather than > > weakining this, we should be fixing the pruning to not have the