Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-12 Thread Andrey Borodin
> 12 янв. 2021 г., в 13:49, Noah Misch написал(а): > > What do you think of abandoning slru-truncate-t-insurance entirely? As of > https://postgr.es/m/20200330052809.gb2324...@rfd.leadboat.com I liked the idea > behind it, despite its complicating the system for hackers and DBAs. The >

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-12 Thread Noah Misch
On Mon, Jan 11, 2021 at 11:22:05AM +0500, Andrey Borodin wrote: > I'm marking patch as ready for committer. Thanks. > I can't tell should we backpatch insurance patch or not: it potentially fixes > unknown bugs, and potentially contains unknown bugs. I can't reason because > of such

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-10 Thread Andrey Borodin
> 10 янв. 2021 г., в 14:43, Noah Misch написал(а): > >> Can you please send revised patches with fixes? > > Attached. > > I'm marking patch as ready for committer. I can't tell should we backpatch insurance patch or not: it potentially fixes unknown bugs, and potentially contains

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-10 Thread Noah Misch
On Sun, Jan 10, 2021 at 11:44:14AM +0500, Andrey Borodin wrote: > > 10 янв. 2021 г., в 03:15, Noah Misch написал(а): > > > > No; it deletes the most recent ~1B and leaves the older segments. An > > exception is multixact, as described in the commit message and the patch's > > change to a

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-09 Thread Andrey Borodin
> 10 янв. 2021 г., в 03:15, Noah Misch написал(а): > > No; it deletes the most recent ~1B and leaves the older segments. An > exception is multixact, as described in the commit message and the patch's > change to a comment in TruncateMultiXact(). Thanks for clarification. One more thing:

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-09 Thread Noah Misch
On Sat, Jan 09, 2021 at 08:25:39PM +0500, Andrey Borodin wrote: > > 9 янв. 2021 г., в 15:17, Noah Misch написал(а): > >> I'm a little bit afraid that this kind of patch can hide bugs (while > >> potentially saving some users data). Besides this patch seems like a > >> useful precaution. Maybe

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-09 Thread Andrey Borodin
> 9 янв. 2021 г., в 15:17, Noah Misch написал(а): > >> This >> int diff_max = ((QUEUE_MAX_PAGE + 1) / 2) - 1, >> seems to be functional equivalent of >> int diff_max = ((QUEUE_MAX_PAGE - 1) / 2), > > Do you think one conveys the concept better than the other? I see now that next comments

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-09 Thread Noah Misch
On Wed, Jan 06, 2021 at 11:28:36AM +0500, Andrey Borodin wrote: > First patch fixes a bug with possible SLRU truncation around wrapping point > too early. > Basic idea of the patch is "If we want to delete a range we must be eligible > to delete it's beginning and ending". > So to test if page

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-05 Thread Andrey Borodin
> 1 янв. 2021 г., в 23:05, Andrey Borodin написал(а): > > I've found this thread in CF looking for something to review. We discussed patches with Noah offlist. I'm resending summary to list. There are two patches: 1. slru-truncate-modulo-v5.patch 2. slru-truncate-t-insurance-v4.patch It

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-02 Thread Noah Misch
On Sat, Jan 02, 2021 at 12:31:45PM +0500, Andrey Borodin wrote: > Do I understand correctly that this is bugfix that needs to be back-patched? The slru-truncate-modulo patch fixes a bug. The slru-truncate-t-insurance patch does not. Neither _needs_ to be back-patched, though I'm proposing to

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-01 Thread Andrey Borodin
> 2 янв. 2021 г., в 01:35, Noah Misch написал(а): > There's no > other connection to this thread, and one can review patches on this thread > without studying commit c732c3f. OK, thanks! Do I understand correctly that this is bugfix that needs to be back-patched? Thus we should not refactor

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-01 Thread Noah Misch
On Fri, Jan 01, 2021 at 11:05:29PM +0500, Andrey Borodin wrote: > I've found this thread in CF looking for something to review. Thanks for taking a look. > > 9 нояб. 2020 г., в 09:53, Noah Misch написал(а): > > > > Rebased both patches, necessitated by commit c732c3f (a repair of commit > >

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2021-01-01 Thread Andrey Borodin
Hi Noah! I've found this thread in CF looking for something to review. > 9 нояб. 2020 г., в 09:53, Noah Misch написал(а): > > Rebased both patches, necessitated by commit c732c3f (a repair of commit > dee663f). As I mentioned on another branch of the thread, I'd be content if > someone

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-11-08 Thread Noah Misch
On Wed, Oct 28, 2020 at 09:01:59PM -0700, Noah Misch wrote: > On Sat, Aug 29, 2020 at 10:34:33PM -0700, Noah Misch wrote: > > On Mon, May 25, 2020 at 12:00:33AM -0700, Noah Misch wrote: > > > [last non-rebase change] > > > > Rebased the second patch. The first patch did not need a rebase. > >

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-10-28 Thread Noah Misch
On Sat, Aug 29, 2020 at 10:34:33PM -0700, Noah Misch wrote: > On Mon, May 25, 2020 at 12:00:33AM -0700, Noah Misch wrote: > > On Mon, Apr 06, 2020 at 09:18:47PM -0700, Noah Misch wrote: > > > On Mon, Apr 06, 2020 at 02:46:09PM -0400, Tom Lane wrote: > > > > Noah Misch writes: > > > > > On Wed,

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-09-30 Thread Noah Misch
On Wed, Sep 30, 2020 at 03:06:40PM +0900, Michael Paquier wrote: > On Sun, Sep 06, 2020 at 08:14:21PM -0700, Noah Misch wrote: > > http://cfbot.cputube.org/patch_29_2026.log applies the two patches in the > > wrong order. For this CF entry, ignore it. > > OK, thanks. This is a bug fix, so I

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-09-30 Thread Michael Paquier
On Sun, Sep 06, 2020 at 08:14:21PM -0700, Noah Misch wrote: > http://cfbot.cputube.org/patch_29_2026.log applies the two patches in the > wrong order. For this CF entry, ignore it. OK, thanks. This is a bug fix, so I have moved that to the next CF for now. Noah, would you prefer more reviews

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-09-06 Thread Noah Misch
On Mon, Sep 07, 2020 at 11:06:12AM +0900, Michael Paquier wrote: > On Sat, Aug 29, 2020 at 10:34:33PM -0700, Noah Misch wrote: > > Rebased the second patch. The first patch did not need a rebase. > > It looks like a new rebase is needed, the CF bot is complaining here.

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-09-06 Thread Michael Paquier
On Sat, Aug 29, 2020 at 10:34:33PM -0700, Noah Misch wrote: > Rebased the second patch. The first patch did not need a rebase. It looks like a new rebase is needed, the CF bot is complaining here. -- Michael signature.asc Description: PGP signature

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-08-29 Thread Noah Misch
On Mon, May 25, 2020 at 12:00:33AM -0700, Noah Misch wrote: > On Mon, Apr 06, 2020 at 09:18:47PM -0700, Noah Misch wrote: > > On Mon, Apr 06, 2020 at 02:46:09PM -0400, Tom Lane wrote: > > > Noah Misch writes: > > > > On Wed, Mar 25, 2020 at 04:42:31PM -0400, Tom Lane wrote: > > > >> So I think

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-05-25 Thread Noah Misch
On Mon, Apr 06, 2020 at 09:18:47PM -0700, Noah Misch wrote: > On Mon, Apr 06, 2020 at 02:46:09PM -0400, Tom Lane wrote: > > Noah Misch writes: > > > On Wed, Mar 25, 2020 at 04:42:31PM -0400, Tom Lane wrote: > > >> So I think what we're actually trying to accomplish here is to > > >> ensure that

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-04-06 Thread Noah Misch
On Mon, Apr 06, 2020 at 02:46:09PM -0400, Tom Lane wrote: > Noah Misch writes: > > On Wed, Mar 25, 2020 at 04:42:31PM -0400, Tom Lane wrote: > >> So I think what we're actually trying to accomplish here is to > >> ensure that instead of deleting up to half of the SLRU space > >> before the

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-04-06 Thread Tom Lane
Noah Misch writes: > On Wed, Mar 25, 2020 at 04:42:31PM -0400, Tom Lane wrote: >> So I think what we're actually trying to accomplish here is to >> ensure that instead of deleting up to half of the SLRU space >> before the cutoff, we delete up to half-less-one-segment. >> Maybe it should be

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-03-29 Thread Noah Misch
On Wed, Mar 25, 2020 at 04:42:31PM -0400, Tom Lane wrote: > Noah Misch writes: > > On Thu, Mar 19, 2020 at 06:04:52PM -0400, Tom Lane wrote: > >> 3. It feels like the proposed test of cutoff position against both > >> ends of a segment is a roundabout way of fixing the problem. I > >> wonder

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-03-25 Thread Tom Lane
Noah Misch writes: > On Thu, Mar 19, 2020 at 06:04:52PM -0400, Tom Lane wrote: >> 1. It seems like this discussion is conflating two different issues. > When the newest XID and the oldest XID fall in "opposite" segments in the XID > space, we must not unlink the segment containing the newest

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-03-22 Thread Noah Misch
On Thu, Mar 19, 2020 at 06:04:52PM -0400, Tom Lane wrote: > Yeah, this patch has been waiting in the queue for way too long :-(. Thanks for reviewing. > I spent some time studying this, and I have a few comments/questions: > > 1. It seems like this discussion is conflating two different issues.

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-03-19 Thread Tom Lane
Noah Misch writes: > On Sun, Jan 05, 2020 at 01:33:55AM +0100, Tomas Vondra wrote: >> It's a bit unfortunate that a patch for a data corruption / loss issue >> (even if a low-probability one) fell through multiple commitfests. > Thanks for investing in steps to fix that. Yeah, this patch has

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-01-04 Thread Noah Misch
On Sun, Jan 05, 2020 at 01:33:55AM +0100, Tomas Vondra wrote: > On Wed, Feb 13, 2019 at 11:26:23PM -0800, Noah Misch wrote: > >a. Fix the rounding in SimpleLruTruncate(). (The patch I posted upthread is > > wrong; I will correct it in a separate message.) > > > >b. Arrange so only one backend

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-01-04 Thread Tomas Vondra
On Wed, Feb 13, 2019 at 11:26:23PM -0800, Noah Misch wrote: On Sat, Feb 02, 2019 at 03:38:22AM -0500, Noah Misch wrote: The main consequence is the false alarm. That conclusion was incorrect. On further study, I was able to reproduce data loss via either of two weaknesses in the "apparent

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2019-07-24 Thread Noah Misch
On Wed, Jul 24, 2019 at 05:27:18PM +0900, Kyotaro Horiguchi wrote: > Sorry in advance for link-breaking message forced by gmail.. Using the archives page "Resend email" link avoids that. > https://www.postgresql.org/message-id/flat/20190202083822.gc32...@gust.leadboat.com > > > 1. The result of

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2019-07-24 Thread Kyotaro Horiguchi
Sorry in advance for link-breaking message forced by gmail.. https://www.postgresql.org/message-id/flat/20190202083822.gc32...@gust.leadboat.com > 1. The result of the test is valid only until we release the SLRU ControlLock, >which we do before SlruScanDirCbDeleteCutoff() uses the cutoff to

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2019-02-16 Thread Noah Misch
On Wed, Feb 13, 2019 at 11:26:23PM -0800, Noah Misch wrote: > On further study, I was able to reproduce data loss > Fixes are available: > > a. Fix the rounding in SimpleLruTruncate(). (The patch I posted upthread is >wrong; I will correct it in a separate message.) Here's a corrected

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2019-02-13 Thread Noah Misch
On Sat, Feb 02, 2019 at 03:38:22AM -0500, Noah Misch wrote: > The main consequence is the false alarm. That conclusion was incorrect. On further study, I was able to reproduce data loss via either of two weaknesses in the "apparent wraparound" test: 1. The result of the test is valid only until

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2019-02-10 Thread Noah Misch
On Sat, Feb 02, 2019 at 03:38:22AM -0500, Noah Misch wrote: > The main consequence is the false alarm. A prudent DBA will want to react to > true wraparound, but no such wraparound has occurred. Also, we temporarily > waste disk space in pg_xact. This feels like a recipe for future bugs. The >