Re: Background writer and checkpointer in crash recovery

2022-09-21 Thread Michael Paquier
On Thu, Sep 15, 2022 at 10:19:01PM -0500, Justin Pryzby wrote: > I don't know that this warrants an Opened Item, but I think some fix > ought to be applied to v15, whether that happens this week or next > month. With RC1 getting close by, I have looked at that again and applied a patch that

Re: Background writer and checkpointer in crash recovery

2022-09-15 Thread Justin Pryzby
I don't know that this warrants an Opened Item, but I think some fix ought to be applied to v15, whether that happens this week or next month.

Re: Background writer and checkpointer in crash recovery

2022-09-13 Thread Justin Pryzby
On Tue, Sep 13, 2022 at 01:32:11PM +0900, Michael Paquier wrote: > On Mon, Sep 12, 2022 at 09:13:11PM -0500, Justin Pryzby wrote: > > Like this, maybe. > > > > It's similar to what I suggested to consider backpatching here: > >

Re: Background writer and checkpointer in crash recovery

2022-09-12 Thread Michael Paquier
On Mon, Sep 12, 2022 at 09:13:11PM -0500, Justin Pryzby wrote: > Like this, maybe. > > It's similar to what I suggested to consider backpatching here: > https://www.postgresql.org/message-id/20201214032224.GA30237%40telsasoft.com At the same time, df9274adf has been done because the

Re: Background writer and checkpointer in crash recovery

2022-09-12 Thread Justin Pryzby
On Sun, Sep 11, 2022 at 07:54:43PM -0500, Justin Pryzby wrote: > On Tue, Aug 03, 2021 at 02:19:22PM +1200, Thomas Munro wrote: > > On Tue, Aug 3, 2021 at 9:52 AM Thomas Munro wrote: > > > On Tue, Aug 3, 2021 at 1:17 AM Robert Haas wrote: > > > > That's great. I just realized that this leaves us

Re: Background writer and checkpointer in crash recovery

2022-09-11 Thread Justin Pryzby
On Tue, Aug 03, 2021 at 02:19:22PM +1200, Thomas Munro wrote: > On Tue, Aug 3, 2021 at 9:52 AM Thomas Munro wrote: > > On Tue, Aug 3, 2021 at 1:17 AM Robert Haas wrote: > > > That's great. I just realized that this leaves us with identical > > > RequestCheckpoint() calls in two nearby places. Is

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Thomas Munro
On Tue, Aug 3, 2021 at 9:52 AM Thomas Munro wrote: > On Tue, Aug 3, 2021 at 1:17 AM Robert Haas wrote: > > That's great. I just realized that this leaves us with identical > > RequestCheckpoint() calls in two nearby places. Is there any reason > > not to further simplify as in the attached? > >

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Thomas Munro
On Tue, Aug 3, 2021 at 1:17 AM Robert Haas wrote: > That's great. I just realized that this leaves us with identical > RequestCheckpoint() calls in two nearby places. Is there any reason > not to further simplify as in the attached? LGTM.

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Robert Haas
On Mon, Aug 2, 2021 at 9:51 AM Jakub Wartak wrote: > BTW, if you now there's this big push for refactoring StartupXLOG() then what > frustrating^H^H^H^H^H could be done better - at least from end-user point of > view - > is that there is lack of near real time cyclic messages (every 1min?)

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Robert Haas
On Mon, Aug 2, 2021 at 9:40 AM Amul Sul wrote: > > That's great. I just realized that this leaves us with identical > > RequestCheckpoint() calls in two nearby places. Is there any reason > > not to further simplify as in the attached? > > > +1, also, can we just get rid off "promoted" flag? The

RE: Background writer and checkpointer in crash recovery

2021-08-02 Thread Jakub Wartak
> On Fri, Jul 30, 2021 at 4:00 PM Andres Freund wrote: > > I don't agree with that? If (user+system) << wall then it is very > > likely that recovery is IO bound. If system is a large percentage of > > wall, then shared buffers is likely too small (or we're replacing the > > wrong > > buffers)

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Amul Sul
On Mon, Aug 2, 2021 at 6:47 PM Robert Haas wrote: > > On Mon, Aug 2, 2021 at 1:37 AM Thomas Munro wrote: > > I pushed 0001. > > That's great. I just realized that this leaves us with identical > RequestCheckpoint() calls in two nearby places. Is there any reason > not to further simplify as in

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Robert Haas
On Mon, Aug 2, 2021 at 1:37 AM Thomas Munro wrote: > I pushed 0001. That's great. I just realized that this leaves us with identical RequestCheckpoint() calls in two nearby places. Is there any reason not to further simplify as in the attached? -- Robert Haas EDB: http://www.enterprisedb.com

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Robert Haas
On Fri, Jul 30, 2021 at 4:00 PM Andres Freund wrote: > I don't agree with that? If (user+system) << wall then it is very likely > that recovery is IO bound. If system is a large percentage of wall, then > shared buffers is likely too small (or we're replacing the wrong > buffers) because you

Re: Background writer and checkpointer in crash recovery

2021-08-01 Thread Thomas Munro
On Sat, Jul 31, 2021 at 2:16 AM Robert Haas wrote: > On Fri, Jul 30, 2021 at 4:42 AM Aleksander Alekseev > wrote: > > v2-0001 and v2-0002 look fine, but I don't like much the idea of > > introducing a new GUC in v2-0003. It's for very specific needs, which most > > of the users, I believe,

Re: Background writer and checkpointer in crash recovery

2021-07-30 Thread Andres Freund
Hi, On 2021-07-30 10:16:44 -0400, Robert Haas wrote: > 2021-07-30 09:39:43.579 EDT [63702] LOG: redo starts at 0/14A2F48 > 2021-07-30 09:39:44.129 EDT [63702] LOG: redo done at 0/15F48230 > system usage: CPU: user: 0.25 s, system: 0.25 s, elapsed: 0.55 s > 2021-07-30 09:39:44.129 EDT [63702]

Re: Background writer and checkpointer in crash recovery

2021-07-30 Thread Robert Haas
On Fri, Jul 30, 2021 at 4:42 AM Aleksander Alekseev wrote: > v2-0001 and v2-0002 look fine, but I don't like much the idea of introducing > a new GUC in v2-0003. It's for very specific needs, which most of the users, > I believe, don't care about. I suggest dealing with v2-0001 and v2-0002

Re: Background writer and checkpointer in crash recovery

2021-07-30 Thread Aleksander Alekseev
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed v2-0001 and v2-0002 look fine, but I don't like much the

Re: Background writer and checkpointer in crash recovery

2021-03-12 Thread Thomas Munro
On Wed, Feb 3, 2021 at 11:11 AM Robert Haas wrote: > I think the way it works right now is stupid and the proposed change > is going in the right direction. We have ample evidence already that > handing off fsyncs to a background process is a good idea, and there's > no reason why that shouldn't

Re: Background writer and checkpointer in crash recovery

2021-02-02 Thread Robert Haas
On Sat, Aug 29, 2020 at 8:13 PM Thomas Munro wrote: > Currently we don't run the bgwriter process during crash recovery. > I've CCed Simon and Heikki who established this in commit cdd46c76. > Based on that commit message, I think the bar to clear to change the > policy is to show that it's

Re: Background writer and checkpointer in crash recovery

2020-11-11 Thread Thomas Munro
On Wed, Nov 11, 2020 at 9:57 PM Simon Riggs wrote: > Having said that, we did raise the checkpoint_timeout by a lot, so the > situation today might be quite different. A large checkpoint_timeout > could eventually overflow shared buffers, with the right workload. FWIW Jakuk Wartak did manage to

Re: Background writer and checkpointer in crash recovery

2020-11-11 Thread Simon Riggs
On Sun, 30 Aug 2020 at 01:39, Tom Lane wrote: > > Thomas Munro writes: > > Once we had the checkpointer running, we could also consider making > > the end-of-recovery checkpoint optional, or at least the wait for it > > to complete. I haven't shown that in this patch, but it's just > >

Re: Background writer and checkpointer in crash recovery

2020-08-29 Thread Tom Lane
Thomas Munro writes: > Once we had the checkpointer running, we could also consider making > the end-of-recovery checkpoint optional, or at least the wait for it > to complete. I haven't shown that in this patch, but it's just > different checkpoint request flags and possibly an end-of-recovery

Background writer and checkpointer in crash recovery

2020-08-29 Thread Thomas Munro
(Forking this thread from the SLRU fsync one[1] to allow for a separate CF entry; it's unrelated, except for being another case of moving work off the recovery process's plate.) Hello hackers, Currently we don't run the bgwriter process during crash recovery. I've CCed Simon and Heikki who