Re: Background writer and checkpointer in crash recovery

2022-09-21 Thread Michael Paquier
On Thu, Sep 15, 2022 at 10:19:01PM -0500, Justin Pryzby wrote: > I don't know that this warrants an Opened Item, but I think some fix > ought to be applied to v15, whether that happens this week or next > month. With RC1 getting close by, I have looked at that again and applied a patch that resets

Re: Background writer and checkpointer in crash recovery

2022-09-15 Thread Justin Pryzby
I don't know that this warrants an Opened Item, but I think some fix ought to be applied to v15, whether that happens this week or next month.

Re: Background writer and checkpointer in crash recovery

2022-09-13 Thread Justin Pryzby
On Tue, Sep 13, 2022 at 01:32:11PM +0900, Michael Paquier wrote: > On Mon, Sep 12, 2022 at 09:13:11PM -0500, Justin Pryzby wrote: > > Like this, maybe. > > > > It's similar to what I suggested to consider backpatching here: > > https://www.postgresql.org/message-id/20201214032224.GA30237%40telsaso

Re: Background writer and checkpointer in crash recovery

2022-09-12 Thread Michael Paquier
On Mon, Sep 12, 2022 at 09:13:11PM -0500, Justin Pryzby wrote: > Like this, maybe. > > It's similar to what I suggested to consider backpatching here: > https://www.postgresql.org/message-id/20201214032224.GA30237%40telsasoft.com At the same time, df9274adf has been done because the end-of-recove

Re: Background writer and checkpointer in crash recovery

2022-09-12 Thread Justin Pryzby
On Sun, Sep 11, 2022 at 07:54:43PM -0500, Justin Pryzby wrote: > On Tue, Aug 03, 2021 at 02:19:22PM +1200, Thomas Munro wrote: > > On Tue, Aug 3, 2021 at 9:52 AM Thomas Munro wrote: > > > On Tue, Aug 3, 2021 at 1:17 AM Robert Haas wrote: > > > > That's great. I just realized that this leaves us w

Re: Background writer and checkpointer in crash recovery

2022-09-11 Thread Justin Pryzby
On Tue, Aug 03, 2021 at 02:19:22PM +1200, Thomas Munro wrote: > On Tue, Aug 3, 2021 at 9:52 AM Thomas Munro wrote: > > On Tue, Aug 3, 2021 at 1:17 AM Robert Haas wrote: > > > That's great. I just realized that this leaves us with identical > > > RequestCheckpoint() calls in two nearby places. Is

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Thomas Munro
On Tue, Aug 3, 2021 at 9:52 AM Thomas Munro wrote: > On Tue, Aug 3, 2021 at 1:17 AM Robert Haas wrote: > > That's great. I just realized that this leaves us with identical > > RequestCheckpoint() calls in two nearby places. Is there any reason > > not to further simplify as in the attached? > > L

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Thomas Munro
On Tue, Aug 3, 2021 at 1:17 AM Robert Haas wrote: > That's great. I just realized that this leaves us with identical > RequestCheckpoint() calls in two nearby places. Is there any reason > not to further simplify as in the attached? LGTM.

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Robert Haas
On Mon, Aug 2, 2021 at 9:51 AM Jakub Wartak wrote: > BTW, if you now there's this big push for refactoring StartupXLOG() then what > frustrating^H^H^H^H^H could be done better - at least from end-user point of > view - > is that there is lack of near real time cyclic messages (every 1min?) about

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Robert Haas
On Mon, Aug 2, 2021 at 9:40 AM Amul Sul wrote: > > That's great. I just realized that this leaves us with identical > > RequestCheckpoint() calls in two nearby places. Is there any reason > > not to further simplify as in the attached? > > > +1, also, can we just get rid off "promoted" flag? The o

RE: Background writer and checkpointer in crash recovery

2021-08-02 Thread Jakub Wartak
> On Fri, Jul 30, 2021 at 4:00 PM Andres Freund wrote: > > I don't agree with that? If (user+system) << wall then it is very > > likely that recovery is IO bound. If system is a large percentage of > > wall, then shared buffers is likely too small (or we're replacing the > > wrong > > buffers) bec

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Amul Sul
On Mon, Aug 2, 2021 at 6:47 PM Robert Haas wrote: > > On Mon, Aug 2, 2021 at 1:37 AM Thomas Munro wrote: > > I pushed 0001. > > That's great. I just realized that this leaves us with identical > RequestCheckpoint() calls in two nearby places. Is there any reason > not to further simplify as in th

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Robert Haas
On Mon, Aug 2, 2021 at 1:37 AM Thomas Munro wrote: > I pushed 0001. That's great. I just realized that this leaves us with identical RequestCheckpoint() calls in two nearby places. Is there any reason not to further simplify as in the attached? -- Robert Haas EDB: http://www.enterprisedb.com

Re: Background writer and checkpointer in crash recovery

2021-08-02 Thread Robert Haas
On Fri, Jul 30, 2021 at 4:00 PM Andres Freund wrote: > I don't agree with that? If (user+system) << wall then it is very likely > that recovery is IO bound. If system is a large percentage of wall, then > shared buffers is likely too small (or we're replacing the wrong > buffers) because you spend

Re: Background writer and checkpointer in crash recovery

2021-08-01 Thread Thomas Munro
On Sat, Jul 31, 2021 at 2:16 AM Robert Haas wrote: > On Fri, Jul 30, 2021 at 4:42 AM Aleksander Alekseev > wrote: > > v2-0001 and v2-0002 look fine, but I don't like much the idea of > > introducing a new GUC in v2-0003. It's for very specific needs, which most > > of the users, I believe, don'

Re: Background writer and checkpointer in crash recovery

2021-07-30 Thread Andres Freund
Hi, On 2021-07-30 10:16:44 -0400, Robert Haas wrote: > 2021-07-30 09:39:43.579 EDT [63702] LOG: redo starts at 0/14A2F48 > 2021-07-30 09:39:44.129 EDT [63702] LOG: redo done at 0/15F48230 > system usage: CPU: user: 0.25 s, system: 0.25 s, elapsed: 0.55 s > 2021-07-30 09:39:44.129 EDT [63702] LOG

Re: Background writer and checkpointer in crash recovery

2021-07-30 Thread Robert Haas
On Fri, Jul 30, 2021 at 4:42 AM Aleksander Alekseev wrote: > v2-0001 and v2-0002 look fine, but I don't like much the idea of introducing > a new GUC in v2-0003. It's for very specific needs, which most of the users, > I believe, don't care about. I suggest dealing with v2-0001 and v2-0002 first

Re: Background writer and checkpointer in crash recovery

2021-07-30 Thread Aleksander Alekseev
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed v2-0001 and v2-0002 look fine, but I don't like much the idea

Re: Background writer and checkpointer in crash recovery

2021-03-12 Thread Thomas Munro
On Wed, Feb 3, 2021 at 11:11 AM Robert Haas wrote: > I think the way it works right now is stupid and the proposed change > is going in the right direction. We have ample evidence already that > handing off fsyncs to a background process is a good idea, and there's > no reason why that shouldn't b

Re: Background writer and checkpointer in crash recovery

2021-02-02 Thread Robert Haas
On Sat, Aug 29, 2020 at 8:13 PM Thomas Munro wrote: > Currently we don't run the bgwriter process during crash recovery. > I've CCed Simon and Heikki who established this in commit cdd46c76. > Based on that commit message, I think the bar to clear to change the > policy is to show that it's useful

Re: Background writer and checkpointer in crash recovery

2020-11-11 Thread Thomas Munro
On Wed, Nov 11, 2020 at 9:57 PM Simon Riggs wrote: > Having said that, we did raise the checkpoint_timeout by a lot, so the > situation today might be quite different. A large checkpoint_timeout > could eventually overflow shared buffers, with the right workload. FWIW Jakuk Wartak did manage to s

Re: Background writer and checkpointer in crash recovery

2020-11-11 Thread Simon Riggs
On Sun, 30 Aug 2020 at 01:39, Tom Lane wrote: > > Thomas Munro writes: > > Once we had the checkpointer running, we could also consider making > > the end-of-recovery checkpoint optional, or at least the wait for it > > to complete. I haven't shown that in this patch, but it's just > > different

Re: Background writer and checkpointer in crash recovery

2020-08-29 Thread Tom Lane
Thomas Munro writes: > Once we had the checkpointer running, we could also consider making > the end-of-recovery checkpoint optional, or at least the wait for it > to complete. I haven't shown that in this patch, but it's just > different checkpoint request flags and possibly an end-of-recovery >