Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-30 Thread Bharath Rupireddy
On Mon, Jan 31, 2022 at 12:47 PM Michael Paquier wrote: > > On Mon, Jan 31, 2022 at 01:54:16PM +0800, Julien Rouhaud wrote: > > For now we have some room, but we will likely keep consuming bytes in that > > file > > for more critical features and it's almost certain that at one point we will > >

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-30 Thread Michael Paquier
On Mon, Jan 31, 2022 at 01:54:16PM +0800, Julien Rouhaud wrote: > For now we have some room, but we will likely keep consuming bytes in that > file > for more critical features and it's almost certain that at one point we will > regret wasting 2 bytes for that. Agreed to drop the patch. That's

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-30 Thread Julien Rouhaud
Hi, On Mon, Jan 31, 2022 at 10:58:31AM +0530, Bharath Rupireddy wrote: > > The size of ControlFileData is 296 bytes currently and the sector > limit is of 512 bytes (PG_CONTROL_MAX_SAFE_SIZE), if we feel that this > extra 2 bytes of checkpoint flags isn't worth storing in the control > file, I'm

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-30 Thread Bharath Rupireddy
On Mon, Jan 31, 2022 at 9:10 AM Julien Rouhaud wrote: > > Hi, > > On Mon, Jan 31, 2022 at 11:10:45AM +0900, Kyotaro Horiguchi wrote: > > > > This means pg_controldata need to translate the flags into human-readable > > text but, to be clear, I still don't think its usefull in the control > >

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-30 Thread Julien Rouhaud
Hi, On Mon, Jan 31, 2022 at 11:10:45AM +0900, Kyotaro Horiguchi wrote: > > This means pg_controldata need to translate the flags into human-readable > text but, to be clear, I still don't think its usefull in the control > data. I've been saying that since my first email, I also don't see any

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-30 Thread Kyotaro Horiguchi
At Sat, 29 Jan 2022 00:10:19 +0800, Julien Rouhaud wrote in > On Fri, Jan 28, 2022 at 08:21:52PM +0530, Bharath Rupireddy wrote: > > > Also, you still didn't fix the possible flag upgrade issue. > > Unless I'm missing something that's an issue that you still haven't addressed > or explained

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-30 Thread Kyotaro Horiguchi
At Fri, 28 Jan 2022 13:49:19 +0530, Bharath Rupireddy wrote in > > - proallargtypes => > > '{pg_lsn,pg_lsn,text,int4,int4,bool,text,oid,xid,xid,xid,oid,xid,xid,oid,xid,xid,timestamptz}', > > + proallargtypes => > >

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Andres Freund
Hi, On 2021-12-07 20:06:22 +0530, Bharath Rupireddy wrote: > One concern is that we don't want to increase the size of pg_controldata by > more than the typical block size (of 8K) to avoid any torn-writes. The limit is 512 bytes (a disk sector), not 8K. There are plenty devices with 4K sectors

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Andres Freund
Hi, On 2021-12-08 03:04:23 +0100, Tomas Vondra wrote: > On 12/8/21 02:54, Bharath Rupireddy wrote: > >> I'm not sure about adding it to control data, though. That doesn't seem > >> like a very good match for something that's mostly for monitoring. > > > > Having it in the control data file

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Julien Rouhaud
Hi, On Fri, Jan 28, 2022 at 08:21:52PM +0530, Bharath Rupireddy wrote: > > I don't think we need to change pg_upgrade's ControlData controldata; > structure as the information may not be needed there and the while > loop there specifically parses/searches for the required > pg_controldata output

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Bharath Rupireddy
On Fri, Jan 28, 2022 at 2:20 PM Julien Rouhaud wrote: > PG_CONTROL_VERSION is different from catversion. You should update it in this > patch. My bad. Updated it. > But Horiguchi-san was also mentioning that pg_upgrade/controldata.c needs some > modifications if you change the format (thus the

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Julien Rouhaud
Hi, On Fri, Jan 28, 2022 at 01:49:19PM +0530, Bharath Rupireddy wrote: > On Thu, Jan 27, 2022 at 3:23 PM Sergey Dudoladov > wrote: > > > Here's v2, rebased onto the latest master. > > > > I've reviewed this patch. The patch builds against the master (commit > >

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Bharath Rupireddy
On Thu, Jan 27, 2022 at 3:23 PM Sergey Dudoladov wrote: > > Here's v2, rebased onto the latest master. > > I've reviewed this patch. The patch builds against the master (commit > e9d4001ec592bcc9a3332547cb1b0211e8794f38) and passes all the tests. > The patch does what it intends to do, namely

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-27 Thread Bharath Rupireddy
On Fri, Jan 28, 2022 at 8:54 AM Julien Rouhaud wrote: > > Hi, > > On Fri, Jan 28, 2022 at 08:17:39AM +0530, Bharath Rupireddy wrote: > > > > Thanks all for the comments. pg_stat_progress_checkpoint is being > > discussed in another thread [1]. > > > > [1] > >

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-27 Thread Julien Rouhaud
Hi, On Fri, Jan 28, 2022 at 08:17:39AM +0530, Bharath Rupireddy wrote: > > Thanks all for the comments. pg_stat_progress_checkpoint is being > discussed in another thread [1]. > > [1] >

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-27 Thread Bharath Rupireddy
On Fri, Jan 28, 2022 at 7:30 AM Julien Rouhaud wrote: > > Hi, > > On Fri, Jan 28, 2022 at 10:38:53AM +0900, Kyotaro Horiguchi wrote: > > > > I'd like to see the PID of the triggering process, but it is really > > not a information suitable in the control file... > > Yes that's something I would

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-27 Thread Kyotaro Horiguchi
At Fri, 28 Jan 2022 10:00:46 +0800, Julien Rouhaud wrote in > Hi, > > On Fri, Jan 28, 2022 at 10:38:53AM +0900, Kyotaro Horiguchi wrote: > > > > I'd like to see the PID of the triggering process, but it is really > > not a information suitable in the control file... > > Yes that's something

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-27 Thread Kyotaro Horiguchi
At Fri, 28 Jan 2022 10:41:28 +0900 (JST), Kyotaro Horiguchi wrote in > Sorry, the last message lacks one citation. > > At Thu, 27 Jan 2022 19:09:29 +0800, Julien Rouhaud wrote > in > > On Thu, Jan 27, 2022 at 06:56:57PM +0800, Julien Rouhaud wrote: > > > > > > What it's showing is the

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-27 Thread Julien Rouhaud
Hi, On Fri, Jan 28, 2022 at 10:38:53AM +0900, Kyotaro Horiguchi wrote: > > I'd like to see the PID of the triggering process, but it is really > not a information suitable in the control file... Yes that's something I would like too. But even if the PIDs could be store, I don't think that

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-27 Thread Kyotaro Horiguchi
Sorry, the last message lacks one citation. At Thu, 27 Jan 2022 19:09:29 +0800, Julien Rouhaud wrote in > On Thu, Jan 27, 2022 at 06:56:57PM +0800, Julien Rouhaud wrote: > > > > What it's showing is the "currently ongoing checkpoint or last completed > > checkpoint" kind. > > Ah after double

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-27 Thread Kyotaro Horiguchi
At Thu, 27 Jan 2022 19:09:29 +0800, Julien Rouhaud wrote in > On Thu, Jan 27, 2022 at 06:56:57PM +0800, Julien Rouhaud wrote: > > > > What it's showing is the "currently ongoing checkpoint or last completed > > checkpoint" kind. > > Ah after double checking I see it's storing the information

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-27 Thread Julien Rouhaud
On Thu, Jan 27, 2022 at 06:56:57PM +0800, Julien Rouhaud wrote: > > What it's showing is the "currently ongoing checkpoint or last completed > checkpoint" kind. Ah after double checking I see it's storing the information *after* the checkpoint completion, so it's indeed the last completed

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-27 Thread Julien Rouhaud
Hi, On Thu, Jan 27, 2022 at 10:53:32AM +0100, Sergey Dudoladov wrote: > Hi all, > > > Here's v2, rebased onto the latest master. > > I've reviewed this patch. The patch builds against the master (commit > e9d4001ec592bcc9a3332547cb1b0211e8794f38) and passes all the tests. > The patch does what

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-27 Thread Sergey Dudoladov
Hi all, > Here's v2, rebased onto the latest master. I've reviewed this patch. The patch builds against the master (commit e9d4001ec592bcc9a3332547cb1b0211e8794f38) and passes all the tests. The patch does what it intends to do, namely store the kind of the last checkpoint in the control file

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2021-12-27 Thread Bharath Rupireddy
On Thu, Dec 9, 2021 at 12:08 PM Bharath Rupireddy wrote: > > On Wed, Dec 8, 2021 at 7:43 AM Bharath Rupireddy > wrote: > > > > On Wed, Dec 8, 2021 at 7:34 AM Tomas Vondra > > wrote: > > > >> I agree it might be useful to provide information about the nature of > > > >> the checkpoint, and

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2021-12-08 Thread Bharath Rupireddy
On Wed, Dec 8, 2021 at 7:43 AM Bharath Rupireddy wrote: > > On Wed, Dec 8, 2021 at 7:34 AM Tomas Vondra > wrote: > > >> I agree it might be useful to provide information about the nature of > > >> the checkpoint, and perhaps even PID of the backend that triggered it > > >> (although that may be

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2021-12-07 Thread Tomas Vondra
On 12/8/21 02:54, Bharath Rupireddy wrote: > On Wed, Dec 8, 2021 at 4:07 AM Justin Pryzby wrote: >>> I'd bet squashing all of this into a single string (not really a flag) >>> will just mean people will have to parse it, etc. Keeping individual >>> boolean flags (or even separate string

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2021-12-07 Thread Bharath Rupireddy
On Wed, Dec 8, 2021 at 7:34 AM Tomas Vondra wrote: > >> I agree it might be useful to provide information about the nature of > >> the checkpoint, and perhaps even PID of the backend that triggered it > >> (although that may be tricky, if the backend terminates). > > > > Thanks. I agree to have

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2021-12-07 Thread Tomas Vondra
On 12/8/21 02:54, Bharath Rupireddy wrote: > On Wed, Dec 8, 2021 at 3:48 AM Tomas Vondra > wrote: >> >> On 12/7/21 15:36, Bharath Rupireddy wrote: >>> Hi, >>> >>> Currently one can know the kind of on-going/last checkpoint (shutdown, >>> end-of-recovery, immediate, force etc.) only via server

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2021-12-07 Thread Bharath Rupireddy
On Wed, Dec 8, 2021 at 4:07 AM Justin Pryzby wrote: > > I'd bet squashing all of this into a single string (not really a flag) > > will just mean people will have to parse it, etc. Keeping individual > > boolean flags (or even separate string fields) would be better, I guess. > > Since the size

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2021-12-07 Thread Bharath Rupireddy
On Wed, Dec 8, 2021 at 3:48 AM Tomas Vondra wrote: > > On 12/7/21 15:36, Bharath Rupireddy wrote: > > Hi, > > > > Currently one can know the kind of on-going/last checkpoint (shutdown, > > end-of-recovery, immediate, force etc.) only via server logs that to > > when log_checkpoints GUC is on. At

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2021-12-07 Thread Justin Pryzby
On Tue, Dec 07, 2021 at 11:18:37PM +0100, Tomas Vondra wrote: > On 12/7/21 15:36, Bharath Rupireddy wrote: > > Currently one can know the kind of on-going/last checkpoint (shutdown, > > end-of-recovery, immediate, force etc.) only via server logs that to > > when log_checkpoints GUC is on. > >

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2021-12-07 Thread Tomas Vondra
On 12/7/21 15:36, Bharath Rupireddy wrote: > Hi, > > Currently one can know the kind of on-going/last checkpoint (shutdown, > end-of-recovery, immediate, force etc.) only via server logs that to > when log_checkpoints GUC is on. At times, the users/service layer > components would want to know

Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2021-12-07 Thread Bharath Rupireddy
Hi, Currently one can know the kind of on-going/last checkpoint (shutdown, end-of-recovery, immediate, force etc.) only via server logs that to when log_checkpoints GUC is on. At times, the users/service layer components would want to know the kind of checkpoint (along with other checkpoint