Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

[Satyanarayana Narlapuram]

Thank you! Got it. 

-Original Message-
From: Stephen Frost [mailto:sfr...@snowman.net] 
Sent: Monday, September 25, 2017 10:57 AM
To: Magnus Hagander <mag...@hagander.net>
Cc: Satyanarayana Narlapuram <satyanarayana.narlapu...@microsoft.com>; 
PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Reading backup label file for checkpoint and redo 
location during crash recovery

* Magnus Hagander (mag...@hagander.net) wrote:
> On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost <sfr...@snowman.net> wrote:
> > * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote:
> > > During crash recovery, last checkpoint record information is 
> > > obtained
> > from the backup label if present, instead of getting it from the 
> > control file. This behavior is causing PostgreSQL database cluster 
> > not to come up until the backup label file is deleted (as the error message 
> > says).
> > >
> > > if (checkPoint.redo < checkPointLoc)
> > >   {
> > >  if (!ReadRecord(xlogreader,
> > checkPoint.redo, LOG, false))
> > > ereport(FATAL,
> > >   (errmsg("could 
> > > not
> > find redo location referenced by checkpoint record"),
> > >   errhint("If you 
> > > are
> > not restoring from a backup, try removing the file 
> > \"%s/backup_label\".", DataDir)));
> > >   }
> > >
> > > If we are recovering from a dump file, reading from the backup 
> > > label
> > files makes sense as the control file could be archived after a few 
> > checkpoints. But this is not the case for crash recovery, and is 
> > always safe to read the checkpoint record information from the control file.
> > > Is this behavior kept this way as there is no clear way to 
> > > distinguish
> > between the recovery from the dump and the regular crash recovery?
> >
> > This is why the exclusive backup method has been deprecated in PG10 
> > in favor of the non-exclusive backup method, which avoids this by 
> > not creating a backup label file (it's up to the backup software to 
> > store the necessary information and create the file for use during 
> > recovery).
> 
> Actally, it was deprecated already in 9.6, not just 10.

Whoops, right.  Thanks for the clarification. :)

Stephen


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

[Stephen Frost]

* Magnus Hagander (mag...@hagander.net) wrote:
> On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost  wrote:
> > * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote:
> > > During crash recovery, last checkpoint record information is obtained
> > from the backup label if present, instead of getting it from the control
> > file. This behavior is causing PostgreSQL database cluster not to come up
> > until the backup label file is deleted (as the error message says).
> > >
> > > if (checkPoint.redo < checkPointLoc)
> > >   {
> > >  if (!ReadRecord(xlogreader,
> > checkPoint.redo, LOG, false))
> > > ereport(FATAL,
> > >   (errmsg("could not
> > find redo location referenced by checkpoint record"),
> > >   errhint("If you are
> > not restoring from a backup, try removing the file \"%s/backup_label\".",
> > DataDir)));
> > >   }
> > >
> > > If we are recovering from a dump file, reading from the backup label
> > files makes sense as the control file could be archived after a few
> > checkpoints. But this is not the case for crash recovery, and is always
> > safe to read the checkpoint record information from the control file.
> > > Is this behavior kept this way as there is no clear way to distinguish
> > between the recovery from the dump and the regular crash recovery?
> >
> > This is why the exclusive backup method has been deprecated in PG10 in
> > favor of the non-exclusive backup method, which avoids this by not
> > creating a backup label file (it's up to the backup software to store
> > the necessary information and create the file for use during recovery).
> 
> Actally, it was deprecated already in 9.6, not just 10.

Whoops, right.  Thanks for the clarification. :)

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

[Magnus Hagander]

On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost  wrote:

> Greetings Satya,
>
> * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote:
> > During crash recovery, last checkpoint record information is obtained
> from the backup label if present, instead of getting it from the control
> file. This behavior is causing PostgreSQL database cluster not to come up
> until the backup label file is deleted (as the error message says).
> >
> > if (checkPoint.redo < checkPointLoc)
> >   {
> >  if (!ReadRecord(xlogreader,
> checkPoint.redo, LOG, false))
> > ereport(FATAL,
> >   (errmsg("could not
> find redo location referenced by checkpoint record"),
> >   errhint("If you are
> not restoring from a backup, try removing the file \"%s/backup_label\".",
> DataDir)));
> >   }
> >
> > If we are recovering from a dump file, reading from the backup label
> files makes sense as the control file could be archived after a few
> checkpoints. But this is not the case for crash recovery, and is always
> safe to read the checkpoint record information from the control file.
> > Is this behavior kept this way as there is no clear way to distinguish
> between the recovery from the dump and the regular crash recovery?
>
> This is why the exclusive backup method has been deprecated in PG10 in
> favor of the non-exclusive backup method, which avoids this by not
> creating a backup label file (it's up to the backup software to store
> the necessary information and create the file for use during recovery).
>


Actally, it was deprecated already in 9.6, not just 10.


-- 
 Magnus Hagander
 Me: https://www.hagander.net/ 
 Work: https://www.redpill-linpro.com/ 

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

[Andres Freund]

On 2017-09-25 13:43:32 -0400, Stephen Frost wrote:
> Greetings Satya,
> 
> * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote:
> > During crash recovery, last checkpoint record information is obtained from 
> > the backup label if present, instead of getting it from the control file. 
> > This behavior is causing PostgreSQL database cluster not to come up until 
> > the backup label file is deleted (as the error message says).
> > 
> > if (checkPoint.redo < checkPointLoc)
> >   {
> >  if (!ReadRecord(xlogreader, checkPoint.redo, 
> > LOG, false))
> > ereport(FATAL,
> >   (errmsg("could not find 
> > redo location referenced by checkpoint record"),
> >   errhint("If you are not 
> > restoring from a backup, try removing the file \"%s/backup_label\".", 
> > DataDir)));
> >   }
> > 
> > If we are recovering from a dump file, reading from the backup label files 
> > makes sense as the control file could be archived after a few checkpoints. 
> > But this is not the case for crash recovery, and is always safe to read the 
> > checkpoint record information from the control file.
> > Is this behavior kept this way as there is no clear way to distinguish 
> > between the recovery from the dump and the regular crash recovery?
> 
> This is why the exclusive backup method has been deprecated in PG10 in
> favor of the non-exclusive backup method, which avoids this by not
> creating a backup label file (it's up to the backup software to store
> the necessary information and create the file for use during recovery).
> 
> Please see:
> 
> https://www.postgresql.org/docs/10/static/continuous-archiving.html
> 
> In particular, section 25.3.3.

Might not be obvious for the more casual contributor:

And even before then, using pg_basebackup / the streaming replication
protocol version of creating a base backup, it was avoided.

Greetings,

Andres Freund


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

[Stephen Frost]

Greetings Satya,

* Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote:
> During crash recovery, last checkpoint record information is obtained from 
> the backup label if present, instead of getting it from the control file. 
> This behavior is causing PostgreSQL database cluster not to come up until the 
> backup label file is deleted (as the error message says).
> 
> if (checkPoint.redo < checkPointLoc)
>   {
>  if (!ReadRecord(xlogreader, checkPoint.redo, 
> LOG, false))
> ereport(FATAL,
>   (errmsg("could not find 
> redo location referenced by checkpoint record"),
>   errhint("If you are not 
> restoring from a backup, try removing the file \"%s/backup_label\".", 
> DataDir)));
>   }
> 
> If we are recovering from a dump file, reading from the backup label files 
> makes sense as the control file could be archived after a few checkpoints. 
> But this is not the case for crash recovery, and is always safe to read the 
> checkpoint record information from the control file.
> Is this behavior kept this way as there is no clear way to distinguish 
> between the recovery from the dump and the regular crash recovery?

This is why the exclusive backup method has been deprecated in PG10 in
favor of the non-exclusive backup method, which avoids this by not
creating a backup label file (it's up to the backup software to store
the necessary information and create the file for use during recovery).

Please see:

https://www.postgresql.org/docs/10/static/continuous-archiving.html

In particular, section 25.3.3.

Thanks!

Stephen


signature.asc
Description: Digital signature