Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

On 12/2/21 12:38, David Steele wrote: On 12/2/21 11:00, Andrew Dunstan wrote: Should we really be getting rid of PostgreSQL::Test::Cluster::backup_fs_hot() ? Agreed, it would be better to update backup_fs_hot() to use exclusive mode and save out backup_label instead. Oops, of course I

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

On 11/30/21 17:26, Tom Lane wrote: "Bossart, Nathan" writes: It looks like the exclusive way has been marked deprecated in all supported versions along with a note that it will eventually be removed. If it's not going to be removed out of fear of breaking backward compatibility, I think the

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

On 1/6/22 20:20, Euler Taveira wrote: On Thu, Jan 6, 2022, at 9:48 PM, Bossart, Nathan wrote: After a quick glance, I didn't see an easy way to hold a session open while the test does other things.  If there isn't one, modifying backup_fs_hot() to work with non-exclusive mode might be more

Re: Possible to go without page headers?

On 2/14/22 16:19, Tom Lane wrote: Chris Cleveland writes: Can I treat pages as just a flat, open 8k buffer and fill them with arbitrary data? No, at least not unless you plan to reimplement much of the WAL mechanism. You do need at least an LSN in the right place. I kinda doubt that you can

Re: Commitfest 2022-03 One Week in. 3 Commits 213 Patches Remaining

On 3/9/22 13:38, Greg Stark wrote: Is there anything I can do to get committers assigned to these patches? Should I do a round-robin style assignment for any of these? I don't think this is a good idea. Committers pick the patches they are going to commit. What prefer to do is bump any

Re: Document what is essential and undocumented in pg_basebackup

On 3/9/22 13:46, Stephen Frost wrote: I don't think it is as reasonable to say, effectively, that you learn what the irreducibly essential steps of an online base backup are by reading the source of pg_basebackup, and then intuiting which of the details you find there are the essential ones

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

On 3/9/22 18:06, Nathan Bossart wrote: On Wed, Mar 09, 2022 at 06:11:19PM -0500, Chapman Flack wrote: I think the listitem In the same connection as before, issue the command: SELECT * FROM pg_backup_stop(true); would be clearer if it used named-parameter form, wait_for_archive => true.

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

On 3/8/22 14:01, Nathan Bossart wrote: On Wed, Mar 02, 2022 at 02:23:51PM -0500, Chapman Flack wrote: I did not notice this earlier (sorry), but there seems to remain in backup.sgml a programlisting example that shows a psql invocation for pg_backup_start, then a tar command, then another psql

Re: Allow root ownership of client certificate key

Hi Tom, On 1/18/22 14:41, Tom Lane wrote: David Steele writes: [ client-key-perm-002.patch ] I took a quick look at this and agree with the proposed behavior change, but also with your self-criticisms: We may want to do the same on the server side to make the code blocks look more similar

Re: Commitfest manager for 2022-03

Hi Greg, On 2/25/22 12:39, Greg Stark wrote: I would like to volunteer. On Fri, 25 Feb 2022 at 05:31, Julien Rouhaud wrote: Is there any volunteer? Since I've done the March CF for the last seven years, I thought it would be good if I chimed in. I've been agonizing a bit because I have

Re: Allow root ownership of client certificate key

On 2/28/22 13:20, Tom Lane wrote: David Steele writes: [ client-key-perm-003.patch ] Pushed with a bit of copy-editing of the comments. Thank you! Any thoughts on back-patching at least the client portion of this? Probably hard to argue that it's a bug, but it is certainly painful

Re: adding 'zstd' as a compression algorithm

On 2/15/22 16:10, Peter Geoghegan wrote: On Tue, Feb 15, 2022 at 12:00 PM Robert Haas wrote: I'm not sure I completely follow this. There are cases where we use compression algorithms for internal purposes, and we can change the defaults without users knowing or caring. For example, we could

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

On 3/1/22 11:32, Nathan Bossart wrote: On Tue, Mar 01, 2022 at 11:09:13AM -0500, Chapman Flack wrote: On 03/01/22 09:44, David Steele wrote: Personally, I am in favor of removing it. We change/rename functions/tables/views when we need to, and this happens in almost every release

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

On 2/28/22 23:51, Nathan Bossart wrote: On Sat, Feb 26, 2022 at 02:06:14PM -0800, Nathan Bossart wrote: On Sat, Feb 26, 2022 at 04:48:52PM +, Chapman Flack wrote: Assuming the value is false, so no error is thrown, is it practical to determine from flinfo->fn_expr whether the value was

Re: Allow root ownership of client certificate key

On 3/2/22 08:40, Tom Lane wrote: Chris Bandy writes: On 3/1/22 3:15 AM, Tom Lane wrote: Anyway, I'd be happier about back-patching if we could document actual requests to make it work like the server side does. PGO runs PostgreSQL 10 through 14 in Kubernetes, and we have to work around

Re: pg_stop_backup() v2 incorrectly marked as proretset

On 3/2/22 11:04, Chapman Flack wrote: On 03/02/22 02:46, Michael Paquier wrote: system function marked as proretset while it builds and returns only one record. And this is a popular one: pg_stop_backup(), labelled v2. I had just recently noticed that while reviewing [0], but shrugged, as I

Re: ubsan

On 3/23/22 16:55, Andres Freund wrote: It's particularly impressive that the cost of running with ASAN is *so* much lower than valgrind. On my workstation a check-world with -fsanitize=alignment,undefined,address takes 3min17s, vs 1min10s or so without -fsanitize. Not something to always use,

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

On 3/28/22 10:09 PM, Nathan Bossart wrote: On Mon, Mar 28, 2022 at 04:30:27PM -0400, Stephen Frost wrote: On a once-over of the rest of the code, I definitely like how much we're able to simplify things in this area and remove various hacks in things like pg_basebackup and pg_rewind where we

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

On 4/4/22 11:42 AM, Nathan Bossart wrote: I noticed a couple of other things that can be removed. Since we no longer wait on exclusive backup mode during smart shutdown, we can change connsAllowed (in postmaster.c) to a boolean and remove CAC_SUPERUSER. We can also remove a couple of related

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

On 4/5/22 11:25 AM, Stephen Frost wrote: Please find attached an updated patch + commit message. Mostly, I just went through and did a bit more in terms of updating the documentation and improving the comments (there were some places that were still worrying about the chance of a 'stray'

Re: is the base backup protocol used by out-of-core tools?

On 2/8/22 12:39, Robert Haas wrote: On Tue, Feb 8, 2022 at 1:03 PM Justin Pryzby wrote: I created this and added that for visibility and so it's not forgotten. ISTM that could be done post-freeze, although I don't know if that's useful or important.

Re: Allow root ownership of client certificate key

On 1/18/22 15:41, Tom Lane wrote: David Steele writes: I took a quick look at this and agree with the proposed behavior change, but also with your self-criticisms: We may want to do the same on the server side to make the code blocks look more similar. Also, on the server side the S_ISREG

Re: trying again to get incremental backup

Hi Robert, On 8/30/23 10:49, Robert Haas wrote: In the limited time that I've had to work on this project lately, I've been trying to come up with a test case for this feature -- and since I've gotten completely stuck, I thought it might be time to post and see if anyone else has a better idea.

Re: New compiler warning

On 8/30/23 08:10, Aleksander Alekseev wrote: I am seeing a new gcc 12.2.0 compiler warning from src/backend/commands/sequence.c: Yep, the compiler is just not smart enough to derive that this actually is not going to happen. Here is a proposed fix. Here's an alternate way to deal with

Add const qualifiers

Hackers, I noticed that there was a mismatch between the const qualifiers for excludeDirContents in src/backend/backup/backup/basebackup.c and src/bin/pg_rewind/file_map.c and that led me to use ^static const.*\*.*= to do a quick search for similar cases. I think at the least we should make

Re: Add const qualifiers

On 9/1/23 11:39, David Steele wrote: Hackers, I noticed that there was a mismatch between the const qualifiers for excludeDirContents in src/backend/backup/backup/basebackup.c and src/bin/pg_rewind/file_map.c and that led me to use ^static const.*\*.*= to do a quick search for similar cases

Re: Rename backup_label to recovery_control

On 10/16/23 00:26, Kyotaro Horiguchi wrote: At Mon, 16 Oct 2023 13:16:42 +0900 (JST), Kyotaro Horiguchi wrote in Just an idea in a slightly different direction, but I'm wondering if we can simply merge the content of backup_label into control file. The file is 8192 bytes long, yet only 256

Re: Rename backup_label to recovery_control

On 10/16/23 10:19, Robert Haas wrote: On Sat, Oct 14, 2023 at 2:22 PM David Steele wrote: I was recently discussing the complexities of dealing with pg_control and backup_label with some hackers at PGConf NYC, when David Christensen commented that backup_label was not a very good name since

Re: The danger of deleting backup_label

On 10/16/23 10:55, Robert Haas wrote: On Sat, Oct 14, 2023 at 11:33 AM David Steele wrote: All of this is fixable in HEAD, but seems incredibly dangerous to back patch. Even so, I have attached the patch in case somebody sees an opportunity that I do not. I really do not think we should

Re: The danger of deleting backup_label

On 10/16/23 12:25, Robert Haas wrote: On Mon, Oct 16, 2023 at 11:45 AM David Steele wrote: Hmmm, the reason to back patch this is that it would fix [1], which sure looks like a problem to me even if it is not a "bug". We can certainly require backup software to retry pg_con

Re: Rename backup_label to recovery_control

On 10/16/23 12:12, Robert Haas wrote: On Mon, Oct 16, 2023 at 12:06 PM Michael Banck wrote: Not sure what to do about this, but as people/companies start moving to 15, I am afraid we will get people complaining about this. I think having exclusive mode still be the default for

Re: Rename backup_label to recovery_control

On 10/16/23 12:06, Michael Banck wrote: On Mon, Oct 16, 2023 at 11:15:53AM -0400, David Steele wrote: On 10/16/23 10:19, Robert Haas wrote: We got rid of exclusive backup mode. We replaced pg_start_backup with pg_backup_start. I do think this was an improvement. For example it allows us

Re: The danger of deleting backup_label

On 10/16/23 15:06, Robert Haas wrote: On Mon, Oct 16, 2023 at 1:00 PM David Steele wrote: After some agonizing (we hope) they decide to delete backup_label and, wow, it just works! So now they merrily go on their way with a corrupted cluster. They also remember for the next time that deleting

Re: trying again to get incremental backup

On 10/19/23 12:05, Robert Haas wrote: On Wed, Oct 4, 2023 at 4:08 PM Robert Haas wrote: Clearly there's a good amount of stuff to sort out here, but we've still got quite a bit of time left before feature freeze so I'd like to have a go at it. Please let me know your thoughts, if you have any.

Remove dead code in pg_ctl.c

Hackers, It looks like this code was missed in 39969e2a when exclusive backup was removed. Regards, -Daviddiff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c index 807d7023a99..4099d240e03 100644 --- a/src/bin/pg_ctl/pg_ctl.c +++ b/src/bin/pg_ctl/pg_ctl.c @@ -96,7 +96,6 @@ static

Re: Add recovery to pg_control and remove backup_label

On 10/26/23 17:27, David G. Johnston wrote: On Thu, Oct 26, 2023 at 2:02 PM David Steele <mailto:da...@pgmasters.net>> wrote: Are we planning on dealing with torn writes in the back branches in some way or are we just throwing in the towel and saying the old method is too error-prone

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

On 10/27/23 03:22, Michael Paquier wrote: On Mon, Oct 16, 2023 at 02:54:35PM +0900, Michael Paquier wrote: On Sat, Oct 14, 2023 at 03:45:33PM -0400, David Steele wrote: On 9/28/23 19:59, Michael Paquier wrote: Another idea I had was to force the creation of recovery.signal by pg_basebackup

Re: Remove dead code in pg_ctl.c

On 10/25/23 17:30, Nathan Bossart wrote: On Wed, Oct 25, 2023 at 03:02:01PM -0500, Nathan Bossart wrote: On Wed, Oct 25, 2023 at 02:53:31PM -0400, David Steele wrote: It looks like this code was missed in 39969e2a when exclusive backup was removed. Indeed. I'll plan on committing

Re: Add recovery to pg_control and remove backup_label

On 10/27/23 13:45, David G. Johnston wrote: Let me modify that to make it a bit more clear, I actually wouldn't care if pg_backup_end outputs an entire binary pg_control file as part of the SQL resultset. My proposal would be to, in addition, place in the temporary directory on the server,

Re: Add recovery to pg_control and remove backup_label

Rebased on 151ffcf6.diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml index 8cb24d6ae54..6be8fb902c5 100644 --- a/doc/src/sgml/backup.sgml +++ b/doc/src/sgml/backup.sgml @@ -935,19 +935,20 @@ SELECT * FROM pg_backup_stop(wait_for_archive => true); ready to archive.

Re: Add recovery to pg_control and remove backup_label

On 11/6/23 01:05, Michael Paquier wrote: On Fri, Oct 27, 2023 at 10:10:42AM -0400, David Steele wrote: We are still planning to address this issue in the back branches. FWIW, redesigning the backend code in charge of doing base backups in the back branches is out of scope. Based on a read

Re: Add recovery to pg_control and remove backup_label

On 11/6/23 02:35, Michael Paquier wrote: On Sun, Nov 05, 2023 at 01:45:39PM -0400, David Steele wrote: Rebased on 151ffcf6. I like this patch a lot. Even if the backup_label file is removed, we still have all the debug information from the backup history file, thanks to its LABEL, BACKUP

Add recovery to pg_control and remove backup_label

Hackers, This was originally proposed in [1] but that thread went through a number of different proposals so it seems better to start anew. The basic idea here is to simplify and harden recovery by getting rid of backup_label and storing recovery information directly in pg_control. Instead

Re: The danger of deleting backup_label

On 10/18/23 08:39, Robert Haas wrote: On Tue, Oct 17, 2023 at 4:17 PM David Steele wrote: Given that the above can't be back patched, I'm thinking we don't need backup_label at all going forward. We just write the values we need for recovery into pg_control and return *that* from

Re: Improving Physical Backup/Restore within the Low Level API

On 10/17/23 14:28, Robert Haas wrote: On Mon, Oct 16, 2023 at 5:21 PM David G. Johnston wrote: But no, by default, and probably so far as pg_basebackup is concerned, a server crash during backup results in requiring outside intervention in order to get the server to restart. Others may

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

On 9/27/23 23:58, Kyotaro Horiguchi wrote: At Fri, 10 Mar 2023 15:59:04 +0900, Michael Paquier wrote in My apologies for the long message, but this deserves some attention, IMHO. So, any thoughts? Sorry for being late. However, I agree with David's concern regarding the unnecessary

The danger of deleting backup_label

Hackers, While reading through [1] I saw there were two instances where backup_label was removed to achieve a "successful" restore. This might work on trivial test restores but is an invitation to (silent) disaster in a production environment where the checkpoint stored in backup_label is

Re: Add const qualifiers

On 9/26/23 06:34, Peter Eisentraut wrote: On 09.09.23 21:03, David Steele wrote: On 9/1/23 11:39, David Steele wrote: Hackers, I noticed that there was a mismatch between the const qualifiers for excludeDirContents in src/backend/backup/backup/basebackup.c and src/bin/pg_rewind/file_map.c

Re: how to manage Cirrus on personal repository

On 9/29/23 18:02, Thomas Munro wrote: On Sat, Sep 30, 2023 at 3:35 AM Nazir Bilal Yavuz wrote: On Fri, 29 Sept 2023 at 13:24, Peter Eisentraut wrote: I have a private repository on GitHub where I park PostgreSQL development work. I also have Cirrus activated on that repository, to build

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

On 10/12/23 09:58, David Steele wrote: On Thu, Oct 12, 2023 at 12:25:34PM +1300, Thomas Munro wrote: I'm planning to push 0002 (retries in frontend programs, which is where this thread began) and 0004 (add missing locks to SQL functions), including back-patches as far as 12, in a day or so

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

On 10/11/23 21:10, Michael Paquier wrote: On Thu, Oct 12, 2023 at 12:25:34PM +1300, Thomas Munro wrote: I'm planning to push 0002 (retries in frontend programs, which is where this thread began) and 0004 (add missing locks to SQL functions), including back-patches as far as 12, in a day or

Re: The danger of deleting backup_label

Hi Thomas, On 10/11/23 18:10, Thomas Munro wrote: Even though I spent a whole bunch of time trying to figure out how to make concurrent reads of the control file sufficiently atomic for backups (pg_basebackup and low level filesystem tools), and we explored multiple avenues with varying

Re: The danger of deleting backup_label

On 10/11/23 18:22, Michael Paquier wrote: On Tue, Oct 10, 2023 at 05:06:45PM -0400, David Steele wrote: That fails because there is a check to make sure the checkpoint is valid when pg_control is loaded. Another possibility is to use a special LSN like we use for unlogged tables. Anything

Re: The danger of deleting backup_label

On 10/14/23 11:30, David Steele wrote: On 10/12/23 10:19, David Steele wrote: On 10/11/23 18:10, Thomas Munro wrote: As Stephen mentioned[1], we could perhaps also complain if both backup label and control file exist, and then hint that the user should remove the *control file

Re: Rename backup_label to recovery_control

On 10/18/23 03:07, Peter Eisentraut wrote: On 16.10.23 17:15, David Steele wrote: I also do wonder with recovery_control is really a better name. Maybe I just have backup_label too firmly stuck in my head, but is what that file does really best described as recovery control? I'm not so sure

Re: The danger of deleting backup_label

On 10/17/23 22:13, Kyotaro Horiguchi wrote: At Tue, 17 Oct 2023 16:16:42 -0400, David Steele wrote in Given that the above can't be back patched, I'm thinking we don't need backup_label at all going forward. We just write the values we need for recovery into pg_control and return *that* from

Re: The danger of deleting backup_label

On 10/19/23 10:24, Robert Haas wrote: On Wed, Oct 18, 2023 at 7:15 PM David Steele wrote: (b) be stored someplace else, I don't think the additional fields *need* to be stored anywhere at all, at least not by us. We can provide them as output from pg_backup_stop() and the caller can do

Re: The danger of deleting backup_label

On 10/19/23 10:56, Robert Haas wrote: On Thu, Oct 19, 2023 at 10:43 AM David Steele wrote: What I meant here (but said badly) is that in the case of snapshot backups, the backup_label and tablespace_map will likely need to be stored somewhere off the server since they can't be part

Re: The danger of deleting backup_label

On 9/28/23 22:30, Michael Paquier wrote: On Thu, Sep 28, 2023 at 05:14:22PM -0400, David Steele wrote: Recovery worked perfectly as long as backup_label was present and failed hard when it was not: LOG: invalid primary checkpoint record PANIC: could not locate a valid checkpoint record

Re: trying again to get incremental backup

On 10/19/23 16:00, Robert Haas wrote: On Thu, Oct 19, 2023 at 3:18 PM David Steele wrote: 0001 looks pretty good to me. The only thing I find a little troublesome is the repeated construction of file names with/without segment numbers in ResetUnloggedRelationsInDbspaceDir(), .e.g

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

On 10/12/23 19:15, Michael Paquier wrote: On Thu, Oct 12, 2023 at 10:41:39AM -0400, David Steele wrote: After some more thought, I think we could massage the "pg_control in backup_label" method into something that could be back patched, with more advanced features (e.g. error on ba

Re: The danger of deleting backup_label

On 10/12/23 10:19, David Steele wrote: On 10/11/23 18:10, Thomas Munro wrote: As Stephen mentioned[1], we could perhaps also complain if both backup label and control file exist, and then hint that the user should remove the *control file* (not the backup label!).  I had originally suggested

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

On 10/13/23 10:40, David Steele wrote: On 10/12/23 19:15, Michael Paquier wrote: On Thu, Oct 12, 2023 at 10:41:39AM -0400, David Steele wrote: After some more thought, I think we could massage the "pg_control in backup_label" method into something that could be back patched,

Re: Add trailing commas to enum definitions

On 10/23/23 17:04, Tom Lane wrote: Nathan Bossart writes: From a long-term perspective, I think standardizing on the trailing comma style will actually improve git-blame because patches won't need to add a comma to the previous line when adding a value. Yeah, that's a good point. I had

Re: trying again to get incremental backup

On 10/23/23 11:44, Robert Haas wrote: On Fri, Oct 20, 2023 at 11:30 AM David Steele wrote: I don't plan to stand in your way on this feature. I'm reviewing what patches I can out of courtesy and to be sure that nothing adjacent to your work is being affected. My apologies if my reviews

Rename backup_label to recovery_control

Hackers, I was recently discussing the complexities of dealing with pg_control and backup_label with some hackers at PGConf NYC, when David Christensen commented that backup_label was not a very good name since it gives the impression of being informational and therefore something the user

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

On 9/28/23 19:59, Michael Paquier wrote: On Thu, Sep 28, 2023 at 04:23:42PM -0400, David Steele wrote: So overall, +1 for Michael's patch, though I have only read through it and not tested it yet. Reviews, thoughts and opinions are welcome. OK, I have now reviewed and tested the patch

Re: Add recovery to pg_control and remove backup_label

On 11/10/23 00:37, Michael Paquier wrote: On Tue, Nov 07, 2023 at 05:20:27PM +0900, Michael Paquier wrote: On Mon, Nov 06, 2023 at 05:39:02PM -0400, David Steele wrote: I've retested today, and miss the failure. I'll let you know if I see this again. I've done a few more dozen runs

Re: First draft of the PG 15 release notes

ted status in previous release notes, so the new text is: Remove long-deprecated exclusive backup mode (David Steele, Nathan Bossart) That works, thanks! A bit late to this conversation, but +1 from me. -- -David da...@pgmasters.net

Re: removing datlastsysoid

On 5/16/22 9:43 AM, Dave Page wrote: On Thu, 20 Jan 2022 at 14:03, Robert Haas > wrote: On Mon, Jan 17, 2022 at 3:43 PM Tom Lane mailto:t...@sss.pgh.pa.us>> wrote: > +1.  Another reason to get rid of it is that it has nothing to do > with the

Re: removing datlastsysoid

On 5/16/22 10:26 AM, Tom Lane wrote: Dave Page writes: On Mon, 16 May 2022 at 15:06, David Steele wrote: Out solution was to use the constant: #define FirstNormalObjectId 16384 And treat anything below that as a system oid. This constant has not changed in a very long time

Re: removing datlastsysoid

On 5/16/22 11:19 AM, Alvaro Herrera wrote: On 2022-May-16, David Steele wrote: On 5/16/22 10:26 AM, Tom Lane wrote: I think that when we approach the point where the system OID range is saturated, we'll give up the principle of system OIDs being globally unique instead of doing

Re: remove more archiving overhead

On 7/7/22 10:37, Robert Haas wrote: On Thu, Jul 7, 2022 at 10:03 AM Fujii Masao wrote: Thanks for updating the patch. It looks good to me. Barring any objection, I'm thinking to commit it. I don't object, but I just started to wonder whether the need to handle re-archiving of the same file

Re: remove more archiving overhead

On 7/7/22 12:18, Nathan Bossart wrote: On Thu, Jul 07, 2022 at 10:46:23AM -0400, David Steele wrote: There are plenty of ways that already-archived WAL might get archived again and this is just one of them. What are some of the others? I was aware of the case that was fixed in ff9f111

Re: remove more archiving overhead

On 7/7/22 14:22, Nathan Bossart wrote: On Thu, Jul 07, 2022 at 10:51:42AM -0700, Nathan Bossart wrote: +library to ensure that it indeed does not overwrite an existing file. When +a pre-existing file is encountered, the archive library may return +true if the WAL file has identical

Re: Add function to return backup_label and tablespace_map

On 7/7/22 12:43, Fujii Masao wrote: Since an exclusive backup method was dropped in v15, in v15 or later, we need to create backup_label and tablespace_map files from the result of pg_backup_stop() when taking a base backup using low level backup API. One issue when doing this is that; there

Re: Add function to return backup_label and tablespace_map

On 7/8/22 08:22, Julien Rouhaud wrote: On Fri, Jul 8, 2022 at 7:42 PM David Steele wrote: On 7/7/22 12:43, Fujii Masao wrote: Since an exclusive backup method was dropped in v15, in v15 or later, we need to create backup_label and tablespace_map files from the result of pg_backup_stop

Re: Add function to return backup_label and tablespace_map

On 7/8/22 07:53, Bharath Rupireddy wrote: On Fri, Jul 8, 2022 at 5:12 PM David Steele wrote: To enable us to do that more easily, how about adding the pg_backup_label() function that returns backup_label and tablespace_map? I'm thinking to make this function available just after

Re: remove more archiving overhead

On 7/7/22 21:56, Kyotaro Horiguchi wrote: At Thu, 7 Jul 2022 15:07:16 -0700, Nathan Bossart wrote in Here's an updated patch. Thinking RFC'ish, the meaning of "may" and "must" is significant in this description. On the other hand it uses both "may" and "can" but I thinkthat their

Re: Add function to return backup_label and tablespace_map

On 7/8/22 09:10, Christoph Berg wrote: Re: David Steele To enable us to do that more easily, how about adding the pg_backup_label() function that returns backup_label and tablespace_map? I'm thinking to make this function available just after pg_backup_start() finishes I was just wondering

Re: Add function to return backup_label and tablespace_map

On 7/8/22 09:09, David Steele wrote: On 7/8/22 08:22, Julien Rouhaud wrote: On Fri, Jul 8, 2022 at 7:42 PM David Steele wrote: On 7/7/22 12:43, Fujii Masao wrote: Since an exclusive backup method was dropped in v15, in v15 or later, we need to create backup_label and tablespace_map files

Re: remove more archiving overhead

On 7/8/22 12:54, Nathan Bossart wrote: On Fri, Jul 08, 2022 at 08:20:09AM -0400, David Steele wrote: Nathan, I don't see the language about being sure to persist to storage here? It's here: When an archive library encounters a pre-existing file, it may return true if the WAL

Re: Comments referring to pg_start/stop_backup

On 6/28/22 01:00, Kyotaro Horiguchi wrote: At Tue, 28 Jun 2022 13:41:58 +0900, Michael Paquier wrote in Hi all, While browsing through the recent changes with the base backup APIs, I have noticed that a couple of comments did not get the renaming of the SQL functions to pg_backup_start/stop,

Re: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?

On 7/25/22 22:49, Kyotaro Horiguchi wrote: At Mon, 25 Jul 2022 14:21:38 +0530, Bharath Rupireddy wrote in Hm. I think we must take this opportunity to clean it up. You are right, we don't need to parse the label file contents (just like we used to do previously after reading it from the file)

Re: Race between KeepFileRestoredFromArchive() and restartpoint

Hi Noah, On 6/19/21 16:39, Noah Misch wrote: On Tue, Feb 02, 2021 at 07:14:16AM -0800, Noah Misch wrote: Recycling and preallocation are wasteful during archive recovery, because KeepFileRestoredFromArchive() unlinks every entry in its path. I propose to fix the race by adding an XLogCtl flag

Re: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?

On 7/26/22 07:59, Bharath Rupireddy wrote: On Tue, Jul 26, 2022 at 5:22 PM David Steele wrote: On 7/25/22 22:49, Kyotaro Horiguchi wrote: At Mon, 25 Jul 2022 14:21:38 +0530, Bharath Rupireddy wrote in Hm. I think we must take this opportunity to clean it up. You are right, we don't

Issue with recovery_target = 'immediate'

Hackers, We have noticed an issue when performing recovery with recovery_target = 'immediate' when the latest timeline cannot be replayed to from the backup (current) timeline. First, create two backups: $ pgbackrest --stanza=demo --type=full --start-fast backup $ pgbackrest --stanza=demo

Re: Issue with recovery_target = 'immediate'

On 7/14/22 04:26, Kyotaro Horiguchi wrote: At Wed, 13 Jul 2022 14:41:40 -0400, David Steele wrote in While it is certainly true that timeline 2 cannot be replayed to from timeline 1, it should not matter for an immediate recovery that stops at consistency. No timeline switch will occur until

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

On 5/3/22 17:17, Stephen Frost wrote: * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: The pg_walcleaner tool isn't intrusive in the sense that it doesn't delete the WAL files that are required for the server to come up (as it checks for the checkpoint redo WAL file), apart

Re: remove more archiving overhead

On 8/2/22 01:02, Nathan Bossart wrote: On Sat, Jul 30, 2022 at 11:51:56PM -0700, Noah Misch wrote: Inviting the administrator to resolve things is more dangerous than just returning true. I recommend making this text more opinionated and simpler: libraries must return true. Alternately, if

Re: remove more archiving overhead

On 9/19/22 07:39, Noah Misch wrote: On Mon, Sep 19, 2022 at 06:08:29AM -0400, Peter Eisentraut wrote: On 18.09.22 09:13, Noah Misch wrote: This documentation change only covers archive_library. How are users of archive_command supposed to handle this? I believe users of archive_command

Re: Race between KeepFileRestoredFromArchive() and restartpoint

On 8/2/22 10:37, Noah Misch wrote: On Tue, Aug 02, 2022 at 10:14:22AM -0400, David Steele wrote: On 7/31/22 02:17, Noah Misch wrote: On Tue, Jul 26, 2022 at 07:21:29AM -0400, David Steele wrote: On 6/19/21 16:39, Noah Misch wrote: On Tue, Feb 02, 2021 at 07:14:16AM -0800, Noah Misch wrote

Re: Race between KeepFileRestoredFromArchive() and restartpoint

On 7/31/22 02:17, Noah Misch wrote: On Tue, Jul 26, 2022 at 07:21:29AM -0400, David Steele wrote: On 6/19/21 16:39, Noah Misch wrote: On Tue, Feb 02, 2021 at 07:14:16AM -0800, Noah Misch wrote: Recycling and preallocation are wasteful during archive recovery, because

Re: Race between KeepFileRestoredFromArchive() and restartpoint

On 8/4/22 04:06, Kyotaro Horiguchi wrote: At Wed, 3 Aug 2022 23:24:56 -0700, Noah Misch wrote in I think in this case a HINT might be sufficient to at least keep people from wasting time tracking down a problem that has already been fixed. Here's a patch to add that HINT. I figure it's

Re: moving basebackup code to its own directory

On 8/9/22 12:12, Magnus Hagander wrote: On Tue, Aug 9, 2022 at 6:08 PM Robert Haas > wrote: Hi, I was thinking that it might make sense, to reduce clutter, to move *backup*.c from src/backend/replication to a new directory, perhaps

Re: moving basebackup code to its own directory

On 8/9/22 12:34, Robert Haas wrote: On Tue, Aug 9, 2022 at 12:12 PM Magnus Hagander wrote: Anyway, I have no objection. If there'd been that many files, or plans to have it, in the beginning we probably would've put them in replication/basebackup or something like that from the beginning.

Re: moving basebackup code to its own directory

On 8/9/22 13:32, Robert Haas wrote: On Tue, Aug 9, 2022 at 12:43 PM Magnus Hagander wrote: So maybe src/backend/backup? Or is that too grandiose for the amount of stuff we have here? +1 for src/backend/backup. I'd also be happy to see the start/stop code move here at some point. Yeah,

Re: moving basebackup code to its own directory

On 8/9/22 14:40, Justin Pryzby wrote: On Tue, Aug 09, 2022 at 01:32:49PM -0400, Robert Haas wrote: On Tue, Aug 9, 2022 at 12:43 PM Magnus Hagander wrote: So maybe src/backend/backup? Or is that too grandiose for the amount of stuff we have here? +1 for src/backend/backup. I'd also be happy

Possible regression setting GUCs on \connect

Hackers, I have been updating pgAudit for PG16 and ran into the following issue in the regression tests: \connect - user1 WARNING: permission denied to set parameter "pgaudit.log_level" This happens after switching back and forth a few times between the current user when the regression

Re: Possible regression setting GUCs on \connect

On 4/27/23 19:13, Tom Lane wrote: David Steele writes: I have been updating pgAudit for PG16 and ran into the following issue in the regression tests: \connect - user1 WARNING: permission denied to set parameter "pgaudit.log_level" This happens after switching back and forth a

<    3   4   5   6   7   8   9   >