Re: Blank archive_command

2022-01-17 Thread Tom Lane
Robert Haas writes: > It might be nice to do something about the fact that you can't change > archive_mode without a server restart, though. I suspect we had a good > reason for that limitation from an engineering perspective, but from a > user perspective, it sucks pretty hard. Agreed. I don't

Re: Blank archive_command

2022-01-17 Thread Robert Haas
On Mon, Jan 17, 2022 at 10:53 AM Bharath Rupireddy wrote: > This parameter can only be set in the postgresql.conf file or on the > server command line. It is ignored unless archive_mode was enabled at > server start. If archive_command is an empty string (the default) > while

Re: Blank archive_command

2022-01-17 Thread Bharath Rupireddy
On Mon, Jan 17, 2022 at 9:05 PM Bharath Rupireddy wrote: > > +1 to not create any GUC setting dependencies. Let's leave the > responsibility of setting appropriate archive_command to the archiving > handlers outside postgres. FWIW, having a note in the archive_command > GUC definit

Re: Blank archive_command

2022-01-17 Thread Thom Brown
On Mon, 17 Jan 2022 at 15:25, Bharath Rupireddy wrote: > > On Mon, Jan 17, 2022 at 8:14 PM Thom Brown wrote: > > > > Hi, > > > > Should archive_command being blank when archiving is enabled result in > > a fatal error? This doesn't even produce a warnin

Re: Blank archive_command

2022-01-17 Thread Bharath Rupireddy
On Mon, Jan 17, 2022 at 9:02 PM Tom Lane wrote: > > Bharath Rupireddy writes: > > However, a reasonable thing to do is to > > emit a WARNING or ERROR-out when archive_command is set to null in > > it's check_archive_command when archive_mode is on? > > We have

Re: Blank archive_command

2022-01-17 Thread Tom Lane
Bharath Rupireddy writes: > However, a reasonable thing to do is to > emit a WARNING or ERROR-out when archive_command is set to null in > it's check_archive_command when archive_mode is on? We have been burned badly in the past by attempts to do that sort of thing (ie, make behavi

Re: Blank archive_command

2022-01-17 Thread Bharath Rupireddy
On Mon, Jan 17, 2022 at 8:14 PM Thom Brown wrote: > > Hi, > > Should archive_command being blank when archiving is enabled result in > a fatal error? This doesn't even produce a warning when restarting, > just an entry in the log when it goes to archive a WAL se

Blank archive_command

2022-01-17 Thread Thom Brown
Hi, Should archive_command being blank when archiving is enabled result in a fatal error? This doesn't even produce a warning when restarting, just an entry in the log when it goes to archive a WAL segment, and finds the archive_command is empty. Is there a valid scenario where someone would

Re: wait event and archive_command

2021-11-21 Thread Fujii Masao
On 2021/11/20 0:19, Fujii Masao wrote: On 2021/11/19 16:54, Michael Paquier wrote: On Thu, Nov 18, 2021 at 10:04:57AM +0530, Bharath Rupireddy wrote: Yeah let's not do that. I'm fine with the wait_event_for_archive_command_v2.patch as is. Switched the patch as RfC, then. Thanks!

Re: wait event and archive_command

2021-11-19 Thread Fujii Masao
On 2021/11/19 16:54, Michael Paquier wrote: On Thu, Nov 18, 2021 at 10:04:57AM +0530, Bharath Rupireddy wrote: Yeah let's not do that. I'm fine with the wait_event_for_archive_command_v2.patch as is. Switched the patch as RfC, then. Thanks! Barring any objection, I will commit the patch.

Re: wait event and archive_command

2021-11-18 Thread Michael Paquier
On Thu, Nov 18, 2021 at 10:04:57AM +0530, Bharath Rupireddy wrote: > Yeah let's not do that. I'm fine with the > wait_event_for_archive_command_v2.patch as is. Switched the patch as RfC, then. -- Michael signature.asc Description: PGP signature

Re: wait event and archive_command

2021-11-17 Thread Bharath Rupireddy
On Thu, Nov 18, 2021 at 7:53 AM Fujii Masao wrote: > > 1) Can't we determine the wait event type based on commandName in > > ExecuteRecoveryCommand instead of passing it as an extra param? > > Yes, that's possible. But isn't it uglier to make ExecuteRecoveryCommand() > have > the map of command

Re: wait event and archive_command

2021-11-17 Thread Michael Paquier
On Thu, Nov 18, 2021 at 11:23:17AM +0900, Fujii Masao wrote: > Yes, that's possible. But isn't it uglier to make ExecuteRecoveryCommand() > have > the map of command name and wait event? So I feel inclined to avoid adding > something like the following code into the function... Thought? FWIW, I

Re: wait event and archive_command

2021-11-17 Thread Fujii Masao
On 2021/11/10 20:19, Bharath Rupireddy wrote: Thanks for the patch. It looks good to me other than the following comment: Thanks for the review! 1) Can't we determine the wait event type based on commandName in ExecuteRecoveryCommand instead of passing it as an extra param? Yes, that's

Re: wait event and archive_command

2021-11-10 Thread Bharath Rupireddy
On Wed, Nov 10, 2021 at 5:00 PM Bharath Rupireddy wrote: > > On Mon, Nov 1, 2021 at 2:31 PM Fujii Masao > wrote: > > > The following activitymsg that are being set to ps display in > > > XLogFileRead and pgarch_archiveXlog have come up for one of our > > > internal team discussions recently: >

Re: wait event and archive_command

2021-11-10 Thread Bharath Rupireddy
On Mon, Nov 1, 2021 at 2:31 PM Fujii Masao wrote: > > The following activitymsg that are being set to ps display in > > XLogFileRead and pgarch_archiveXlog have come up for one of our > > internal team discussions recently: > > > > snprintf(activitymsg, sizeof(activitymsg), "waiting

Re: wait event and archive_command

2021-11-10 Thread Sascha Kuhl
Great, so great. Thanks you Bharath Rupireddy schrieb am Mi., 10. Nov. 2021, 12:20: > On Mon, Nov 1, 2021 at 2:31 PM Fujii Masao > wrote: > > On 2021/10/21 23:55, Bharath Rupireddy wrote: > > >> Also how about adding wait events for other commands like > > >> archive_cleanup_command,

Re: wait event and archive_command

2021-11-10 Thread Bharath Rupireddy
On Mon, Nov 1, 2021 at 2:31 PM Fujii Masao wrote: > On 2021/10/21 23:55, Bharath Rupireddy wrote: > >> Also how about adding wait events for other commands like > >> archive_cleanup_command, restore_command and recovery_end_command? > > > > +1 for the wait event. > > Thanks! > I added the wait

Re: wait event and archive_command

2021-11-01 Thread Fujii Masao
On 2021/10/22 18:32, Michael Paquier wrote: On Thu, Oct 21, 2021 at 10:57:50PM +0900, Fujii Masao wrote: Also how about adding wait events for other commands like archive_cleanup_command, restore_command and recovery_end_command? +1 to add something for all of them as we track the startup

Re: wait event and archive_command

2021-11-01 Thread Fujii Masao
On 2021/10/21 23:55, Bharath Rupireddy wrote: Also how about adding wait events for other commands like archive_cleanup_command, restore_command and recovery_end_command? +1 for the wait event. Thanks! I added the wait events for also restore_command, etc into the patch. I attached that

Re: wait event and archive_command

2021-10-22 Thread Michael Paquier
On Thu, Oct 21, 2021 at 10:57:50PM +0900, Fujii Masao wrote: > Also how about adding wait events for other commands like > archive_cleanup_command, restore_command and recovery_end_command? +1 to add something for all of them as we track the startup process in pg_stat_activity. Thinking with a

Re: wait event and archive_command

2021-10-21 Thread Bharath Rupireddy
On Thu, Oct 21, 2021 at 7:28 PM Fujii Masao wrote: > > Hi, > > I'd like to propose to add new wait event reported while archiver process > is executing archive_command. This would be helpful to observe > what archiver is doing and check whether it has some troubles or not. &g

wait event and archive_command

2021-10-21 Thread Fujii Masao
Hi, I'd like to propose to add new wait event reported while archiver process is executing archive_command. This would be helpful to observe what archiver is doing and check whether it has some troubles or not. Thought? PoC patch attached. Also how about adding wait events for other commands

Re: support for windows robocopy in archive_command and restore_command

2021-08-18 Thread Andrew Dunstan
On 8/18/21 10:32 PM, Julien Rouhaud wrote: > On Thu, Aug 19, 2021 at 5:01 AM Andrew Dunstan wrote: >> It doesn't allow for files to be renamed, but luckily we don't normally >> need it to. > Doesn't the recovery ask to recover the WAL files in > pg_wal/RECOVERYXLOG and the history files in

Re: support for windows robocopy in archive_command and restore_command

2021-08-18 Thread Julien Rouhaud
On Thu, Aug 19, 2021 at 5:01 AM Andrew Dunstan wrote: > > It doesn't allow for files to be renamed, but luckily we don't normally > need it to. Doesn't the recovery ask to recover the WAL files in pg_wal/RECOVERYXLOG and the history files in pg_wal/RECOVERYHISTORY rather than the original names?

support for windows robocopy in archive_command and restore_command

2021-08-18 Thread Andrew Dunstan
obocopy sourcedir destdir filename" It doesn't allow for files to be renamed, but luckily we don't normally need it to. However, what we do need is the ability to get the directory name from the %p parameter for archive_command and restore_command, stripped of the actual file name. I therefore su

Re: archive_command / pg_stat_archiver & documentation

2021-03-03 Thread Michael Paquier
On Wed, Mar 03, 2021 at 09:13:09PM +0800, Julien Rouhaud wrote: > I think that we should consider this as committed. It should, so done now. -- Michael signature.asc Description: PGP signature

Re: archive_command / pg_stat_archiver & documentation

2021-03-03 Thread Julien Rouhaud
On Wed, Mar 03, 2021 at 07:37:02AM -0500, David Steele wrote: > On 3/1/21 8:29 PM, Michael Paquier wrote: > > On Mon, Mar 01, 2021 at 05:17:06PM +0800, Julien Rouhaud wrote: > > > Maybe this can be better addressed than with a link in the > > > documentation. The final outcome is that it can be

Re: archive_command / pg_stat_archiver & documentation

2021-03-03 Thread David Steele
On 3/1/21 8:29 PM, Michael Paquier wrote: On Mon, Mar 01, 2021 at 05:17:06PM +0800, Julien Rouhaud wrote: Maybe this can be better addressed than with a link in the documentation. The final outcome is that it can be difficult to monitor the archiver state in such case. That's orthogonal to

Re: archive_command / pg_stat_archiver & documentation

2021-03-02 Thread Benoit Lobréau
Thanks ! Le mar. 2 mars 2021 à 04:10, Julien Rouhaud a écrit : > On Tue, Mar 2, 2021 at 9:29 AM Michael Paquier > wrote: > > > > On Mon, Mar 01, 2021 at 05:17:06PM +0800, Julien Rouhaud wrote: > > > Maybe this can be better addressed than with a link in the > > > documentation. The final

Re: archive_command / pg_stat_archiver & documentation

2021-03-01 Thread Julien Rouhaud
On Tue, Mar 2, 2021 at 9:29 AM Michael Paquier wrote: > > On Mon, Mar 01, 2021 at 05:17:06PM +0800, Julien Rouhaud wrote: > > Maybe this can be better addressed than with a link in the > > documentation. The final outcome is that it can be difficult to > > monitor the archiver state in such

Re: archive_command / pg_stat_archiver & documentation

2021-03-01 Thread Michael Paquier
On Mon, Mar 01, 2021 at 05:17:06PM +0800, Julien Rouhaud wrote: > Maybe this can be better addressed than with a link in the > documentation. The final outcome is that it can be difficult to > monitor the archiver state in such case. That's orthogonal to this > patch but maybe we can add a new

Re: archive_command / pg_stat_archiver & documentation

2021-03-01 Thread Julien Rouhaud
On Mon, Mar 1, 2021 at 5:24 PM Benoit Lobréau wrote: > > I like the idea ! > > If it's not too complicated, I'd like to take a stab at it. Great! And it shouldn't be too complicated. Note that unfortunately this will likely not be included in pg14 as the last commitfest should begin today.

Re: archive_command / pg_stat_archiver & documentation

2021-03-01 Thread Benoit Lobréau
I like the idea ! If it's not too complicated, I'd like to take a stab at it. Le lun. 1 mars 2021 à 10:16, Julien Rouhaud a écrit : > On Mon, Mar 1, 2021 at 4:33 PM Benoit Lobréau > wrote: > > > > Le lun. 1 mars 2021 à 08:36, Michael Paquier a > écrit : > >> > >> On Fri, Feb 26, 2021 at

Re: archive_command / pg_stat_archiver & documentation

2021-03-01 Thread Julien Rouhaud
On Mon, Mar 1, 2021 at 4:33 PM Benoit Lobréau wrote: > > Le lun. 1 mars 2021 à 08:36, Michael Paquier a écrit : >> >> On Fri, Feb 26, 2021 at 10:03:05AM +0100, Benoit Lobréau wrote: >> > Done here : https://commitfest.postgresql.org/32/3012/ >> >> Documenting that properly for the archive

Re: archive_command / pg_stat_archiver & documentation

2021-03-01 Thread Benoit Lobréau
Le lun. 1 mars 2021 à 08:36, Michael Paquier a écrit : > On Fri, Feb 26, 2021 at 10:03:05AM +0100, Benoit Lobréau wrote: > > Done here : https://commitfest.postgresql.org/32/3012/ > > Documenting that properly for the archive command, as already done for > restore_command, sounds good to me. I

Re: archive_command / pg_stat_archiver & documentation

2021-02-28 Thread Julien Rouhaud
On Mon, Mar 1, 2021 at 3:36 PM Michael Paquier wrote: > > On Fri, Feb 26, 2021 at 10:03:05AM +0100, Benoit Lobréau wrote: > > Done here : https://commitfest.postgresql.org/32/3012/ > > Documenting that properly for the archive command, as already done for > restore_command, sounds good to me. I

Re: archive_command / pg_stat_archiver & documentation

2021-02-28 Thread Michael Paquier
On Fri, Feb 26, 2021 at 10:03:05AM +0100, Benoit Lobréau wrote: > Done here : https://commitfest.postgresql.org/32/3012/ Documenting that properly for the archive command, as already done for restore_command, sounds good to me. I am not sure that there is much point in doing a cross-reference to

Re: archive_command / pg_stat_archiver & documentation

2021-02-26 Thread Benoit Lobréau
Done here : https://commitfest.postgresql.org/32/3012/ Le jeu. 25 févr. 2021 à 15:34, Julien Rouhaud a écrit : > On Thu, Feb 25, 2021 at 7:25 PM Benoit Lobréau > wrote: > > > > Le mer. 24 févr. 2021 à 14:52, Julien Rouhaud a > écrit : > >> > >> I thought that this behavior was documented,

Re: archive_command / pg_stat_archiver & documentation

2021-02-25 Thread Julien Rouhaud
On Thu, Feb 25, 2021 at 7:25 PM Benoit Lobréau wrote: > > Le mer. 24 févr. 2021 à 14:52, Julien Rouhaud a écrit : >> >> I thought that this behavior was documented, especially for the lack >> of update of pg_stat_archiver. If it's not the case then we should >> definitely fix that! > > I tried

Re: archive_command / pg_stat_archiver & documentation

2021-02-25 Thread Benoit Lobréau
terminated by a signal > (other than SIGTERM, which is used as part of a database server shutdown) > or an error by the shell (such as command not found), then recovery will > abort and the server will not start up. > > " > > > > But, I dont see such a note on

Re: archive_command / pg_stat_archiver & documentation

2021-02-24 Thread Julien Rouhaud
shutdown) or an error by > the shell (such as command not found), then recovery will abort and the > server will not start up. > " > > But, I dont see such a note on the archive_command side of thing. [2] > > It could happend in case the archive command is not checked be

archive_command / pg_stat_archiver & documentation

2021-02-24 Thread talk to ben
ot start up. " But, I dont see such a note on the archive_command side of thing. [2] It could happend in case the archive command is not checked beforehand or if the archive command becomes unavailable while PostgreSQL is running. rsync can also return 255 in some cases (bad ssh configura

archive_command

2018-03-07 Thread Rui DeSousa
Hi, I’ve been encouraged to submit this code as there has been talk in the past about a simple pgcopy command to use with the archive_command. Currently there is really no good solution in most base systems without having to introduce a dedicated third party Postgres solution. The best