Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-31 Thread Dimitri Fontaine
Heikki Linnakangas writes: > That's not related to the -R option, the situation with config_file is the > same with or without it. But yes, if you use config_file option to point > outside the data directory, the config file won't be backed up. That feels > intuitive to me, I wouldn't expect it to

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Simon Riggs
On 28 March 2013 13:47, Heikki Linnakangas wrote: >> Therefore anybody using pg_basebackup and the config_file parameter >> does *not* have an executable backup when used with the -R option, as >> Heikki was suggesting was a requirement for this patch. > > > That's not related to the -R option, t

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Heikki Linnakangas
On 28.03.2013 14:45, Simon Riggs wrote: On 28 March 2013 11:36, Robert Haas wrote: On Thu, Mar 28, 2013 at 6:23 AM, Simon Riggs wrote: No, it *would* take effect. The parameter is set in a config file that is not part of the backup, so if you start the server from the backup then it doesn't k

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Simon Riggs
On 28 March 2013 11:36, Robert Haas wrote: > On Thu, Mar 28, 2013 at 6:23 AM, Simon Riggs wrote: >> No, it *would* take effect. The parameter is set in a config file that >> is not part of the backup, so if you start the server from the backup >> then it doesn't know that the recovery_config_dire

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Robert Haas
On Thu, Mar 28, 2013 at 6:23 AM, Simon Riggs wrote: > No, it *would* take effect. The parameter is set in a config file that > is not part of the backup, so if you start the server from the backup > then it doesn't know that the recovery_config_directory had been set > and so it would read the rec

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Simon Riggs
On 28 March 2013 08:31, Heikki Linnakangas wrote: > On 27.03.2013 23:36, Simon Riggs wrote: >> >> Why do you think recovery_config_directory are different to >> config_file with respect to pg_basebackup? > > > pg_basebackup doesn't try to *write* anything to config_file. It does write > to $PGDATA

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Heikki Linnakangas
On 27.03.2013 23:36, Simon Riggs wrote: Why do you think recovery_config_directory are different to config_file with respect to pg_basebackup? pg_basebackup doesn't try to *write* anything to config_file. It does write to $PGDATA/recovery.conf, with the expectation that it takes effect when y

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Thu, Mar 28, 2013 at 6:05 AM, Heikki Linnakangas wrote: > What exactly was wrong with pg_basebackup -R, without > recovery_config_directory? > pg_basebackup -R generates automatically recovery.conf inside data folder, so if recovery_config_directory is specified the slave startup will fail. Th

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 21:05, Heikki Linnakangas wrote: >> Same argument applies to all conf files, not just recovery.conf. >> >> Sounds like the patch to add -R to pg_basebackup should be revoked as >> being not well thought out. Or it should be fixed, in which case this >> works the same. > > What ex

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Kevin Grittner
Simon Riggs wrote: > doc/src/sgml/config.sgml  |  17 + So that doc builds could proceed without error I fixed the obvious pasto in config.sgml. -- Kevin Grittner EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mail

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 20:02, Simon Riggs wrote: On 27 March 2013 17:50, Heikki Linnakangas wrote: Imagine that you have set recovery_config_directory='/foo/' in the master server. Now you want to set up a standby, so you do:' pg_basebackup -D data-standby -R With the -R option, pg_basebackup creates

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 18:15, Fujii Masao wrote: > This patch breaks also pg_ctl promote because it always checks the existence > of > $PGDATA/recovery.conf. You're right. It does. Good catch. That also suggest a solution: create a status file called $PGDATA/recovery_in_progress which would then all

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Thu, Mar 28, 2013 at 2:58 AM, Simon Riggs wrote: > Arguments against? > Also the fact that many discussions have been done on recovery.conf between the time this feature has been decided and actually committed (perhaps too promptly just by looking at how this thread is becoming long)? -- Mich

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Fujii Masao
On Thu, Mar 28, 2013 at 1:24 AM, Heikki Linnakangas wrote: > On 27.03.2013 18:10, Simon Riggs wrote: >> >> On 27 March 2013 15:35, Heikki Linnakangas >> wrote: >>> >>> Ok, cool. Can you please revert this commit so that we can move on, then? >> >> >> Please explain why you want this reverted, with

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 17:50, Heikki Linnakangas wrote: > On 27.03.2013 19:34, Simon Riggs wrote: >> >> On 27 March 2013 16:24, Heikki Linnakangas >> wrote: >>> >>> Lastly, it breaks the new pg_basebackup -R >>> >>> functionality; pg_basebackup will create the recovery.conf file, but it >>> won't take e

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 17:23, Tom Lane wrote: > Heikki Linnakangas writes: >> On 27.03.2013 18:10, Simon Riggs wrote: >>> On 27 March 2013 15:35, Heikki Linnakangas wrote: Ok, cool. Can you please revert this commit so that we can move on, then? > >>> Please explain why you want this reverted, wi

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 19:34, Simon Riggs wrote: On 27 March 2013 16:24, Heikki Linnakangas wrote: Lastly, it breaks the new pg_basebackup -R functionality; pg_basebackup will create the recovery.conf file, but it won't take effect. AFAIK pg_basebackup doesn't backup files not in the data directory an

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 16:24, Heikki Linnakangas wrote: > On 27.03.2013 18:10, Simon Riggs wrote: >> >> On 27 March 2013 15:35, Heikki Linnakangas >> wrote: >>> >>> Ok, cool. Can you please revert this commit so that we can move on, then? >> >> >> Please explain why you want this reverted, without menti

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Tom Lane
Heikki Linnakangas writes: > On 27.03.2013 18:10, Simon Riggs wrote: >> On 27 March 2013 15:35, Heikki Linnakangas wrote: >>> Ok, cool. Can you please revert this commit so that we can move on, then? >> Please explain why you want this reverted, without mentioning the >> other task we agree is r

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 18:10, Simon Riggs wrote: On 27 March 2013 15:35, Heikki Linnakangas wrote: Ok, cool. Can you please revert this commit so that we can move on, then? Please explain why you want this reverted, without mentioning the other task we agree is required. If an admin can't trust that

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 15:35, Heikki Linnakangas wrote: > On 27.03.2013 17:23, Simon Riggs wrote: >> >> On 27 March 2013 14:20, Heikki Linnakangas >> wrote: >>> >>> You didn't answer the question. Does this get us any closer to merging >>> >>> postgresql.conf and recovery.conf? Why is this bundled in wi

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 17:23, Simon Riggs wrote: On 27 March 2013 14:20, Heikki Linnakangas wrote: You didn't answer the question. Does this get us any closer to merging postgresql.conf and recovery.conf? Why is this bundled in with that? Why do you think these points are bundled? Because you say th

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 14:20, Heikki Linnakangas wrote: > On 27.03.2013 15:09, Simon Riggs wrote: >> >> On 27 March 2013 12:12, Heikki Linnakangas >> wrote: >>> >>> On 27.03.2013 13:47, Simon Riggs wrote: Allow external recovery_config_directory If required, recovery.conf can now be

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Wed, Mar 27, 2013 at 11:26 PM, Robert Haas wrote: > There are also weird edge cases here when the server is promoted. The > recovery.conf file won't exist any more, but the GUC settings changes > it contains will live on until the next SIGHUP. > Yes indeed, I forgot that. The proposal Greg S

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Robert Haas
On Wed, Mar 27, 2013 at 10:15 AM, Michael Paquier wrote: > Here would be the plan: > 1) we move all the recovery parameters to GUC as proposed Masao's patch (and > somewhat my patch now). > 2) as default, the custom file that is used to trigger recovery is called > recovery.conf and needs to be lo

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 15:09, Simon Riggs wrote: On 27 March 2013 12:12, Heikki Linnakangas wrote: On 27.03.2013 13:47, Simon Riggs wrote: Allow external recovery_config_directory If required, recovery.conf can now be located outside of the data directory. Server needs read/write permissions on this d

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Wed, Mar 27, 2013 at 10:43 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > 3. Would it make sense to make the option "recovery_config_file", pointing > to the file, instead of just the directory? > +1 on that. I just sent the same suggestion. -- Michael

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Wed, Mar 27, 2013 at 10:11 PM, Simon Riggs wrote: > On 27 March 2013 12:59, Michael Paquier wrote: > > > Also, based on Greg's spec (that Robert and I basically agreed on), if > > recovery.conf is found at the root of data folder an error is returned to > > user, recommending him to migrate c

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Robert Haas
On Wed, Mar 27, 2013 at 10:06 AM, Simon Riggs wrote: > On 27 March 2013 13:53, Robert Haas wrote: >>> Please look again. >> >> I have a better idea: how about if you tell me where you see any such >> message? Because I think the reason I don't see it is because it >> doesn't exist. > > http://ww

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 13:53, Robert Haas wrote: >> Please look again. > > I have a better idea: how about if you tell me where you see any such > message? Because I think the reason I don't see it is because it > doesn't exist. http://www.postgresql.org/message-id/20130109204225.gb21...@momjian.us h

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Robert Haas
On Wed, Mar 27, 2013 at 9:40 AM, Simon Riggs wrote: > On 27 March 2013 13:21, Robert Haas wrote: >> On Wed, Mar 27, 2013 at 9:11 AM, Simon Riggs wrote: >>> On 27 March 2013 12:59, Michael Paquier wrote: >>> Also, based on Greg's spec (that Robert and I basically agreed on), if recover

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 15:11, Simon Riggs wrote: On 27 March 2013 12:59, Michael Paquier wrote: Also, based on Greg's spec (that Robert and I basically agreed on), if recovery.conf is found at the root of data folder an error is returned to user, recommending him to migrate correctly by referring to de

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 13:21, Robert Haas wrote: > On Wed, Mar 27, 2013 at 9:11 AM, Simon Riggs wrote: >> On 27 March 2013 12:59, Michael Paquier wrote: >> >>> Also, based on Greg's spec (that Robert and I basically agreed on), if >>> recovery.conf is found at the root of data folder an error is retur

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Robert Haas
On Wed, Mar 27, 2013 at 9:11 AM, Simon Riggs wrote: > On 27 March 2013 12:59, Michael Paquier wrote: > >> Also, based on Greg's spec (that Robert and I basically agreed on), if >> recovery.conf is found at the root of data folder an error is returned to >> user, recommending him to migrate correc

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 12:59, Michael Paquier wrote: > Also, based on Greg's spec (that Robert and I basically agreed on), if > recovery.conf is found at the root of data folder an error is returned to > user, recommending him to migrate correctly by referring to dedicated > documentation. I'm followi

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 12:12, Heikki Linnakangas wrote: > On 27.03.2013 13:47, Simon Riggs wrote: >> >> Allow external recovery_config_directory >> If required, recovery.conf can now be located outside of the data >> directory. >> Server needs read/write permissions on this directory. > > > This was a s

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Wed, Mar 27, 2013 at 9:59 PM, Michael Paquier wrote: > > > On Wed, Mar 27, 2013 at 9:12 PM, Heikki Linnakangas < > hlinnakan...@vmware.com> wrote: > >> On 27.03.2013 13:47, Simon Riggs wrote: >> >>> Allow external recovery_config_directory >>> If required, recovery.conf can now be located outsi

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Wed, Mar 27, 2013 at 9:12 PM, Heikki Linnakangas wrote: > On 27.03.2013 13:47, Simon Riggs wrote: > >> Allow external recovery_config_directory >> If required, recovery.conf can now be located outside of the data >> directory. >> Server needs read/write permissions on this directory. >> > > Th

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 13:47, Simon Riggs wrote: Allow external recovery_config_directory If required, recovery.conf can now be located outside of the data directory. Server needs read/write permissions on this directory. This was a surprising commit. Does this get us any closer to merging postgresql.c