Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-31 Thread Dimitri Fontaine
Heikki Linnakangas hlinnakan...@vmware.com writes: That's not related to the -R option, the situation with config_file is the same with or without it. But yes, if you use config_file option to point outside the data directory, the config file won't be backed up. That feels intuitive to me, I

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Heikki Linnakangas
On 27.03.2013 23:36, Simon Riggs wrote: Why do you think recovery_config_directory are different to config_file with respect to pg_basebackup? pg_basebackup doesn't try to *write* anything to config_file. It does write to $PGDATA/recovery.conf, with the expectation that it takes effect when

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Simon Riggs
On 28 March 2013 08:31, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 27.03.2013 23:36, Simon Riggs wrote: Why do you think recovery_config_directory are different to config_file with respect to pg_basebackup? pg_basebackup doesn't try to *write* anything to config_file. It does

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Robert Haas
On Thu, Mar 28, 2013 at 6:23 AM, Simon Riggs si...@2ndquadrant.com wrote: No, it *would* take effect. The parameter is set in a config file that is not part of the backup, so if you start the server from the backup then it doesn't know that the recovery_config_directory had been set and so it

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Simon Riggs
On 28 March 2013 11:36, Robert Haas robertmh...@gmail.com wrote: On Thu, Mar 28, 2013 at 6:23 AM, Simon Riggs si...@2ndquadrant.com wrote: No, it *would* take effect. The parameter is set in a config file that is not part of the backup, so if you start the server from the backup then it

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Heikki Linnakangas
On 28.03.2013 14:45, Simon Riggs wrote: On 28 March 2013 11:36, Robert Haasrobertmh...@gmail.com wrote: On Thu, Mar 28, 2013 at 6:23 AM, Simon Riggssi...@2ndquadrant.com wrote: No, it *would* take effect. The parameter is set in a config file that is not part of the backup, so if you start

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-28 Thread Simon Riggs
On 28 March 2013 13:47, Heikki Linnakangas hlinnakan...@vmware.com wrote: Therefore anybody using pg_basebackup and the config_file parameter does *not* have an executable backup when used with the -R option, as Heikki was suggesting was a requirement for this patch. That's not related to

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 13:47, Simon Riggs wrote: Allow external recovery_config_directory If required, recovery.conf can now be located outside of the data directory. Server needs read/write permissions on this directory. This was a surprising commit. Does this get us any closer to merging

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Wed, Mar 27, 2013 at 9:12 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 27.03.2013 13:47, Simon Riggs wrote: Allow external recovery_config_directory If required, recovery.conf can now be located outside of the data directory. Server needs read/write permissions on this

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Wed, Mar 27, 2013 at 9:59 PM, Michael Paquier michael.paqu...@gmail.comwrote: On Wed, Mar 27, 2013 at 9:12 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 27.03.2013 13:47, Simon Riggs wrote: Allow external recovery_config_directory If required, recovery.conf can now be

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 12:12, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 27.03.2013 13:47, Simon Riggs wrote: Allow external recovery_config_directory If required, recovery.conf can now be located outside of the data directory. Server needs read/write permissions on this directory.

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 12:59, Michael Paquier michael.paqu...@gmail.com wrote: Also, based on Greg's spec (that Robert and I basically agreed on), if recovery.conf is found at the root of data folder an error is returned to user, recommending him to migrate correctly by referring to dedicated

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Robert Haas
On Wed, Mar 27, 2013 at 9:11 AM, Simon Riggs si...@2ndquadrant.com wrote: On 27 March 2013 12:59, Michael Paquier michael.paqu...@gmail.com wrote: Also, based on Greg's spec (that Robert and I basically agreed on), if recovery.conf is found at the root of data folder an error is returned to

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 13:21, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 27, 2013 at 9:11 AM, Simon Riggs si...@2ndquadrant.com wrote: On 27 March 2013 12:59, Michael Paquier michael.paqu...@gmail.com wrote: Also, based on Greg's spec (that Robert and I basically agreed on), if

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 15:11, Simon Riggs wrote: On 27 March 2013 12:59, Michael Paquiermichael.paqu...@gmail.com wrote: Also, based on Greg's spec (that Robert and I basically agreed on), if recovery.conf is found at the root of data folder an error is returned to user, recommending him to migrate

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Robert Haas
On Wed, Mar 27, 2013 at 9:40 AM, Simon Riggs si...@2ndquadrant.com wrote: On 27 March 2013 13:21, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 27, 2013 at 9:11 AM, Simon Riggs si...@2ndquadrant.com wrote: On 27 March 2013 12:59, Michael Paquier michael.paqu...@gmail.com wrote: Also,

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 13:53, Robert Haas robertmh...@gmail.com wrote: Please look again. I have a better idea: how about if you tell me where you see any such message? Because I think the reason I don't see it is because it doesn't exist.

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Robert Haas
On Wed, Mar 27, 2013 at 10:06 AM, Simon Riggs si...@2ndquadrant.com wrote: On 27 March 2013 13:53, Robert Haas robertmh...@gmail.com wrote: Please look again. I have a better idea: how about if you tell me where you see any such message? Because I think the reason I don't see it is because

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Wed, Mar 27, 2013 at 10:11 PM, Simon Riggs si...@2ndquadrant.com wrote: On 27 March 2013 12:59, Michael Paquier michael.paqu...@gmail.com wrote: Also, based on Greg's spec (that Robert and I basically agreed on), if recovery.conf is found at the root of data folder an error is returned

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Wed, Mar 27, 2013 at 10:43 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 3. Would it make sense to make the option recovery_config_file, pointing to the file, instead of just the directory? +1 on that. I just sent the same suggestion. -- Michael

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 15:09, Simon Riggs wrote: On 27 March 2013 12:12, Heikki Linnakangashlinnakan...@vmware.com wrote: On 27.03.2013 13:47, Simon Riggs wrote: Allow external recovery_config_directory If required, recovery.conf can now be located outside of the data directory. Server needs

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Robert Haas
On Wed, Mar 27, 2013 at 10:15 AM, Michael Paquier michael.paqu...@gmail.com wrote: Here would be the plan: 1) we move all the recovery parameters to GUC as proposed Masao's patch (and somewhat my patch now). 2) as default, the custom file that is used to trigger recovery is called

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Wed, Mar 27, 2013 at 11:26 PM, Robert Haas robertmh...@gmail.com wrote: There are also weird edge cases here when the server is promoted. The recovery.conf file won't exist any more, but the GUC settings changes it contains will live on until the next SIGHUP. Yes indeed, I forgot that.

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 14:20, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 27.03.2013 15:09, Simon Riggs wrote: On 27 March 2013 12:12, Heikki Linnakangashlinnakan...@vmware.com wrote: On 27.03.2013 13:47, Simon Riggs wrote: Allow external recovery_config_directory If required,

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 17:23, Simon Riggs wrote: On 27 March 2013 14:20, Heikki Linnakangashlinnakan...@vmware.com wrote: You didn't answer the question. Does this get us any closer to merging postgresql.conf and recovery.conf? Why is this bundled in with that? Why do you think these points are

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 15:35, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 27.03.2013 17:23, Simon Riggs wrote: On 27 March 2013 14:20, Heikki Linnakangashlinnakan...@vmware.com wrote: You didn't answer the question. Does this get us any closer to merging postgresql.conf and

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 18:10, Simon Riggs wrote: On 27 March 2013 15:35, Heikki Linnakangashlinnakan...@vmware.com wrote: Ok, cool. Can you please revert this commit so that we can move on, then? Please explain why you want this reverted, without mentioning the other task we agree is required. If

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 27.03.2013 18:10, Simon Riggs wrote: On 27 March 2013 15:35, Heikki Linnakangashlinnakan...@vmware.com wrote: Ok, cool. Can you please revert this commit so that we can move on, then? Please explain why you want this reverted, without

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 16:24, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 27.03.2013 18:10, Simon Riggs wrote: On 27 March 2013 15:35, Heikki Linnakangashlinnakan...@vmware.com wrote: Ok, cool. Can you please revert this commit so that we can move on, then? Please explain why you want

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 19:34, Simon Riggs wrote: On 27 March 2013 16:24, Heikki Linnakangashlinnakan...@vmware.com wrote: Lastly, it breaks the new pg_basebackup -R functionality; pg_basebackup will create the recovery.conf file, but it won't take effect. AFAIK pg_basebackup doesn't backup files not

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 17:23, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 27.03.2013 18:10, Simon Riggs wrote: On 27 March 2013 15:35, Heikki Linnakangashlinnakan...@vmware.com wrote: Ok, cool. Can you please revert this commit so that we can move on,

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 17:50, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 27.03.2013 19:34, Simon Riggs wrote: On 27 March 2013 16:24, Heikki Linnakangashlinnakan...@vmware.com wrote: Lastly, it breaks the new pg_basebackup -R functionality; pg_basebackup will create the recovery.conf

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Fujii Masao
On Thu, Mar 28, 2013 at 1:24 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 27.03.2013 18:10, Simon Riggs wrote: On 27 March 2013 15:35, Heikki Linnakangashlinnakan...@vmware.com wrote: Ok, cool. Can you please revert this commit so that we can move on, then? Please explain why

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Thu, Mar 28, 2013 at 2:58 AM, Simon Riggs si...@2ndquadrant.com wrote: Arguments against? Also the fact that many discussions have been done on recovery.conf between the time this feature has been decided and actually committed (perhaps too promptly just by looking at how this thread is

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 18:15, Fujii Masao masao.fu...@gmail.com wrote: This patch breaks also pg_ctl promote because it always checks the existence of $PGDATA/recovery.conf. You're right. It does. Good catch. That also suggest a solution: create a status file called $PGDATA/recovery_in_progress

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Heikki Linnakangas
On 27.03.2013 20:02, Simon Riggs wrote: On 27 March 2013 17:50, Heikki Linnakangashlinnakan...@vmware.com wrote: Imagine that you have set recovery_config_directory='/foo/' in the master server. Now you want to set up a standby, so you do:' pg_basebackup -D data-standby -R With the -R

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: doc/src/sgml/config.sgml  |  17 + So that doc builds could proceed without error I fixed the obvious pasto in config.sgml. -- Kevin Grittner EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Simon Riggs
On 27 March 2013 21:05, Heikki Linnakangas hlinnakan...@vmware.com wrote: Same argument applies to all conf files, not just recovery.conf. Sounds like the patch to add -R to pg_basebackup should be revoked as being not well thought out. Or it should be fixed, in which case this works the

Re: [HACKERS] [COMMITTERS] pgsql: Allow external recovery_config_directory

2013-03-27 Thread Michael Paquier
On Thu, Mar 28, 2013 at 6:05 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: What exactly was wrong with pg_basebackup -R, without recovery_config_directory? pg_basebackup -R generates automatically recovery.conf inside data folder, so if recovery_config_directory is specified the slave