Re: [HACKERS] Recovery target 'immediate'

2013-05-07 Thread Simon Riggs
On 3 May 2013 14:40, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 03.05.2013 16:29, Bruce Momjian wrote: On Fri, May 3, 2013 at 01:02:08PM +0200, Cédric Villemain wrote: This changes the existing API which will confuse people that know it and invalidate everything written in

Re: [HACKERS] Recovery target 'immediate'

2013-05-07 Thread Heikki Linnakangas
On 07.05.2013 15:38, Simon Riggs wrote: On 3 May 2013 14:40, Heikki Linnakangashlinnakan...@vmware.com wrote: If we want to avoid adding a new option for this, how about a magic restore point called consistent or immediate: recovery_target_name='immediate' That would stop recovery right

Re: [HACKERS] Recovery target 'immediate'

2013-05-07 Thread Fujii Masao
On Tue, May 7, 2013 at 9:38 PM, Simon Riggs si...@2ndquadrant.com wrote: On 3 May 2013 14:40, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 03.05.2013 16:29, Bruce Momjian wrote: On Fri, May 3, 2013 at 01:02:08PM +0200, Cédric Villemain wrote: This changes the existing API which will

Re: [HACKERS] Recovery target 'immediate'

2013-05-07 Thread Simon Riggs
On 7 May 2013 13:50, Heikki Linnakangas hlinnakan...@vmware.com wrote: Can I suggest that we discuss a range of related changes together? So we have a roadmap of agreed changes in this area. That will be more efficient than discussing each one individually; often each one makes sense only as

Re: [HACKERS] Recovery target 'immediate'

2013-05-06 Thread Robert Haas
On Fri, May 3, 2013 at 11:13 AM, Cédric Villemain ced...@2ndquadrant.com wrote: If we want to avoid adding a new option for this, how about a magic restore point called consistent or immediate: recovery_target_name='immediate' That would stop recovery right after reaching consistency, but

Re: [HACKERS] Recovery target 'immediate'

2013-05-03 Thread Cédric Villemain
Le vendredi 3 mai 2013 02:54:15, Michael Paquier a écrit : On Fri, May 3, 2013 at 8:56 AM, Bruce Momjian br...@momjian.us wrote: On Thu, May 2, 2013 at 09:31:03AM +0200, Magnus Hagander wrote: Actually, there is - I hear it quite often from people not so experienced in PostgreSQL. Though

Re: [HACKERS] Recovery target 'immediate'

2013-05-03 Thread Bruce Momjian
On Fri, May 3, 2013 at 01:02:08PM +0200, Cédric Villemain wrote: This changes the existing API which will confuse people that know it and invalidate everything written in software and on wikis as to how to do it. That means all the in case of fire break glass instructions are

Re: [HACKERS] Recovery target 'immediate'

2013-05-03 Thread Heikki Linnakangas
On 03.05.2013 16:29, Bruce Momjian wrote: On Fri, May 3, 2013 at 01:02:08PM +0200, Cédric Villemain wrote: This changes the existing API which will confuse people that know it and invalidate everything written in software and on wikis as to how to do it. That means all the in case of fire

Re: [HACKERS] Recovery target 'immediate'

2013-05-03 Thread Cédric Villemain
Le vendredi 3 mai 2013 15:40:51, Heikki Linnakangas a écrit : On 03.05.2013 16:29, Bruce Momjian wrote: On Fri, May 3, 2013 at 01:02:08PM +0200, Cédric Villemain wrote: This changes the existing API which will confuse people that know it and invalidate everything written in software and on

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Simon Riggs
On 26 April 2013 18:13, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 26.04.2013 19:50, Magnus Hagander wrote: On Fri, Apr 26, 2013 at 6:43 PM, Simon Riggssi...@2ndquadrant.com wrote: On 26 April 2013 17:25, Heikki Linnakangashlinnakan...@vmware.com wrote: Actually, from a

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Magnus Hagander
On Thu, May 2, 2013 at 8:55 AM, Simon Riggs si...@2ndquadrant.com wrote: On 26 April 2013 18:13, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 26.04.2013 19:50, Magnus Hagander wrote: On Fri, Apr 26, 2013 at 6:43 PM, Simon Riggssi...@2ndquadrant.com wrote: On 26 April 2013 17:25,

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Simon Riggs
On 2 May 2013 08:31, Magnus Hagander mag...@hagander.net wrote: That said, there is one property that's very unclear now and that's that you can only set one of recovery_target_time, recovery_target_xid and recovery_target_name. But they can be freely combined with recovery_target_timeline

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Bruce Momjian
On Thu, May 2, 2013 at 09:04:20AM +0100, Simon Riggs wrote: If we feel strongly about user interface design problems we should treat them the same way we treat performance issues. Profile to identify problem areas, analyze problems in those areas and suggest solutions, then make tests to

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Bruce Momjian
On Thu, May 2, 2013 at 09:31:03AM +0200, Magnus Hagander wrote: Actually, there is - I hear it quite often from people not so experienced in PostgreSQL. Though in fairness, I'm not entirely sure the new syntax would help - some of those need a tool to do it for them, really (and such tools

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Michael Paquier
On Fri, May 3, 2013 at 8:56 AM, Bruce Momjian br...@momjian.us wrote: On Thu, May 2, 2013 at 09:31:03AM +0200, Magnus Hagander wrote: Actually, there is - I hear it quite often from people not so experienced in PostgreSQL. Though in fairness, I'm not entirely sure the new syntax would

Re: [HACKERS] Recovery target 'immediate'

2013-05-01 Thread Bruce Momjian
On Fri, Apr 26, 2013 at 09:48:48AM -0400, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: That said, maybe the easier choice for a *system* (such as v-thingy) would be to simply to the full backup using pg_basebackup -x (or similar), therefor not needing the log archive at all

Re: [HACKERS] Recovery target 'immediate'

2013-05-01 Thread Michael Paquier
On Thu, May 2, 2013 at 7:40 AM, Bruce Momjian br...@momjian.us wrote: On Fri, Apr 26, 2013 at 09:48:48AM -0400, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: That said, maybe the easier choice for a *system* (such as v-thingy) would be to simply to the full backup using

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 18 April 2013 19:11, Heikki Linnakangas hlinnakan...@vmware.com wrote: I just found out that if you use continuous archiving and online backups, it's surprisingly difficult to restore a backup, without replaying any more WAL than necessary. I didn't add it myself because I don't see the

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Heikki Linnakangas
On 26.04.2013 12:16, Simon Riggs wrote: On 18 April 2013 19:11, Heikki Linnakangashlinnakan...@vmware.com wrote: I just found out that if you use continuous archiving and online backups, it's surprisingly difficult to restore a backup, without replaying any more WAL than necessary. I didn't

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 11:29, Heikki Linnakangas hlinnakan...@vmware.com wrote: But there is also an operation to simply restore a backup. Just because a tool supports an imprecise definition of a restore, doesn't mean Postgres should encourage and support that. Restore a backup is more suited to

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Magnus Hagander
On Fri, Apr 26, 2013 at 1:47 PM, Simon Riggs si...@2ndquadrant.com wrote: On 26 April 2013 11:29, Heikki Linnakangas hlinnakan...@vmware.com wrote: But there is also an operation to simply restore a backup. Just because a tool supports an imprecise definition of a restore, doesn't mean

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 12:54, Magnus Hagander mag...@hagander.net wrote: That said, maybe the easier choice for a *system* (such as v-thingy) would be to simply to the full backup using pg_basebackup -x (or similar), therefor not needing the log archive at all when restoring. Yes, it makes the base

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Heikki Linnakangas
On 26.04.2013 14:54, Magnus Hagander wrote: On Fri, Apr 26, 2013 at 1:47 PM, Simon Riggssi...@2ndquadrant.com wrote: On 26 April 2013 11:29, Heikki Linnakangashlinnakan...@vmware.com wrote: But there is also an operation to simply restore a backup. Just because a tool supports an

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: That said, maybe the easier choice for a *system* (such as v-thingy) would be to simply to the full backup using pg_basebackup -x (or similar), therefor not needing the log archive at all when restoring. Yes, it makes the base backup slightly larger,

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 14:48, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: That said, maybe the easier choice for a *system* (such as v-thingy) would be to simply to the full backup using pg_basebackup -x (or similar), therefor not needing the log archive at all

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Robert Haas
On Fri, Apr 26, 2013 at 10:05 AM, Simon Riggs si...@2ndquadrant.com wrote: Restore points are definitely the way to go here, this is what they were created for. Stopping at a labelled location has a defined meaning for the user, which is much better than just stop anywhere convenient, which I

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Magnus Hagander
On Apr 26, 2013 4:38 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Apr 26, 2013 at 10:05 AM, Simon Riggs si...@2ndquadrant.com wrote: Restore points are definitely the way to go here, this is what they were created for. Stopping at a labelled location has a defined meaning for the

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 15:38, Robert Haas robertmh...@gmail.com wrote: On Fri, Apr 26, 2013 at 10:05 AM, Simon Riggs si...@2ndquadrant.com wrote: Restore points are definitely the way to go here, this is what they were created for. Stopping at a labelled location has a defined meaning for the user,

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Robert Haas
On Fri, Apr 26, 2013 at 11:35 AM, Simon Riggs si...@2ndquadrant.com wrote: Given that I was describing how we might implement Heikki's suggestion, I find this comment confusing. Please explain. Heikki's suggestion is simply to have a mode that stops as soon as consistency is reached. The

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 16:38, Robert Haas robertmh...@gmail.com wrote: On Fri, Apr 26, 2013 at 11:35 AM, Simon Riggs si...@2ndquadrant.com wrote: Given that I was describing how we might implement Heikki's suggestion, I find this comment confusing. Please explain. Heikki's suggestion is simply to

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Heikki Linnakangas
On 26.04.2013 19:05, Simon Riggs wrote: On 26 April 2013 16:38, Robert Haasrobertmh...@gmail.com wrote: On Fri, Apr 26, 2013 at 11:35 AM, Simon Riggssi...@2ndquadrant.com wrote: Given that I was describing how we might implement Heikki's suggestion, I find this comment confusing. Please

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Robert Haas
On Fri, Apr 26, 2013 at 12:25 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Doing it the other way means you need to add a new kind of recovery target to the API just for this. recovery_target_immediate = on Sounds good to me. Yeah, I don't have a problem with that, at all.

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 17:25, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 26.04.2013 19:05, Simon Riggs wrote: On 26 April 2013 16:38, Robert Haasrobertmh...@gmail.com wrote: On Fri, Apr 26, 2013 at 11:35 AM, Simon Riggssi...@2ndquadrant.com wrote: Given that I was describing how we

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Magnus Hagander
On Fri, Apr 26, 2013 at 6:43 PM, Simon Riggs si...@2ndquadrant.com wrote: On 26 April 2013 17:25, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 26.04.2013 19:05, Simon Riggs wrote: On 26 April 2013 16:38, Robert Haasrobertmh...@gmail.com wrote: On Fri, Apr 26, 2013 at 11:35 AM, Simon

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Heikki Linnakangas
On 26.04.2013 19:50, Magnus Hagander wrote: On Fri, Apr 26, 2013 at 6:43 PM, Simon Riggssi...@2ndquadrant.com wrote: On 26 April 2013 17:25, Heikki Linnakangashlinnakan...@vmware.com wrote: Actually, from a usability point of view I think would be nice to have just one setting,

Re: [HACKERS] Recovery target 'immediate'

2013-04-21 Thread Fujii Masao
On Fri, Apr 19, 2013 at 10:30 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Apr 18, 2013 at 2:11 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I just found out that if you use continuous archiving and online backups, it's surprisingly difficult to restore a backup, without

Re: [HACKERS] Recovery target 'immediate'

2013-04-21 Thread Jaime Casanova
On Fri, Apr 19, 2013 at 8:30 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Apr 18, 2013 at 2:11 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: It seems that we're missing a setting, something like recovery_target = 'immediate', which would mean stop as soon as consistency is

Re: [HACKERS] Recovery target 'immediate'

2013-04-21 Thread Sergey Burladyan
On Thu, Apr 18, 2013 at 10:11 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I just found out that if you use continuous archiving and online backups, it's surprisingly difficult to restore a backup, without replaying any more WAL than necessary. You can find first WAL file name in

Re: [HACKERS] Recovery target 'immediate'

2013-04-21 Thread Michael Paquier
On Fri, Apr 19, 2013 at 3:11 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I just found out that if you use continuous archiving and online backups, it's surprisingly difficult to restore a backup, without replaying any more WAL than necessary. If you don't set a recovery target,

Re: [HACKERS] Recovery target 'immediate'

2013-04-19 Thread Robert Haas
On Thu, Apr 18, 2013 at 2:11 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I just found out that if you use continuous archiving and online backups, it's surprisingly difficult to restore a backup, without replaying any more WAL than necessary. If you don't set a recovery target,

[HACKERS] Recovery target 'immediate'

2013-04-18 Thread Heikki Linnakangas
I just found out that if you use continuous archiving and online backups, it's surprisingly difficult to restore a backup, without replaying any more WAL than necessary. If you don't set a recovery target, PostgreSQL will recover all the WAL it finds. You can set recovery target time to a