Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-27 Thread Giuseppe Broccolo
Hi Marco, On 16/01/15 16:55, Marco Nenciarini wrote: > On 14/01/15 17:22, Gabriele Bartolini wrote: > > > > My opinion, Marco, is that for version 5 of this patch, you: > > > > 1) update the information on the wiki (it is outdated - I know you have > > been busy with LSN map optimisation) > > Done

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-16 Thread Marco Nenciarini
On 14/01/15 17:22, Gabriele Bartolini wrote: > > My opinion, Marco, is that for version 5 of this patch, you: > > 1) update the information on the wiki (it is outdated - I know you have > been busy with LSN map optimisation) Done. > 2) modify pg_basebackup in order to accept a directory (or tar

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-14 Thread Gabriele Bartolini
Hi Marco, thank you for sending an updated patch. I am writing down a report of this initial (and partial) review. IMPORTANT: This patch is not complete, as stated by Marco. See the "Conclusions" section for my proposed TODO list. == Patch application I have been able to successfully apply yo

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-13 Thread Marco Nenciarini
Il 13/01/15 12:53, Gabriele Bartolini ha scritto: > Hi Marco, > > could you please send an updated version the patch against the current > HEAD in order to facilitate reviewers? > Here is the updated patch for incremental file based backup. It is based on the current HEAD. I'm now working to

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-13 Thread Gabriele Bartolini
Hi Marco, could you please send an updated version the patch against the current HEAD in order to facilitate reviewers? Thanks, Gabriele -- Gabriele Bartolini - 2ndQuadrant Italia - Managing Director PostgreSQL Training, Services and Support gabriele.bartol...@2ndquadrant.it | www.2ndQuadra

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-07 Thread Marco Nenciarini
Il 06/01/15 14:26, Robert Haas ha scritto: > I suggest leaving this out altogether for the first version. I can > think of three possible ways that we can determine which blocks need > to be backed up. One, just read every block in the database and look > at the LSN of each one. Two, maintain a

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-06 Thread Jehan-Guillaume de Rorthais
On Tue, 6 Jan 2015 08:26:22 -0500 Robert Haas wrote: > Three, scan the WAL generated since the incremental backup and summarize it > into a list of blocks that need to be backed up. This can be done from the archive side. I was talking about some months ago now: http://www.postgresql.org/mes

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-06 Thread Robert Haas
On Tue, Oct 14, 2014 at 1:17 PM, Marco Nenciarini wrote: > I would to replace the getMaxLSN function with a more-or-less persistent > structure which contains the maxLSN for each data segment. > > To make it work I would hook into the ForwardFsyncRequest() function in > src/backend/postmaster/chec

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-05 Thread Michael Paquier
On Mon, Jan 5, 2015 at 7:56 PM, Marco Nenciarini wrote: > I've noticed that I missed to add this to the commitfest. > > I've just added it. > > It is not meant to end up in a committable state, but at this point I'm > searching for some code review and more discusison. > > I'm also about to send a

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-05 Thread Marco Nenciarini
I've noticed that I missed to add this to the commitfest. I've just added it. It is not meant to end up in a committable state, but at this point I'm searching for some code review and more discusison. I'm also about to send an additional patch to implement an LSN map as an additional fork for h

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-05 Thread Marco Nenciarini
I've noticed that I missed to add this to the commitfest. I've just added it. It is not meant to end up in a committable state, but at this point I'm searching for some code review and more discusison. I'm also about to send an additional patch to implement an LSN map as an additional fork for h

[HACKERS] [RFC] Incremental backup v3: incremental PoC

2014-10-14 Thread Marco Nenciarini
Hi Hackers, following the advices gathered on the list I've prepared a third partial patch on the way of implementing incremental pg_basebackup as described here https://wiki.postgresql.org/wiki/Incremental_backup == Changes Compared to the previous version I've made the following changes: * T