Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-24 Thread Tom Lane
"Zeugswetter Andreas SB SD" <[EMAIL PROTECTED]> writes: > But it can contain newer blocks than WAL records would expect. > Will it not matter if e.g. a page split for a btree index is already in the > index file, but a WAL record exists, that references the not yet split page? No. This is not dif

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-24 Thread Zeugswetter Andreas SB SD
> Re-thinking the whole purpose of the additional full page images appended to > the xlog records, I now understand and agree with Tom's comment in the docs > that we don't need to include those additional full page images for PITR - > they only exist to correctly recover the database in the event

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-24 Thread Zeugswetter Andreas SB SD
> PS: but something you *could* do in 8.0 is replace "cp" by "gzip" to > archive compressed files that way. How about replacing the page image records with a same size dummy record that only contains a dummy header and all 0's. If the archiver cares about space he will use some sort of compress

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-24 Thread Simon Riggs
> Bruce Momjian... > Alvaro Herrera wrote: > > Anyway I think we are way past feature freeze, and XLog "compression" > > could made it into 8.1 without loss of functionality. At that point we > > will say "now PITR uses less space on the archiver" and that's it. > > > > (Maybe we could even think

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Alvaro Herrera
On Tue, Aug 24, 2004 at 12:05:33AM +0200, Gaetano Mendola wrote: > Simon Riggs wrote: > > >As a result, I have thought that there may be a way to remove those pages > >from the xlog files immediately before being copied away to archive, > >without effecting crash recovery logic AT ALL. > > So th

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > As a result, I have thought that there may be a way to remove those pages > from the xlog files immediately before being copied away to archive, without > effecting crash recovery logic AT ALL. This isn't all that easy. The main problem I can see is tha

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Gaetano Mendola
Simon Riggs wrote: As a result, I have thought that there may be a way to remove those pages from the xlog files immediately before being copied away to archive, without effecting crash recovery logic AT ALL. The archiver process could read the xlog files and re-write them exactly as read to anothe

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Bruce Momjian
Alvaro Herrera wrote: > On Mon, Aug 23, 2004 at 10:03:26PM +0100, Simon Riggs wrote: > > Simon, > > > I raise this now because I'm thinking that this functionality really ought > > to be in the main production 8.0 release. Not sure if anybody will > > agreebut that's what I'm thinking now, ba

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Alvaro Herrera
On Mon, Aug 23, 2004 at 10:03:26PM +0100, Simon Riggs wrote: Simon, > I raise this now because I'm thinking that this functionality really ought > to be in the main production 8.0 release. Not sure if anybody will > agreebut that's what I'm thinking now, based upon what seems like a > simple

[HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Simon Riggs
One of the possible barriers to adoption of PITR is the volume of the logs themselves. Maybe this isn't a problem for now, maybe it is. Re-thinking the whole purpose of the additional full page images appended to the xlog records, I now understand and agree with Tom's comment in the docs that we