Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Zeugswetter Andreas ADI SD
> > 1) To deal with partial/inconsisitent write to the data file at crash > > recovery, we need full page writes at the first modification to pages > > after each checkpoint. It consumes much of WAL space. > > We need to find a way around this someday. Other DBs don't > do this; it may be be

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-24 Thread Zeugswetter Andreas ADI SD
> 3) To maintain crash recovery chance and reduce the amount of > archive log, removal of unnecessary full page writes from > archive logs is a good choice. Definitely, yes. pg_compresslog could even move the full pages written during backup out of WAL and put them in a different file that nee

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-23 Thread Zeugswetter Andreas ADI SD
> I don't insist the name and the default of the GUC parameter. > I'm afraid wal_fullpage_optimization = on (default) makes > some confusion because the default behavior becomes a bit > different on WAL itself. Seems my wal_fullpage_optimization is not a good name if it caused misinterpretati

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-20 Thread Zeugswetter Andreas ADI SD
> With DBT-2 benchmark, I've already compared the amount of WAL. The > result was as follows: > > Amount of WAL after 60min. run of DBT-2 benchmark > wal_add_optimization_info = off (default) 3.13GB how about wal_fullpage_optimization = on (default) > wal_add_optimization_info = on (new ca

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-13 Thread Zeugswetter Andreas ADI SD
> > Yup, this is a good summary. > > > > You say you need to remove the optimization that avoids the logging of > > a new tuple because the full page image exists. > > I think we must already have the info in WAL which tuple inside the > > full page image is new (the one for which we avoided th

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-12 Thread Zeugswetter Andreas ADI SD
> I don't fully understand what "transaction log" means. If it means > "archived WAL", the current (8.2) code handle WAL as follows: Probably we can define "transaction log" to be the part of WAL that is not full pages. > 1) If full_page_writes=off, then no full page writes will be > written

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Zeugswetter Andreas ADI SD
> Without a switch, because both full page writes and > corresponding logical log is included in WAL, this will > increase WAL size slightly > (maybe about five percent or so). If everybody is happy > with this, we > don't need a switch. Sorry, I still don't understand that. What is the "c

Re: [HACKERS] [PATCHES] wal_checksum = on (default) | off

2007-01-05 Thread Zeugswetter Andreas ADI SD
> > Ok, so when you need CRC's on a replicate (but not on the master) you > > turn it > > off during standby replay, but turn it on when you start the replicate > > for normal operation. > > Thought: even when it's off, the CRC had better be computed for > shutdown-checkpoint records. Else there

Re: [HACKERS] [PATCHES] wal_checksum = on (default) | off

2007-01-05 Thread Zeugswetter Andreas ADI SD
> > > > What's the use-case for changing the variable on the fly anyway? Seems a > > better > > > > solution is just to lock down the setting at postmaster start. > > > > I guess that the use case is more for a WAL based replicate, that > > has/wants a different setting. Maybe we want a WAL entr

Re: [HACKERS] [PATCHES] wal_checksum = on (default) | off

2007-01-05 Thread Zeugswetter Andreas ADI SD
> > >>> Recovery can occur with/without same setting of wal_checksum, to avoid > > >>> complications from crashes immediately after turning GUC on. > > >> > > >> Surely not. Otherwise even the "on" setting is not really a defense. > > > > > Only when the CRC is exactly zero, which happens very