Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-20 Thread Simon Riggs
On Thu, 2004-12-16 at 17:54, Richard Huxton wrote: Josh Berkus wrote: Clearly, OSDL-DBT2 is not a real world test! That is its benefit, since it is heavily instrumented and we are able to re-run it many times without different parameter settings. The application is well known and doesn't

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-18 Thread Simon Riggs
On Thu, 2004-12-16 at 17:54, Richard Huxton wrote: Josh Berkus wrote: Clearly, OSDL-DBT2 is not a real world test! That is its benefit, since it is heavily instrumented and we are able to re-run it many times without different parameter settings. The application is well known and doesn't

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-17 Thread Greg Stark
Jan Wieck [EMAIL PROTECTED] writes: Doesn't cranking up the bgwriter_percent to 100 effectively make the entire shared memory a write-through cache? In other words, with 100% the bgwriter will allways write all dirty blocks out and it becomes unlikely to avoid an IO for subsequent

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-16 Thread Zeugswetter Andreas DAZ SD
Hmmm, I've not seen this. For example, with people who are having trouble with checkpoint spikes on Linux, I've taken to recommending that they call sync() (via cron) every 5-10 seconds (thanks, Bruce, for suggestion!). Believe it or not, this does help smooth out the spikes and give

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-16 Thread Richard Huxton
Josh Berkus wrote: Simon, Clearly, OSDL-DBT2 is not a real world test! That is its benefit, since it is heavily instrumented and we are able to re-run it many times without different parameter settings. The application is well known and doesn't suffer that badly from factors that would allow

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Jan Wieck
On 12/12/2004 5:08 PM, Simon Riggs wrote: On Sun, 2004-12-12 at 05:46, Neil Conway wrote: Simon Riggs wrote: If the bgwriter_percent = 100, then we should actually do the sensible thing and prepare the list that we need, i.e. limit StrategyDirtyBufferList to finding at most bgwriter_maxpages.

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Jan Wieck
On 12/12/2004 9:43 PM, Neil Conway wrote: On Sun, 2004-12-12 at 22:08 +, Simon Riggs wrote: On Sun, 2004-12-12 at 05:46, Neil Conway wrote: Is the plan to make bgwriter_percent = 100 the default setting? Hmm...must confess that my only plan is: i) discover dynamic behaviour of bgwriter ii)

Re: Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread simon
Josh Berkus [EMAIL PROTECTED] wrote on 15.12.2004, 18:36:53: Hmmm, I've not seen this. For example, with people who are having trouble with checkpoint spikes on Linux, I've taken to recommending that they call sync() (via cron) every 5-10 seconds (thanks, Bruce, for suggestion!).

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Josh Berkus
Jan, I too don't think that this approach will retain the checkpoing smooting effect, the current implementation has. The real problem is that the cleaner the buffer pool is, the longer the scan for dirty buffers will take because the dirty blocks tend to be at the very end of the scan

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Josh Berkus
Folks, To allow DBT2 to be used for real bgwriter benchmarking, Mark would need to change the following: 1) Randomize the timing of the commits, so that sometimes there is only 30 writes/minute, and other times there is 300. A timing pattern that would produce a sine wave with occasional

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Josh Berkus
Simon, Clearly, OSDL-DBT2 is not a real world test! That is its benefit, since it is heavily instrumented and we are able to re-run it many times without different parameter settings. The application is well known and doesn't suffer that badly from factors that would allow certain effects to

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-14 Thread Mark Wong
Sorry for the delay; here are results with the bg3.patch with database parameters that should match run 207. I haven't been able to take the time too look over the results myself, but I tried to make sure this run was the same as 207:

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-14 Thread Mark Wong
Sorry, wrong link, right one here: http://www.osdl.org/projects/dbt2dev/results/dev4-010/211 Mark ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-14 Thread Simon Riggs
On Wed, 2004-12-15 at 00:00, Mark Wong wrote: http://www.osdl.org/projects/dbt2dev/results/dev4-010/211 Thanks Mark for turning that around so quickly. Looks good... Results performed to compare test 207 http://www.osdl.org/projects/dbt2dev/results/dev4-010/207 test 211 with bg3.patch

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-13 Thread Simon Riggs
On Mon, 2004-12-13 at 04:39, Mark Kirkwood wrote: I am seeing a reasonably reproducible performance boost after applying your patch (I'm not sure if that was one of the main objectives, but it certainly is nice). I *was* seeing a noticeable decrease between 7.4.6 and 8.0.0RC1 running

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-13 Thread Simon Riggs
On Mon, 2004-12-13 at 02:43, Neil Conway wrote: On Sun, 2004-12-12 at 22:08 +, Simon Riggs wrote: On Sun, 2004-12-12 at 05:46, Neil Conway wrote: Is the plan to make bgwriter_percent = 100 the default setting? Hmm...must confess that my only plan is: i) discover dynamic behaviour

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-12 Thread Simon Riggs
On Sun, 2004-12-12 at 05:46, Neil Conway wrote: Simon Riggs wrote: If the bgwriter_percent = 100, then we should actually do the sensible thing and prepare the list that we need, i.e. limit StrategyDirtyBufferList to finding at most bgwriter_maxpages. Is the plan to make

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-12 Thread Neil Conway
On Sun, 2004-12-12 at 22:08 +, Simon Riggs wrote: On Sun, 2004-12-12 at 05:46, Neil Conway wrote: Is the plan to make bgwriter_percent = 100 the default setting? Hmm...must confess that my only plan is: i) discover dynamic behaviour of bgwriter ii) fix any bugs or wierdness as quickly

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-12 Thread Mark Kirkwood
Simon, I am seeing a reasonably reproducible performance boost after applying your patch (I'm not sure if that was one of the main objectives, but it certainly is nice). I *was* seeing a noticeable decrease between 7.4.6 and 8.0.0RC1 running pgbench. However, after applying your patch, 8.0 is

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-11 Thread Neil Conway
I wonder if we even need to retain the bgwriter_percent GUC var. Is there actually a situation in which the combination of bgwriter_maxpages and bgwriter_delay does not give the DBA sufficient flexibility in tuning bgwriter behavior? Simon Riggs wrote: If the bgwriter_percent = 100, then we