Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-17 Thread Bruce Momjian
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. ---

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-15 Thread Heikki Linnakangas
Greg Smith wrote: I realized recently the task I should take on here is to run some more experiments with the latest code and pass along suggested techniques for producing/identifying the kind of problem conditions I've run into in the past; then we can see if other people can reproduce them.

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-15 Thread Jim C. Nasby
Moving to -performance. On Mon, May 14, 2007 at 09:55:16PM -0700, daveg wrote: Apologies for asking this on the wrong list, but it is at least the right thread. What is the current thinking on bg_writer setttings for systems such as 4 core Opteron with 16GB or 32GB of memory and heavy

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-14 Thread ITAGAKI Takahiro
Greg Smith [EMAIL PROTECTED] wrote: The first patch (buf-alloc-stats) takes the two most interesting pieces of data the original patch collected, the number of buffers allocated recently and the number that the clients wrote out, and ties all that into the new stats structure. The second

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-14 Thread Greg Smith
On Mon, 14 May 2007, ITAGAKI Takahiro wrote: BTW, your patch will cut LRU writes short, but will not encourage to do more works. So should set more aggressive values to bgwriter_lru_percent and bgwriter_lru_maxpages as defaults? Setting a bigger default maximum is one possibility I was

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-14 Thread Heikki Linnakangas
Greg Smith wrote: On Mon, 14 May 2007, ITAGAKI Takahiro wrote: BTW, your patch will cut LRU writes short, but will not encourage to do more works. So should set more aggressive values to bgwriter_lru_percent and bgwriter_lru_maxpages as defaults? Setting a bigger default maximum is one

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-14 Thread Tom Lane
Greg Smith [EMAIL PROTECTED] writes: Since the whole background writer setup is kind of complicated, the other thing I was working on is writing a guide on how to use the new pg_stat_bgwriter information to figure out if you need to increase bgwriter_[all|lru]_pages (and the other

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-14 Thread Greg Smith
On Mon, 14 May 2007, Heikki Linnakangas wrote: If it's safe to set it high, let's default it to infinity. The maximum right now is 1000, and that would be a reasonable new default. You really don't to write more than 1000 per interval anyway without taking a break for checkpoints; the more

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-14 Thread Greg Smith
On Mon, 14 May 2007, Tom Lane wrote: If you can write something like that, why do we need the parameter at all? Couple of reasons: -As I already mentioned in my last message, I think it's unwise to let the LRU writes go completely unbounded. I still think there should be a maximum, and if

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-14 Thread daveg
On Mon, May 14, 2007 at 11:19:23PM -0400, Greg Smith wrote: On Mon, 14 May 2007, Tom Lane wrote: If you can write something like that, why do we need the parameter at all? Couple of reasons: -As I already mentioned in my last message, I think it's unwise to let the LRU writes go

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-13 Thread Heikki Linnakangas
Greg Smith wrote: The original code came from before there was a pg_stat_bgwriter. The first patch (buf-alloc-stats) takes the two most interesting pieces of data the original patch collected, the number of buffers allocated recently and the number that the clients wrote out, and ties all

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-13 Thread Greg Smith
On Sun, 13 May 2007, Heikki Linnakangas wrote: StrategyReportWrite increments numClientWrites without holding the BufFreeListLock, that's a race condition. The terminology needs some adjustment; clients don't write buffers, backends do. That was another piece of debugging code I moved into

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-12 Thread Greg Smith
Attached are two patches that try to recast the ideas of Itagaki Takahiro's auto bgwriter_lru_maxpages patch in the direction I think this code needs to move. Epic-length commentary follows. The original code came from before there was a pg_stat_bgwriter. The first patch (buf-alloc-stats)

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-04-12 Thread Heikki Linnakangas
ITAGAKI Takahiro wrote: Jim C. Nasby [EMAIL PROTECTED] wrote: Perhaps it would be better to have the bgwriter take a look at how many dead tuples (or how much space the dead tuples account for) when it writes a page out and adjust the DSM at that time. Yeah, I feel it is worth optimizable,

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-03-24 Thread Bruce Momjian
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. ---

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-03-05 Thread ITAGAKI Takahiro
Sorry, I had a mistake in the patch I sent. This is a fixed version. I wrote: I'm working on making the bgwriter to write almost of dirty pages. This is the proposal for it using automatic adjustment of bgwriter_lru_maxpages. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center

[PATCHES] Automatic adjustment of bgwriter_lru_maxpages (was: Dead Space Map version 2)

2007-03-04 Thread ITAGAKI Takahiro
Jim C. Nasby [EMAIL PROTECTED] wrote: Perhaps it would be better to have the bgwriter take a look at how many dead tuples (or how much space the dead tuples account for) when it writes a page out and adjust the DSM at that time. Yeah, I feel it is worth optimizable, too. One