Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-24 Thread Ants Aasma
On Tue, May 22, 2012 at 11:36 PM, Ants Aasma a...@cybertec.at wrote: ... The free list itself is a bit trickier, but if it's still necessary/useful then SC-firstFreeBuffer and buf-freeNext are in effect a linked-list stack, there should plenty of tested lock free algorithms floating around

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Amit Kapila
, May 22, 2012 7:55 PM To: Robert Haas Cc: Amit Kapila; PostgreSQL-development Subject: Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence Robert Haas robertmh...@gmail.com writes: On Tue, May 22, 2012 at 10:01 AM, Tom Lane t...@sss.pgh.pa.us wrote: Well, keep in mind

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Greg Smith
On 05/23/2012 11:36 AM, Amit Kapila wrote: Do you feel I can attempt to address this problem with some prototypes and discuss here after few days when I have some results ready. I don't think there is a clear picture yet of what benchmark to use for testing changes here. Items like Consider

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Amit Kapila
: Greg Smith [mailto:g...@2ndquadrant.com] Sent: Wednesday, May 23, 2012 10:35 PM To: pgsql-hackers@postgresql.org; amit.kap...@huawei.com Subject: Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence On 05/23/2012 11:36 AM, Amit Kapila wrote: Do you feel I can attempt to address

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Jeff Janes
On Wed, May 23, 2012 at 10:33 AM, Amit Kapila amit.kap...@huawei.com wrote: I don't think there is a clear picture yet of what benchmark to use for testing changes here. I will first try to generate such a scenario(benchmark). I have still not thought completely. However the idea in my mind is

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Jeff Janes
On Wed, May 23, 2012 at 8:36 AM, Amit Kapila amit.kap...@huawei.com wrote: And besides if the decrements are decoupled from the allocation requests it's no longer obvious that the algorithm is even an approximation of LRU. I was trying to highlight that we can do the clocksweep in bgwriter and

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Amit Kapila
, 2012 11:39 PM To: Amit Kapila Cc: Greg Smith; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence On Wed, May 23, 2012 at 10:33 AM, Amit Kapila amit.kap...@huawei.com wrote: I don't think there is a clear picture yet of what benchmark to use

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: One thing I wanted to play with is having newly read buffers get a usage count of 0 rather than 1. The problem is that there is no way to test it in enough different situations to convince people it would be a general improvement. Hmm ... ISTM that

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Amit Kapila
to generate the scenarios? -Original Message- From: Jeff Janes [mailto:jeff.ja...@gmail.com] Sent: Wednesday, May 23, 2012 11:48 PM To: Amit Kapila Cc: Tom Lane; Robert Haas; PostgreSQL-development Subject: Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence On Wed, May 23

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Jeff Janes
On Wed, May 23, 2012 at 11:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: One thing I wanted to play with is having newly read buffers get a usage count of 0 rather than 1.  The problem is that there is no way to test it in enough different situations to

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Robert Haas
On Wed, May 23, 2012 at 2:09 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Wed, May 23, 2012 at 10:33 AM, Amit Kapila amit.kap...@huawei.com wrote: I don't think there is a clear picture yet of what benchmark to use for testing changes here. I will first try to generate such a

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: On Wed, May 23, 2012 at 11:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Hmm ... ISTM that that was discussed back when we instituted buffer usage counts, and rejected on the grounds that a newly-read buffer could then have negligible life expectancy.  The

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-22 Thread Robert Haas
On Tue, May 8, 2012 at 9:37 PM, Amit Kapila amit.kap...@huawei.com wrote: I have checked the code and logic according to which usage counter is increased when the buffer is pinned. Fixed, thanks for the report. Another Doubt : Why in function BufferAlloc, it needs to hold the BufFreelistLock

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Mind you, I think this whole area of the code needs some reengineering for better performance, but I'm not sure this is the right place to start. What I think is really bad is that we're forcing every BufferAlloc() to iterate over buffers checking

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-22 Thread Robert Haas
On Tue, May 22, 2012 at 10:01 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Mind you, I think this whole area of the code needs some reengineering for better performance, but I'm not sure this is the right place to start.  What I think is really bad is that

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, May 22, 2012 at 10:01 AM, Tom Lane t...@sss.pgh.pa.us wrote: Well, keep in mind that that action is not merely there to obtain a victim buffer; it is also maintaining the global LRU state (by decrementing the usage counts of buffers it passes

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-22 Thread Robert Haas
On Tue, May 22, 2012 at 10:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: The idea would be to have a background process (like bgwriter) maintain the global LRU state and push candidate buffers onto the freelist. Amit was trying to convince me of the same idea at PGCon, but I don't buy it.  

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, May 22, 2012 at 10:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: My own thoughts about this had pointed in the direction of getting rid of the central freelist entirely, instead letting each backend run its own independent clock sweep as needed.

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-22 Thread Robert Haas
On Tue, May 22, 2012 at 12:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: With respect to the control logic for the background writer, one idea I had was to get rid of the idea that the background writer's job is to write in advance of the strategy point.  Instead, every time the clock sweep passes

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: If we're going to throw our current algorithm over wholesale, I'd rather use some approach that has been demonstrated to work well in other systems. Buffer eviction is a problem that's been around since the 1970s, and our algorithm is just about that

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-22 Thread Robert Haas
On Tue, May 22, 2012 at 12:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: If we're going to throw our current algorithm over wholesale, I'd rather use some approach that has been demonstrated to work well in other systems.  Buffer eviction is a problem

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-22 Thread Ants Aasma
On Tue, May 22, 2012 at 8:28 PM, Robert Haas robertmh...@gmail.com wrote: Totally agreed.  We're not the first people to think of this, either: CLOCK and GLOCK have been extensively studied and found to be almost as good as LRU in selecting good victim pages, but with less contention.  That's

[HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-08 Thread Amit Kapila
While going through Readme in backend\storage\buffer, I found some point misleading. Normal Buffer Replacement Strategy -- -- .. .. Each buffer header contains a usage counter, which is incremented (up to a small limit value) whenever the buffer