[HACKERS] Buffer management improvement wiki page

2015-04-21 Thread Jim Nasby
There's been far more ideas and testing done around improving shared buffer management than I can remember, and I suspect I'm not alone in that regard. So I've created a wiki page as a place to pull this information together. I'll try and keep highlights/important links posted there, but help w

Re: [HACKERS] Buffer Management

2002-06-26 Thread Curt Sampson
On Wed, 26 Jun 2002, Tom Lane wrote: > Curt Sampson <[EMAIL PROTECTED]> writes: > > Note that your proposal of using mmap to replace sysv shared memory > > relies on the behaviour I've described too. > > True, but I was not envisioning mapping an actual file --- at least > on HPUX, the only way t

Re: [HACKERS] Buffer Management

2002-06-26 Thread Bruce Momjian
Tom Lane wrote: > Curt Sampson <[EMAIL PROTECTED]> writes: > > Note that your proposal of using mmap to replace sysv shared memory > > relies on the behaviour I've described too. > > True, but I was not envisioning mapping an actual file --- at least > on HPUX, the only way to generate an arbitra

Re: [HACKERS] Buffer Management

2002-06-26 Thread Tom Lane
Curt Sampson <[EMAIL PROTECTED]> writes: > Note that your proposal of using mmap to replace sysv shared memory > relies on the behaviour I've described too. True, but I was not envisioning mapping an actual file --- at least on HPUX, the only way to generate an arbitrary-sized shared memory regio

Re: [HACKERS] Buffer Management

2002-06-25 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Can we mmap WAL without problems? Not sure if there is any gain to it > > because we just write it and rarely read from it. > > Perhaps, but I don't see any point to it. Agreed. I have been poking around google looking for an arti

Re: [HACKERS] Buffer Management

2002-06-25 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Can we mmap WAL without problems? Not sure if there is any gain to it > because we just write it and rarely read from it. Perhaps, but I don't see any point to it. regards, tom lane ---(end of broadca

Re: [HACKERS] Buffer Management

2002-06-25 Thread Bruce Momjian
Tom Lane wrote: > Curt Sampson <[EMAIL PROTECTED]> writes: > > On Tue, 25 Jun 2002, Tom Lane wrote: > >> The other discussion seemed to be considering how to mmap individual > >> data files right into backends' address space. I do not believe this > >> can possibly work, because of loss of contro

Re: [HACKERS] Buffer Management

2002-06-25 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > This will also work well when we have non-SysV semaphore support, like > > Posix semaphores, so we would be able to run with no SysV stuff. > > You do realize that we can use Posix semaphores today? The Darwin (OS X) > port uses 'em

Re: [HACKERS] Buffer Management

2002-06-25 Thread Tom Lane
"Mario Weilguni" <[EMAIL PROTECTED]> writes: > Isn't that what msync() is for? Or is this not portable? msync can force not-yet-written changes down to disk. It does not prevent the OS from choosing to write changes *before* you invoke msync. For example, the HPUX man page for msync says:

Re: [HACKERS] Buffer Management

2002-06-25 Thread Mario Weilguni
Isn't that what msync() is for? Or is this not portable? -Ursprüngliche Nachricht- Von: Tom Lane [mailto:[EMAIL PROTECTED]] Gesendet: Dienstag, 25. Juni 2002 16:30 An: Curt Sampson Cc: J. R. Nield; Bruce Momjian; PostgreSQL Hacker Betreff: Re: [HACKERS] Buffer Management Curt Sa

Re: [HACKERS] Buffer Management

2002-06-25 Thread Tom Lane
Curt Sampson <[EMAIL PROTECTED]> writes: > On Tue, 25 Jun 2002, Tom Lane wrote: >> The other discussion seemed to be considering how to mmap individual >> data files right into backends' address space. I do not believe this >> can possibly work, because of loss of control over visibility of data

Re: [HACKERS] Buffer Management

2002-06-25 Thread Bruce Momjian
Tom Lane wrote: > Curt Sampson <[EMAIL PROTECTED]> writes: > > So, while we're at it, what's the current state of people's thinking > > on using mmap rather than shared memory for data file buffers? > > There seem to be a couple of different threads in doc/TODO.detail/mmap. > > One envisions mma

Re: [HACKERS] Buffer Management

2002-06-25 Thread Curt Sampson
On Tue, 25 Jun 2002, Tom Lane wrote: > The only thing you'd really have to think about is how to replace the > current behavior that uses shmem attach counts to discover whether any > old backends are left over from a previous crashed postmaster. I dunno > if mmap offers any comparable facility.

Re: [HACKERS] Buffer Management

2002-06-25 Thread Larry Rosenman
On Tue, 2002-06-25 at 09:09, Tom Lane wrote: > Curt Sampson <[EMAIL PROTECTED]> writes: > > So, while we're at it, what's the current state of people's thinking > > on using mmap rather than shared memory for data file buffers? > [snip] > > (Hey Marc, can one do mmap in a BSD jail?) I believe th

Re: [HACKERS] Buffer Management

2002-06-25 Thread Tom Lane
Curt Sampson <[EMAIL PROTECTED]> writes: > So, while we're at it, what's the current state of people's thinking > on using mmap rather than shared memory for data file buffers? There seem to be a couple of different threads in doc/TODO.detail/mmap. One envisions mmap as a one-for-one replacement

Re: [HACKERS] Buffer Management

2002-06-25 Thread Curt Sampson
So, while we're at it, what's the current state of people's thinking on using mmap rather than shared memory for data file buffers? I see some pretty powerful advantages to this approach, and I'm not (yet :-)) convinced that the disadvantages are as bad as people think. I think I can address most

[HACKERS] Buffer Management

2002-06-24 Thread Curt Sampson
I'm splitting off this buffer mangement stuff into a separate thread. On 24 Jun 2002, J. R. Nield wrote: > I'll back off on that. I don't know if we want to use the OS buffer > manager, but shouldn't we try to have our buffer manager group writes > together by files, and pro-actively get them ou