Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-23 Thread Tom Lane
Curt Sampson [EMAIL PROTECTED] writes: Back when I was working out how to do this, I reckoned that you could use mmap by keeping a write queue for each modified page. Reading, you'd have to read the datum from the page and then check the write queue for that page to see if that datum had been

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-15 Thread Kevin Brown
Tom Lane wrote: Kevin Brown [EMAIL PROTECTED] writes: Hmm...something just occurred to me about this. Would a hybrid approach be possible? That is, use mmap() to handle reads, and use write() to handle writes? Nope. Have you read the specs regarding mmap-vs-stdio synchronization?

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-15 Thread Sean Chittenden
this. The SUS text is a bit weaselly (the application must ensure correct synchronization) but the HPUX mmap man page, among others, lays it on the line: It is also unspecified whether write references to a memory region mapped with MAP_SHARED are visible to processes reading the file

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-14 Thread Simon Riggs
To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ... Folks, I'm hoping that some of you can shed some light on this. I've been trying to peg the sweet spot for shared memory using OSDL's equipment. With Jan's

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-14 Thread Josh Berkus
Simon, lots of good stuff clipped If you draw a graph of speedup (y) against cache size as a % of total database size, the graph looks like an upside-down L - i.e. the graph rises steeply as you give it more memory, then turns sharply at a particular point, after which it flattens out. The

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-14 Thread Christopher Browne
Quoth [EMAIL PROTECTED] (Simon Riggs): I say this: ARC in 8.0 PostgreSQL allows us to sensibly allocate as large a shared_buffers cache as is required by the database workload, and this should not be constrained to a small percentage of server RAM. I don't think that this particularly follows

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-14 Thread Tom Lane
Kevin Brown [EMAIL PROTECTED] writes: Hmm...something just occurred to me about this. Would a hybrid approach be possible? That is, use mmap() to handle reads, and use write() to handle writes? Nope. Have you read the specs regarding mmap-vs-stdio synchronization? Basically it says that

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-09 Thread Kevin Brown
I wrote: That said, if it's typical for many changes to made to a page internally before PG needs to commit that page to disk, then your argument makes sense, and that's especially true if we simply cannot have the page written to disk in a partially-modified state (something I can easily see

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-09 Thread Tom Lane
Kevin Brown [EMAIL PROTECTED] writes: Tom Lane wrote: mmap() is Right Out because it does not afford us sufficient control over when changes to the in-memory data will propagate to disk. ... that's especially true if we simply cannot have the page written to disk in a partially-modified

[PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-08 Thread Josh Berkus
Folks, I'm hoping that some of you can shed some light on this. I've been trying to peg the sweet spot for shared memory using OSDL's equipment. With Jan's new ARC patch, I was expecting that the desired amount of shared_buffers to be greatly increased. This has not turned out to be the

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-08 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: Here's a top-level summary: shared_buffers% RAM NOTPM20* 1000 0.2%1287 23000 5% 1507 46000 10% 1481 69000 15% 1382 92000

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-08 Thread Josh Berkus
Tom, BTW, what is the actual size of the test database (disk footprint wise) and how much of that do you think is heavily accessed during the run? It's possible that the test conditions are such that adjusting shared_buffers isn't going to mean anything anyway. The raw data is 32GB, but a

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-08 Thread Christopher Browne
[EMAIL PROTECTED] (Josh Berkus) wrote: I've been trying to peg the sweet spot for shared memory using OSDL's equipment. With Jan's new ARC patch, I was expecting that the desired amount of shared_buffers to be greatly increased. This has not turned out to be the case. That doesn't surprise