Re: [PATCHES] Sorting writes during checkpoint

2008-07-10 Thread Simon Riggs
On Wed, 2008-07-09 at 21:39 -0400, Greg Smith wrote: On Fri, 4 Jul 2008, Simon Riggs wrote: No action on this seen since last commitfest, but I think we should do something with it, rather than just ignore it. Just no action worth reporting yet. Over the weekend I finally reached the

Re: [PATCHES] Sorting writes during checkpoint

2008-07-04 Thread Simon Riggs
On Mon, 2008-05-05 at 00:23 -0400, Tom Lane wrote: Greg Smith [EMAIL PROTECTED] writes: On Sun, 4 May 2008, Tom Lane wrote: Well, I tried a pgbench test similar to that one --- on smaller hardware than was reported, so it was a bit smaller test case, but it should have given similar

Re: [PATCHES] Sorting writes during checkpoint

2008-07-04 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: Anyway, I note that we don't have an easy way of sorting by tablespace, Say what? tablespace is the first component of relfilenode. but I'm sure it would be possible to look up the tablespace for a file within a plugin. If the information weren't

Re: [PATCHES] Sorting writes during checkpoint

2008-07-04 Thread Simon Riggs
On Fri, 2008-07-04 at 12:05 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Anyway, I note that we don't have an easy way of sorting by tablespace, Say what? tablespace is the first component of relfilenode. OK, thats a mistake... what about the rest of the idea? -- Simon

Re: [PATCHES] Sorting writes during checkpoint

2008-05-05 Thread Greg Smith
On Mon, 5 May 2008, Tom Lane wrote: It bothers me a bit that the patch forces writes to be done all of file A in order, then all of file B in order, etc. We don't know enough about the disk layout of the files to be sure that that's good. (This might also mean that whether there is a win is

Re: [PATCHES] Sorting writes during checkpoint

2008-05-04 Thread Greg Smith
On Sun, 4 May 2008, Tom Lane wrote: However, I am completely unable to measure any performance improvement from it. Given the possible risk of out-of-memory failures, I think the patch should not be applied without some direct proof of performance benefits, and I don't see any. Fair enough.

Re: [PATCHES] Sorting writes during checkpoint

2008-05-04 Thread Tom Lane
Greg Smith [EMAIL PROTECTED] writes: On Sun, 4 May 2008, Tom Lane wrote: However, I am completely unable to measure any performance improvement from it. Given the possible risk of out-of-memory failures, I think the patch should not be applied without some direct proof of performance

Re: [PATCHES] Sorting writes during checkpoint

2008-05-04 Thread Greg Smith
On Sun, 4 May 2008, Tom Lane wrote: Well, I tried a pgbench test similar to that one --- on smaller hardware than was reported, so it was a bit smaller test case, but it should have given similar results. My pet theory on cases where sorting will help suggests you may need a write-caching

Re: [PATCHES] Sorting writes during checkpoint

2008-05-04 Thread Tom Lane
Greg Smith [EMAIL PROTECTED] writes: On Sun, 4 May 2008, Tom Lane wrote: Well, I tried a pgbench test similar to that one --- on smaller hardware than was reported, so it was a bit smaller test case, but it should have given similar results. ... If you're not offloading to another device

Re: [PATCHES] Sorting writes during checkpoint

2008-05-03 Thread Tom Lane
ITAGAKI Takahiro [EMAIL PROTECTED] writes: Greg Smith [EMAIL PROTECTED] wrote: If shared_buffers(=NBuffers) is set to something big, this could give some memory churn. And I think it's a bad idea to allocate something this large at checkpoint time, because what happens if that fails?

Re: [PATCHES] Sorting writes during checkpoint

2008-04-16 Thread Greg Smith
On Wed, 16 Apr 2008, ITAGAKI Takahiro wrote: Dirty region of database was probably larger than disk controller's cache. Might be worthwhile to run with log_checkpoints on and collect some statistics there next time you're running these tests. It's a good habit to get other testers into

[PATCHES] Sorting writes during checkpoint

2008-04-15 Thread ITAGAKI Takahiro
Here is a patch for TODO item, Consider sorting writes during checkpoint. It writes dirty buffers in the order of block number during checkpoint so that buffers are written sequentially. I proposed the patch before, but it was rejected because 8.3 feature has been frozen already at that time.

Re: [PATCHES] Sorting writes during checkpoint

2008-04-15 Thread Greg Smith
On Tue, 15 Apr 2008, ITAGAKI Takahiro wrote: 2x Quad core Xeon, 16GB RAM, 4x HDD (RAID-0) What is the disk controller in this system? I'm specifically curious about what write cache was involved, so I can get a better feel for the hardware your results came from. I'm busy rebuilding my

Re: [PATCHES] Sorting writes during checkpoint

2008-04-15 Thread ITAGAKI Takahiro
Greg Smith [EMAIL PROTECTED] wrote: On Tue, 15 Apr 2008, ITAGAKI Takahiro wrote: 2x Quad core Xeon, 16GB RAM, 4x HDD (RAID-0) What is the disk controller in this system? I'm specifically curious about what write cache was involved, so I can get a better feel for the hardware your