Re: [HACKERS] sorted writes for checkpoints

2010-11-08 Thread Jeff Janes
On Sun, Nov 7, 2010 at 4:13 PM, Greg Smith g...@2ndquadrant.com wrote: Jeff Janes wrote: Assuming the ordering is useful, the only way the OS can do as good a job as the checkpoint code can, is if the OS stores the entire checkpoint worth of data as dirty blocks and doesn't start writing

Re: [HACKERS] sorted writes for checkpoints

2010-11-07 Thread Greg Smith
Jeff Janes wrote: Assuming the ordering is useful, the only way the OS can do as good a job as the checkpoint code can, is if the OS stores the entire checkpoint worth of data as dirty blocks and doesn't start writing until an fsync comes in. This strikes me as a pathologically configured

Re: [HACKERS] sorted writes for checkpoints

2010-11-06 Thread Jeff Janes
On Fri, Oct 29, 2010 at 6:17 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Oct 29, 2010 at 2:58 AM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Fri, Oct 29, 2010 at 3:23 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Simon's argument in the thread that the

Re: [HACKERS] sorted writes for checkpoints

2010-11-06 Thread Robert Haas
On Sat, Nov 6, 2010 at 7:25 PM, Jeff Janes jeff.ja...@gmail.com wrote: There are really two separate things here: (1) trying to do all the writes to file A before you start doing writes to file B, and (2) trying to write out blocks to each file in ascending logical block number order I'm

Re: [HACKERS] sorted writes for checkpoints

2010-10-29 Thread Heikki Linnakangas
On 29.10.2010 06:00, Jeff Janes wrote: One of the items on the Wiki ToDo list is sorted writes for checkpoints. The consensus seemed to be that this should be done by adding hook(s) into the main code, and then a contrib module to work with those hooks. Is there an existing contrib module that

Re: [HACKERS] sorted writes for checkpoints

2010-10-29 Thread Itagaki Takahiro
On Fri, Oct 29, 2010 at 3:23 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Simon's argument in the thread that the todo item points to (http://archives.postgresql.org/pgsql-patches/2008-07/msg00123.php) is basically that we don't know what the best algorithm is yet and

Re: [HACKERS] sorted writes for checkpoints

2010-10-29 Thread Robert Haas
On Fri, Oct 29, 2010 at 2:58 AM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Fri, Oct 29, 2010 at 3:23 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Simon's argument in the thread that the todo item points to

Re: [HACKERS] sorted writes for checkpoints

2010-10-29 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Simon's argument in the thread that the todo item points to (http://archives.postgresql.org/pgsql-patches/2008-07/msg00123.php) is basically that we don't know what the best algorithm is yet and benchmarking is a lot of work, so

Re: [HACKERS] sorted writes for checkpoints

2010-10-29 Thread Greg Smith
Itagaki Takahiro wrote: When I submitted the patch, I tested it on disk-based RAID-5 machine: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00541.php But there were no additional benchmarking reports at that time. We still need benchmarking before we re-examine the feature. For

[HACKERS] sorted writes for checkpoints

2010-10-28 Thread Jeff Janes
One of the items on the Wiki ToDo list is sorted writes for checkpoints. The consensus seemed to be that this should be done by adding hook(s) into the main code, and then a contrib module to work with those hooks. Is there an existing contrib module that one could best look to for inspiration

Re: [HACKERS] sorted writes for checkpoints

2010-10-28 Thread Alvaro Herrera
Excerpts from Jeff Janes's message of vie oct 29 00:00:24 -0300 2010: One of the items on the Wiki ToDo list is sorted writes for checkpoints. The consensus seemed to be that this should be done by adding hook(s) into the main code, and then a contrib module to work with those hooks. Is