Re: [PATCHES] O_DIRECT for WAL writes

2005-06-04 Thread Bruce Momjian
I think the conclusion from the discussion is that O_DIRECT is in addition to the sync method, rather than in place of it, because O_DIRECT doesn't have the same media write guarantees as fsync(). Would you update the patch to do O_DIRECT in addition to O_SYNC or fsync() and see if there is a

Re: [PATCHES] O_DIRECT for WAL writes

2005-06-03 Thread Mary Edie Meredith
On Fri, 2005-06-03 at 10:37 +1000, Neil Conway wrote: On Thu, 2005-06-02 at 11:49 -0700, Mary Edie Meredith wrote: My understanding is that O_DIRECT means direct as in no buffering by the OS which implies that if you write from your buffer, the write is not going to return unless the OS

Re: [PATCHES] O_DIRECT for WAL writes

2005-06-03 Thread Bruno Wolff III
On Fri, Jun 03, 2005 at 09:43:13 -0700, Mary Edie Meredith [EMAIL PROTECTED] wrote: Looking at this from another angle, is there really any way that you can say a write is truly guaranteed in the event of a failure? I think in the end to be safe, you cannot. That's why (and I'm not

Re: [PATCHES] O_DIRECT for WAL writes

2005-06-01 Thread Mary Edie Meredith
On Mon, 2005-05-30 at 16:29 +1000, Neil Conway wrote: On Mon, 2005-05-30 at 10:59 +0900, ITAGAKI Takahiro wrote: Yes, I've tested pgbench and dbt2 and their performances have improved. The two results are as follows: 1. pgbench -s 100 on one Pentium4, 1GB mem, 2 ATA disks, Linux 2.6.8

Re: [PATCHES] O_DIRECT for WAL writes

2005-06-01 Thread Neil Conway
On Wed, 2005-06-01 at 17:08 -0700, Mary Edie Meredith wrote: I know I'm late to this discussion, and I haven't made it all the way through this thread to see if your questions on Linux writes were resolved. If you are still interested, I recommend read a very good one page description of

Re: [PATCHES] O_DIRECT for WAL writes

2005-05-30 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: I wonder if we're benchmarking the right thing, though: is opening a file with O_DIRECT sufficient to ensure that a write(2) does not return until the data has hit disk? Some googling suggests so, eg http://www.die.net/doc/linux/man/man2/open.2.html There

Re: [PATCHES] O_DIRECT for WAL writes

2005-05-30 Thread Neil Conway
On Mon, 2005-05-30 at 02:52 -0400, Tom Lane wrote: Some googling suggests so, eg http://www.die.net/doc/linux/man/man2/open.2.html Well, that claims that data is guaranteed to have been transferred, but transferred to *where* is the question :) Transferring data to the disk's buffers and then

Re: [PATCHES] O_DIRECT for WAL writes

2005-05-30 Thread Ron Mayer
Tom Lane wrote: Neil Conway [EMAIL PROTECTED] writes: is opening a file with O_DIRECT sufficient to ensure that a write(2) does not return until the data has hit disk? Some googling suggests so, eg http://www.die.net/doc/linux/man/man2/open.2.html Really? On that page I read:

Re: [PATCHES] O_DIRECT for WAL writes

2005-05-30 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: On Mon, 2005-05-30 at 02:52 -0400, Tom Lane wrote: Well, that claims that data is guaranteed to have been transferred, but transferred to *where* is the question :) Oh, I see what you are worried about. I think you are right: what the doc promises is only

Re: [PATCHES] O_DIRECT for WAL writes

2005-05-30 Thread Neil Conway
On Mon, 2005-05-30 at 11:24 -0400, Tom Lane wrote: Wouldn't count on it :-(. One thing I'm particularly worried about is buffer cache consistency: does the kernel guarantee to flush any buffers it has that overlap the O_DIRECT write operation? At least on Linux I believe the kernel guarantees

Re: [PATCHES] O_DIRECT for WAL writes

2005-05-29 Thread ITAGAKI Takahiro
Neil Conway [EMAIL PROTECTED] wrote: The patch adds a new choice open_direct to wal_sync_method. Have you looked at what the performance difference of this option is? Yes, I've tested pgbench and dbt2 and their performances have improved. The two results are as follows: 1. pgbench -s 100 on