Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2011-01-15 Thread Bruce Momjian
Josh Berkus wrote: > On 12/6/10 6:13 PM, Tom Lane wrote: > > Josh Berkus writes: > >> OK, patch coming then. Right now test_fsync aborts when O_DIRECT fails. > >> What should I have it do instead? > > > > Report that it fails, and keep testing the other methods. > > Patch attached. Includes a

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-23 Thread Josh Berkus
Greg, All: Results for Solaris 10u8, on ZFS on a 7-drive attached storage array: bash-3.00# ./test_fsync -f /dbdata/pgdata/test.out Loops = 1 Simple write: 8k write 59988.002/second Compare file sync methods using one write: open_datasync 8k write

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-23 Thread Josh Berkus
> Which means if we just export the macros, we would still not have caught > this. I would like to share all the defines --- I am just saying it > isn't trivial. I just called all the define variables manually rather than relying on the macros. Seemed to work fine. --

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-22 Thread Bruce Momjian
Tom Lane wrote: > Josh Berkus writes: > > Making it support O_DIRECT would be possible but more complex; I don't > > see the point unless we think we're going to have open_sync_with_odirect > > as a seperate option. > > Whether it's complex or not isn't really the issue. The issue is that > what

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-22 Thread Bruce Momjian
Josh Berkus wrote: > On 12/6/10 6:13 PM, Tom Lane wrote: > > Josh Berkus writes: > >> OK, patch coming then. Right now test_fsync aborts when O_DIRECT fails. > >> What should I have it do instead? > > > > Report that it fails, and keep testing the other methods. > > Patch attached. Includes a

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-09 Thread Josh Berkus
On 12/6/10 6:13 PM, Tom Lane wrote: > Josh Berkus writes: >> OK, patch coming then. Right now test_fsync aborts when O_DIRECT fails. >> What should I have it do instead? > > Report that it fails, and keep testing the other methods. Patch attached. Includes a fair amount of comment cleanup, si

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-06 Thread Tom Lane
Josh Berkus writes: > OK, patch coming then. Right now test_fsync aborts when O_DIRECT fails. > What should I have it do instead? Report that it fails, and keep testing the other methods. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-06 Thread Josh Berkus
> Whether it's complex or not isn't really the issue. The issue is that > what test_fsync is testing had better match what the backend does, or > people will be making choices based on not-comparable test results. > I think we should have test_fsync just automatically fold in O_DIRECT > the same

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-06 Thread Andrew Dunstan
On 12/06/2010 08:38 PM, Tom Lane wrote: Josh Berkus writes: Making it support O_DIRECT would be possible but more complex; I don't see the point unless we think we're going to have open_sync_with_odirect as a seperate option. Whether it's complex or not isn't really the issue. The issue is

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-06 Thread Tom Lane
Josh Berkus writes: > Making it support O_DIRECT would be possible but more complex; I don't > see the point unless we think we're going to have open_sync_with_odirect > as a seperate option. Whether it's complex or not isn't really the issue. The issue is that what test_fsync is testing had bet

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-06 Thread Josh Berkus
On 12/5/10 2:12 PM, Greg Smith wrote: > Josh Berkus wrote: >> I modified test_fsync in two ways to run this; first, to make it support >> O_DIRECT, and second to make it run in the *current* directory. > > Patch please? I agree with the latter change; what test_fsync does is > surprising. Attach

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-05 Thread Greg Smith
Josh Berkus wrote: I modified test_fsync in two ways to run this; first, to make it support O_DIRECT, and second to make it run in the *current* directory. Patch please? I agree with the latter change; what test_fsync does is surprising. I suggested a while ago that we refactor test_fsync t

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-04 Thread Josh Berkus
All, While I have this machine available I've been trying to run some performance tests using pgbench and various wal_sync_methods. However, I seem to be maxing out at the speed of pgbench itself; no matter which wal_sync_method I use (including "fsync"), it tops out at around 2750 TPS. Of

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-03 Thread Josh Berkus
All, So, this week I've had my hands on a medium-high-end test system where I could test various wal_sync_methods. This is a 24-core Intel Xeon machine with 72GB of ram, and 8 internal 10K SAS disks attached to a raid controller with 512MB BBU write cache. 2 of the disks are in a RAID1, which su

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-14 Thread Greg Smith
Marti Raudsepp wrote: PostgreSQL's default settings change when built with Linux kernel headers 2.6.33 or newer. As discussed on the pgsql-performance list, this causes a significant performance regression: http://archives.postgresql.org/pgsql-performance/2010-10/msg00602.php NB! I am not propos

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-07 Thread Greg Smith
Tom Lane wrote: I'm hoping that Greg Smith will take the lead on testing this, since he seems to have spent the most time in the area so far. It's not coincidence that the chapter of my book I convinced the publisher to release as a sample is the one that covers this area; this mess has be

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Josh Berkus
On 11/5/10 3:31 PM, Tom Lane wrote: > Josh Berkus writes: >> What plaforms do we need to test to get a reasonable idea? Solaris, >> FreeBSD, Windows? > > At least. I'm hoping that Greg Smith will take the lead on testing > this, since he seems to have spent the most time in the area so far. I c

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Tom Lane
Josh Berkus writes: > What plaforms do we need to test to get a reasonable idea? Solaris, > FreeBSD, Windows? At least. I'm hoping that Greg Smith will take the lead on testing this, since he seems to have spent the most time in the area so far. regards, tom lane -- Se

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Josh Berkus
> I think the original idea was that if you had a dedicated WAL drive then > sync-on-write would be reasonable. But that was a very long time ago > and I'm not sure that the system's behavior is anything like what it was > then; for that matter I'm not sure we had proof that it was an optimal > c

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Andres Freund
On Friday 05 November 2010 22:53:37 Greg Smith wrote: > > If open_dsync is so bad for performance on Linux, maybe it's bad > > everywhere? Should we be rethinking the default preference order? > > > > > > And I've seen the expected sync write performance gain over fdatasync on > a system with

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Tom Lane
Andres Freund writes: > On Friday 05 November 2010 19:13:47 Tom Lane wrote: >> If open_dsync is so bad for performance on Linux, maybe it's bad >> everywhere? Should we be rethinking the default preference order? > I fail to see how it could be beneficial on *any* non-buggy platform. > Especiall

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Greg Smith
Tom Lane wrote: If open_dsync is so bad for performance on Linux, maybe it's bad everywhere? Should we be rethinking the default preference order? And I've seen the expected sync write performance gain over fdatasync on a system with a battery-backed cache running VxFS on Linux, because w

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Marti Raudsepp
On Fri, Nov 5, 2010 at 22:16, Tom Lane wrote: > I don't really think that "change the default on Linux" is that > much nicer than "change the default everywhere" when it comes to > what we ought to consider back-patching.  In any case, you're getting > ahead of the game: we need to decide on the d

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Andres Freund
On Friday 05 November 2010 19:13:47 Tom Lane wrote: > Marti Raudsepp writes: > > PostgreSQL's default settings change when built with Linux kernel > > headers 2.6.33 or newer. As discussed on the pgsql-performance list, > > this causes a significant performance regression: > > http://archives.post

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Tom Lane
Marti Raudsepp writes: > On Fri, Nov 5, 2010 at 21:20, Tom Lane wrote: >> What's that got to do with it? > I'm not sure what you're asking. > Surely changing the default wal_sync_method for all OSes in > maintenance releases is out of the question, no? Well, if we could leave well enough alone

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Marti Raudsepp
On Fri, Nov 5, 2010 at 21:20, Tom Lane wrote: > Marti Raudsepp writes: >> On Fri, Nov 5, 2010 at 20:13, Tom Lane wrote: >>> If open_dsync is so bad for performance on Linux, maybe it's bad >>> everywhere?  Should we be rethinking the default preference order? > >> So I think we should aim to fix

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Tom Lane
Marti Raudsepp writes: > On Fri, Nov 5, 2010 at 20:13, Tom Lane wrote: >> If open_dsync is so bad for performance on Linux, maybe it's bad >> everywhere?  Should we be rethinking the default preference order? > So I think we should aim to fix old versions first. Do you disagree? What's that go

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Marti Raudsepp
On Fri, Nov 5, 2010 at 20:13, Tom Lane wrote: > I'm less than convinced this is the right approach ... > > If open_dsync is so bad for performance on Linux, maybe it's bad > everywhere?  Should we be rethinking the default preference order? Sure, maybe for PostgreSQL 9.1 But the immediate proble

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Tom Lane
Marti Raudsepp writes: > PostgreSQL's default settings change when built with Linux kernel > headers 2.6.33 or newer. As discussed on the pgsql-performance list, > this causes a significant performance regression: > http://archives.postgresql.org/pgsql-performance/2010-10/msg00602.php > NB! I am

[HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-11-05 Thread Marti Raudsepp
Hi list, PostgreSQL's default settings change when built with Linux kernel headers 2.6.33 or newer. As discussed on the pgsql-performance list, this causes a significant performance regression: http://archives.postgresql.org/pgsql-performance/2010-10/msg00602.php NB! I am not proposing to change