Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-04-02 Thread Robert Haas
On Wed, Mar 25, 2015 at 3:45 PM, Jim Nasby jim.na...@bluetreble.com wrote: I see 3 settings that allow people to accidentally shoot themselves in the foot; fsync, wal_sync_method and full_page_writes. Those aren't even the top three in my experience, let alone the only three. -- Robert Haas

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-26 Thread Jim Nasby
On 3/25/15 8:35 PM, Jeff Janes wrote: On Wed, Mar 25, 2015 at 12:45 PM, Jim Nasby jim.na...@bluetreble.com mailto:jim.na...@bluetreble.com wrote: I see 3 settings that allow people to accidentally shoot themselves in the foot; fsync, wal_sync_method and full_page_writes. How about

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-25 Thread Jim Nasby
On 3/21/15 12:25 PM, Jeff Janes wrote: On Sat, Mar 21, 2015 at 8:54 AM, Tom Lane t...@sss.pgh.pa.us mailto:t...@sss.pgh.pa.us wrote: Stephen Frost sfr...@snowman.net mailto:sfr...@snowman.net writes: At the moment, one could look at our default postgresql.conf and the turns forced

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-25 Thread Jeff Janes
On Wed, Mar 25, 2015 at 12:45 PM, Jim Nasby jim.na...@bluetreble.com wrote: I see 3 settings that allow people to accidentally shoot themselves in the foot; fsync, wal_sync_method and full_page_writes. How about just grouping those 3 together with a bulk disclaimer along the lines of The

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-25 Thread Jim Nasby
On 3/22/15 4:50 PM, Greg Stark wrote: On Sun, Mar 22, 2015 at 3:34 PM, Euler Taveira eu...@timbira.com.br wrote: On 21-03-2015 17:53, Josh Berkus wrote: Now, I have *long* been an advocate that we should ship a stripped PostgreSQL.conf which has only the most commonly used settings, and leave

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-23 Thread Andres Freund
On 2015-03-22 12:54:37 -0700, Josh Berkus wrote: On 03/22/2015 06:45 AM, Andres Freund wrote: FWIW, I think that's a myth. One I heard various versions of by now. As long as the OSs page size (4kb nearly everywhere) is different from postgres' (8kb) you can have torn pages. Even if

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-22 Thread Andres Freund
On 2015-03-21 13:53:47 -0700, Josh Berkus wrote: Coincidentally, I am just at this moment performance testing running with scissors mode for PostgreSQL on AWS. When intentional, this mode is useful for spinning up lots of read-only replicas which are intended mainly as cache support,

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-22 Thread Euler Taveira
On 21-03-2015 17:53, Josh Berkus wrote: Now, I have *long* been an advocate that we should ship a stripped PostgreSQL.conf which has only the most commonly used settings, and leave the rest of the settings in the docs and share/postgresql/postgresql.conf.advanced. Here's my example of such a

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-22 Thread Florian Weimer
* David G. Johnston: ​enables or disables data durability ​promise of ACID. ? “fsync = on” only works if the storage stack doesn't do funny things. Depending on the system, it might not be sufficient. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-22 Thread Josh Berkus
On 03/22/2015 06:45 AM, Andres Freund wrote: On 2015-03-21 13:53:47 -0700, Josh Berkus wrote: Coincidentally, I am just at this moment performance testing running with scissors mode for PostgreSQL on AWS. When intentional, this mode is useful for spinning up lots of read-only replicas which

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-22 Thread David G. Johnston
On Sunday, March 22, 2015, Florian Weimer f...@deneb.enyo.de wrote: * David G. Johnston: ​enables or disables data durability ​promise of ACID. ? “fsync = on” only works if the storage stack doesn't do funny things. Depending on the system, it might not be sufficient. Allows for

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-22 Thread Mark Kirkwood
On 22/03/15 08:14, Jaime Casanova wrote: El mar 21, 2015 2:00 AM, Mark Kirkwood mark.kirkw...@catalyst.net.nz mailto:mark.kirkw...@catalyst.net.nz escribió: On 21/03/15 19:28, Jaime Casanova wrote: what about not removing it but not showing it in postgresql.conf? as a side note, i

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Mark Kirkwood
On 21/03/15 19:28, Jaime Casanova wrote: On Fri, Mar 20, 2015 at 11:29 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Sat, Mar 21, 2015 at 2:47 AM, Peter Geoghegan p...@heroku.com wrote: On Fri, Mar 20, 2015 at 9:52 AM, Joshua D. Drake j...@commandprompt.com wrote: There are just as

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Jaime Casanova
On Fri, Mar 20, 2015 at 11:29 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Sat, Mar 21, 2015 at 2:47 AM, Peter Geoghegan p...@heroku.com wrote: On Fri, Mar 20, 2015 at 9:52 AM, Joshua D. Drake j...@commandprompt.com wrote: There are just as many people that are running with

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: At the moment, one could look at our default postgresql.conf and the turns forced synchronization on or off and think it's something akin or somehow related to synchronous_commit (which is completely different, but the options are right next to each

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread David Fetter
On Sat, Mar 21, 2015 at 11:54:00AM -0400, Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: At the moment, one could look at our default postgresql.conf and the turns forced synchronization on or off and think it's something akin or somehow related to synchronous_commit (which is

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread David G. Johnston
On Sat, Mar 21, 2015 at 8:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: Stephen Frost sfr...@snowman.net writes: At the moment, one could look at our default postgresql.conf and the turns forced synchronization on or off and think it's something akin or somehow related to synchronous_commit

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Jeff Janes
On Sat, Mar 21, 2015 at 8:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: Stephen Frost sfr...@snowman.net writes: At the moment, one could look at our default postgresql.conf and the turns forced synchronization on or off and think it's something akin or somehow related to synchronous_commit

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Joshua D. Drake
On 03/20/2015 04:11 PM, Jim Nasby wrote: As for why; Postgres already has a big reputation for being hard to use and hard to setup. Leaving footguns laying around that could easily be warned about is part of the reason for that reputation. Reality is that there are a lot of people using

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Joshua D. Drake
On 03/21/2015 12:45 PM, Gavin Flower wrote: How about 2 config files? One marked adult^H^H^H^H^H power users only, or some such, with the really dangerous or unusual options? That has come up before in many threads. I don't know that we need to go down that path again. Consider, power

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Gavin Flower
On 22/03/15 08:34, Joshua D. Drake wrote: On 03/21/2015 12:00 AM, Mark Kirkwood wrote: -1 Personally I'm against hiding *any* settings. Choosing sensible defaults - yes! Hiding them - that reeks of secret squirrel nonsense and overpaid Oracle dbas that knew the undocumented settings for

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Gavin Flower
On 22/03/15 08:48, Joshua D. Drake wrote: On 03/21/2015 12:45 PM, Gavin Flower wrote: How about 2 config files? One marked adult^H^H^H^H^H power users only, or some such, with the really dangerous or unusual options? That has come up before in many threads. I don't know that we need to

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Joshua D. Drake
On 03/21/2015 12:00 AM, Mark Kirkwood wrote: -1 Personally I'm against hiding *any* settings. Choosing sensible defaults - yes! Hiding them - that reeks of secret squirrel nonsense and overpaid Oracle dbas that knew the undocumented settings for various capabilities. I think/hope that no

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Joshua D. Drake
On 03/21/2015 12:32 PM, Gavin Flower wrote: What does ACID mean??? I don't want to trip out on acid, and if I do, I don't want it hanging around. Safer to set this to off!!! I actual do know what ACID means, but some 'children' have write access to a the postgresql.conf file without

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Jaime Casanova
El mar 21, 2015 2:00 AM, Mark Kirkwood mark.kirkw...@catalyst.net.nz escribió: On 21/03/15 19:28, Jaime Casanova wrote: what about not removing it but not showing it in postgresql.conf? as a side note, i wonder why trace_sort is not in postgresql.conf... other option is to make it a compile

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Gavin Flower
On 22/03/15 05:42, David G. Johnston wrote: On Sat, Mar 21, 2015 at 8:54 AM, Tom Lane t...@sss.pgh.pa.us mailto:t...@sss.pgh.pa.uswrote: Stephen Frost sfr...@snowman.net mailto:sfr...@snowman.net writes: At the moment, one could look at our default postgresql.conf and the turns

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Joshua D. Drake
On 03/20/2015 04:09 PM, Robert Haas wrote: Thus far, the rule for postgresql.conf has been that pretty much everything goes in there, and that's a defensible position. Other reasonable options would be to ship the file with a small handful of settings in it and leave everything else, or to

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Joshua D. Drake
On 03/20/2015 11:28 PM, Jaime Casanova wrote: I fought to remove fsync before so i understand JD concerns. and yes, i have seen fsync=off in the field too... what about not removing it but not showing it in postgresql.conf? as a side note, i wonder why trace_sort is not in

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Jaime Casanova
On Sat, Mar 21, 2015 at 2:33 PM, Joshua D. Drake j...@commandprompt.com wrote: On 03/20/2015 11:28 PM, Jaime Casanova wrote: I fought to remove fsync before so i understand JD concerns. and yes, i have seen fsync=off in the field too... what about not removing it but not showing it in

[HACKERS] Re[2]: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Миша Тюрин
why does we take so many attention to fsync issue? but there are also table spaces in tmpfs, wal in tmpfs, disks with cache without bbu, writeback writes and fs without ordering and journal, any CLOUDS, etc etc... in our real world installations. more over not all of these issues are usually

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Josh Berkus
On 03/20/2015 09:29 PM, Michael Paquier wrote: On Sat, Mar 21, 2015 at 2:47 AM, Peter Geoghegan p...@heroku.com wrote: On Fri, Mar 20, 2015 at 9:52 AM, Joshua D. Drake j...@commandprompt.com wrote: There are just as many people that are running with scissors that are now running (or

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Jerry Sievers
Joshua D. Drake j...@commandprompt.com writes: On 03/21/2015 12:45 PM, Gavin Flower wrote: How about 2 config files? One marked adult^H^H^H^H^H power users only, or some such, with the really dangerous or unusual options? That has come up before in many threads. I don't know that we

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-20 Thread Jim Nasby
On 3/20/15 2:49 PM, Stephen Frost wrote: How about a big warning around fsync and make it more indepenent from the options around it? +1, and the same for full_page_writes and wal_sync_method. I think that's the best we can do at this point. As for why; Postgres already has a big reputation

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-20 Thread Robert Haas
On Fri, Mar 20, 2015 at 3:26 PM, Joshua D. Drake j...@commandprompt.com wrote: Fair enough. I am not going to name names but over the years (and just today) I ran into another user that corrupted their database by turning off fsync. My experience is different than yours: I haven't found this

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-20 Thread Jim Nasby
On 3/20/15 6:09 PM, Robert Haas wrote: On Fri, Mar 20, 2015 at 3:26 PM, Joshua D. Drake j...@commandprompt.com wrote: Fair enough. I am not going to name names but over the years (and just today) I ran into another user that corrupted their database by turning off fsync. My experience is

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-20 Thread Michael Paquier
On Sat, Mar 21, 2015 at 2:47 AM, Peter Geoghegan p...@heroku.com wrote: On Fri, Mar 20, 2015 at 9:52 AM, Joshua D. Drake j...@commandprompt.com wrote: There are just as many people that are running with scissors that are now running (or attempting to run) our elephant in production. Does it

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-20 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: There are just as many people that are running with scissors that are now running (or attempting to run) our elephant in production. Evidence please. Does it make sense to remove fsync (and possibly full_page_writes) from such a visible

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-20 Thread Peter Geoghegan
On Fri, Mar 20, 2015 at 9:52 AM, Joshua D. Drake j...@commandprompt.com wrote: There are just as many people that are running with scissors that are now running (or attempting to run) our elephant in production. Does it make sense to remove fsync (and possibly full_page_writes) from such a

[HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-20 Thread Joshua D. Drake
Hello, There are just as many people that are running with scissors that are now running (or attempting to run) our elephant in production. Does it make sense to remove fsync (and possibly full_page_writes) from such a visible place as postgresql.conf? I don't think we should remove the

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-20 Thread Joshua D. Drake
On 03/20/2015 10:45 AM, Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: There are just as many people that are running with scissors that are now running (or attempting to run) our elephant in production. Evidence please. Fair enough. I am not going to name names but over

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-20 Thread Joshua D. Drake
On 03/20/2015 10:47 AM, Peter Geoghegan wrote: On Fri, Mar 20, 2015 at 9:52 AM, Joshua D. Drake j...@commandprompt.com wrote: There are just as many people that are running with scissors that are now running (or attempting to run) our elephant in production. Does it make sense to remove fsync

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-20 Thread Greg Stark
On Fri, Mar 20, 2015 at 7:29 PM, Joshua D. Drake j...@commandprompt.com wrote: I am not going to raise a huge stink or anything but it seems rather simple. ALTER SYSTEM is, if anything, more accessible and easier to do without reading comments and warnings than config files. If it were a

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-20 Thread Stephen Frost
* Joshua D. Drake (j...@commandprompt.com) wrote: On 03/20/2015 10:45 AM, Tom Lane wrote: I would object to that, because it would make it vastly more difficult to use fsync=off easily for development. How so? alter system fsync on/off (meta) restart That seems easier than editing the