Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Michael Paquier
On Sat, Nov 23, 2013 at 8:06 AM, Peter Geoghegan wrote: > On Fri, Nov 22, 2013 at 2:57 PM, Bruce Momjian wrote: >> The program is diskchecker: >> >> http://brad.livejournal.com/2116715.html >> >> I got the author to re-host the source code on github a few years ago. > > It might be worth

Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Bruce Momjian
On Fri, Nov 22, 2013 at 03:27:29PM -0800, Josh Berkus wrote: > On 11/22/2013 03:23 PM, Bruce Momjian wrote: > > On Fri, Nov 22, 2013 at 03:06:31PM -0800, Peter Geoghegan wrote: > >> On Fri, Nov 22, 2013 at 2:57 PM, Bruce Momjian wrote: > >>> The program is diskchecker: > >>> > >>> http://b

Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Josh Berkus
On 11/22/2013 03:23 PM, Bruce Momjian wrote: > On Fri, Nov 22, 2013 at 03:06:31PM -0800, Peter Geoghegan wrote: >> On Fri, Nov 22, 2013 at 2:57 PM, Bruce Momjian wrote: >>> The program is diskchecker: >>> >>> http://brad.livejournal.com/2116715.html >>> >>> I got the author to re-host the

Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Bruce Momjian
On Fri, Nov 22, 2013 at 03:06:31PM -0800, Peter Geoghegan wrote: > On Fri, Nov 22, 2013 at 2:57 PM, Bruce Momjian wrote: > > The program is diskchecker: > > > > http://brad.livejournal.com/2116715.html > > > > I got the author to re-host the source code on github a few years ago. > > It m

Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Peter Geoghegan
On Fri, Nov 22, 2013 at 2:57 PM, Bruce Momjian wrote: > The program is diskchecker: > > http://brad.livejournal.com/2116715.html > > I got the author to re-host the source code on github a few years ago. It might be worth re-implementing this for -contrib. The fact that we mention diskche

Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Bruce Momjian
On Fri, Nov 22, 2013 at 11:16:06AM -0500, Tom Lane wrote: > Greg Stark writes: > > On Thu, Nov 21, 2013 at 1:43 AM, Tom Lane wrote: > >> Also, it's not that hard to do plug-pull testing to verify that your > >> system is telling the truth about fsync. This really ought to be part > >> of accepta

Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Claudio Freire
On Fri, Nov 22, 2013 at 1:16 PM, Tom Lane wrote: >> The original mail was referencing a problem with syncing *meta* data >> though. The semantics around meta data syncs are much less clearly >> specified, in part because file systems traditionally made nearly all meta >> data operations synchronou

Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Tom Lane
Greg Stark writes: > On Thu, Nov 21, 2013 at 1:43 AM, Tom Lane wrote: >> Also, it's not that hard to do plug-pull testing to verify that your >> system is telling the truth about fsync. This really ought to be part >> of acceptance testing for any new DB server. > I've never tried it but I alwa

Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Greg Stark
On Thu, Nov 21, 2013 at 1:43 AM, Tom Lane wrote: > Also, it's not that hard to do plug-pull testing to verify that your > system is telling the truth about fsync. This really ought to be part > of acceptance testing for any new DB server. > I've never tried it but I always wondered how easy it

Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Florian Weimer
On 11/21/2013 12:45 AM, Craig Ringer wrote: I'm really concerned by this post on Linux's fsync and disk flush behaviour: http://milek.blogspot.com.au/2010/12/linux-osync-and-write-barriers.html and seeking opinions from folks here who've been deeply involved in write reliability work. With ex

Re: [HACKERS] Can we trust fsync?

2013-11-20 Thread Joshua D. Drake
On 11/20/2013 03:45 PM, Craig Ringer wrote: I'm really concerned by this post on Linux's fsync and disk flush behaviour: http://milek.blogspot.com.au/2010/12/linux-osync-and-write-barriers.html and seeking opinions from folks here who've been deeply involved in write reliability work. The am

Re: [HACKERS] Can we trust fsync?

2013-11-20 Thread Tom Lane
Craig Ringer writes: > The amount of change in write reliablity behaviour in Linux across > kernel versions, file systems and storage abstraction layers is worrying > - different results for LVM vs !LVM, md vs !md, ext3 vs other, etc. Well, we pretty much *have to* trust fsync --- there's not a l

Re: [HACKERS] Can we trust fsync?

2013-11-20 Thread Tatsuo Ishii
> On 11/21/2013 07:45 AM, Craig Ringer wrote: >> I'm really concerned by this post on Linux's fsync and disk flush behaviour: >> >> http://milek.blogspot.com.au/2010/12/linux-osync-and-write-barriers.html > > ... and yes, I realise that's partly why we have the "fsync" param to > control differen

Re: [HACKERS] Can we trust fsync?

2013-11-20 Thread Craig Ringer
On 11/21/2013 07:45 AM, Craig Ringer wrote: > I'm really concerned by this post on Linux's fsync and disk flush behaviour: > > http://milek.blogspot.com.au/2010/12/linux-osync-and-write-barriers.html ... and yes, I realise that's partly why we have the "fsync" param to control different sync mode