Re: Windows now has fdatasync()

2022-08-09 Thread Thomas Munro
David kindly ran some tests of this thing on real hardware. The results were mostly in line with expectations, but we learned some new things. TL;DR We probably should consider this as a safer default, but it'd be good for someone more hands-on with this OS and knowledgeable about storage to

Re: Windows now has fdatasync()

2022-07-21 Thread Thomas Munro
Hearing no objection, I committed the patch to remove O_FSYNC. The next cleanup one I'll just leave here for now. From a50957d4c079a3703c9f8074287c7edb5654022c Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Wed, 20 Jul 2022 14:10:37 +1200 Subject: [PATCH v3] Remove fdatasync configure probe.

Re: Windows now has fdatasync()

2022-07-19 Thread Thomas Munro
On Wed, Jul 20, 2022 at 4:14 PM Thomas Munro wrote: > On Wed, Jul 20, 2022 at 4:08 PM David Rowley wrote: > > On Wed, 20 Jul 2022 at 15:22, Thomas Munro wrote: > > > Ok, I've pushed the Windows patch. I'll watch the build farm to see > > > if I've broken any of the frankentoolchain Windows

Re: Windows now has fdatasync()

2022-07-19 Thread Thomas Munro
On Wed, Jul 20, 2022 at 4:08 PM David Rowley wrote: > On Wed, 20 Jul 2022 at 15:22, Thomas Munro wrote: > > Ok, I've pushed the Windows patch. I'll watch the build farm to see > > if I've broken any of the frankentoolchain Windows animals. > > Just to get in there before the farm does... I just

Re: Windows now has fdatasync()

2022-07-19 Thread David Rowley
On Wed, 20 Jul 2022 at 15:22, Thomas Munro wrote: > Ok, I've pushed the Windows patch. I'll watch the build farm to see > if I've broken any of the frankentoolchain Windows animals. Just to get in there before the farm does... I just got a boatload of redefinition of HAVE_FDATASYNC warnings. I

Re: Windows now has fdatasync()

2022-07-19 Thread Thomas Munro
Ok, I've pushed the Windows patch. I'll watch the build farm to see if I've broken any of the frankentoolchain Windows animals. Mikael kindly upgraded conchuela, so that leaves just prairiedog without fdatasync. I've attached a patch to drop the configure probe for that once prairiedog's host

Re: Windows now has fdatasync()

2022-07-19 Thread Tom Lane
Thomas Munro writes: > The reason we need it for macOS is that they have had fdatasync > function for many years now, and configure detects it, but they > haven't ever declared it in a header, so we (accidentally?) do it in > c.h. We didn't set that up for Apple! The commit that added it was >

Re: Windows now has fdatasync()

2022-07-18 Thread Thomas Munro
On Tue, Jul 19, 2022 at 4:54 PM Michael Paquier wrote: > Do you still need HAVE_DECL_FDATASYNC? I guess so, because that is currently used for macOS, and with this patch would also be used to control the declaration for Windows. The alternative would be to explicitly test for WIN32 or

Re: Windows now has fdatasync()

2022-07-18 Thread Michael Paquier
On Mon, Jul 18, 2022 at 03:26:36PM +1200, Thomas Munro wrote: > My plan now is to commit this patch so that problem #1 is solved, prod > conchuela's owner to upgrade to solve #2, and wait until Tom shuts > down prairiedog to solve #3. Then we could consider removing the > HAVE_FDATASYNC probe and

Re: Windows now has fdatasync()

2022-07-17 Thread Tom Lane
Thomas Munro writes: > ... I was just noting an upcoming > opportunity to remove the configure/meson probes for fdatasync, which > made me feel better about the slightly kludgy way this patch is > defining HAVE_FDATASYNC explicitly on Windows. Hm. There is certainly not any harm in the meson

Re: Windows now has fdatasync()

2022-07-17 Thread Thomas Munro
On Mon, Jul 18, 2022 at 3:43 PM Tom Lane wrote: > Thomas Munro writes: > > With my garbage collector hat on, I see that all systems we target > > have fdatasync(), except: > > > 1. Windows, but this patch supplies src/port/fdatasync.c. > > 2. DragonflyBSD before 6.1. We have 6.0 in the build

Re: Windows now has fdatasync()

2022-07-17 Thread Tom Lane
Thomas Munro writes: > With my garbage collector hat on, I see that all systems we target > have fdatasync(), except: > 1. Windows, but this patch supplies src/port/fdatasync.c. > 2. DragonflyBSD before 6.1. We have 6.0 in the build farm. > 3. Ancient macOS. Current releases have it, though

Re: Windows now has fdatasync()

2022-07-17 Thread Thomas Munro
On Fri, Apr 8, 2022 at 7:56 PM Dave Page wrote: > Windows 8 was a pretty unpopular release, so I would expect shifting to > 10/2016+ for PG 16 would be unlikely to be a major problem. Thanks to Michael for making that happen. That removes the main thing I didn't know how to deal with in this

Re: Windows now has fdatasync()

2022-04-08 Thread Dave Page
On Fri, 8 Apr 2022 at 05:41, Tom Lane wrote: > Michael Paquier writes: > > On Fri, Apr 08, 2022 at 02:56:15PM +1200, Thomas Munro wrote: > >> I propose that we drop support for Windows versions older than > >> 10/Server 2016 in the PostgreSQL 16 cycle, > > Do we have any data on what people are

Re: Windows now has fdatasync()

2022-04-08 Thread Michael Paquier
On Fri, Apr 08, 2022 at 12:40:55AM -0400, Tom Lane wrote: > As long as the C11-isms are in MSVC-only code, it seems like this is > exactly equivalent to setting a minimum MSVC version. I don't see > an objection-in-principle there, it's just a practical question of > how far back is reasonable to

Re: Windows now has fdatasync()

2022-04-07 Thread Tom Lane
Michael Paquier writes: > On Fri, Apr 08, 2022 at 02:56:15PM +1200, Thomas Munro wrote: >> I propose that we drop support for Windows versions older than >> 10/Server 2016 in the PostgreSQL 16 cycle, Do we have any data on what people are actually using? > Do you think that we could raise the

Re: Windows now has fdatasync()

2022-04-07 Thread Michael Paquier
On Fri, Apr 08, 2022 at 02:56:15PM +1200, Thomas Munro wrote: > I propose that we drop support for Windows versions older than > 10/Server 2016 in the PostgreSQL 16 cycle, because the OS patches for > everything older come to an end in October next year[1], and we have a > lot of patches relating

Re: Windows now has fdatasync()

2022-04-07 Thread Thomas Munro
I've bumped this to the next cycle, so I can hopefully skip the missing version detection stuff that I have no way to test (no CI, no build farm, and I have zero interest in dumpster diving for Windows 7 or whatever installations). I propose that we drop support for Windows versions older than

Re: Windows now has fdatasync()

2022-02-10 Thread Thomas Munro
On Sun, Feb 6, 2022 at 7:20 PM Michael Paquier wrote: > On Sun, Dec 12, 2021 at 03:48:10PM +1300, Thomas Munro wrote: > > I tried out a quick POC patch and it runs a bit faster than fsync(), as > > expected. > > Good news, as a too high difference would be suspect :) > > How much difference does

Re: Windows now has fdatasync()

2022-02-05 Thread Michael Paquier
On Sun, Dec 12, 2021 at 03:48:10PM +1300, Thomas Munro wrote: > I tried out a quick POC patch and it runs a bit faster than fsync(), as > expected. Good news, as a too high difference would be suspect :) How much difference does it make in % and are the numbers rather reproducible? Just

Re: Windows now has fdatasync()

2022-02-04 Thread Thomas Munro
On Sat, Feb 5, 2022 at 12:54 PM Robert Haas wrote: > On Fri, Feb 4, 2022 at 4:24 PM Thomas Munro wrote: > > I'm not proposing we change our default to this new level, because it > > doesn't work on non-NTFS, an annoying complication. This patch would > > just provide something faster to put

Re: Windows now has fdatasync()

2022-02-04 Thread Robert Haas
On Fri, Feb 4, 2022 at 4:24 PM Thomas Munro wrote: > I'm not proposing we change our default to this new level, because it > doesn't work on non-NTFS, an annoying complication. This patch would > just provide something faster to put after "Alternatively". Hmm. I thought NTFS had kind of won the

Re: Windows now has fdatasync()

2022-02-04 Thread Thomas Munro
On Sat, Feb 5, 2022 at 2:10 AM Robert Haas wrote: > So my impression is that today we ship defaults that are unsafe on > Windows. I don't really understand much of what you are saying here, > but if there's a way we can stop doing that, +1 from me, especially if > it allows us to retain

Re: Windows now has fdatasync()

2022-02-04 Thread Robert Haas
On Sun, Dec 12, 2021 at 4:32 PM Thomas Munro wrote: > One reason to consider developing this further is the problem > discussed in the aptly named thread "Loaded footgun open_datasync on > Windows"[1] (not the problem that was fixed in pg_test_fsync, but the > problem with cache control, or lack

Re: Windows now has fdatasync()

2022-02-03 Thread Thomas Munro
I added a commitfest entry for this to try to attract Windows-hacker reviews. I wondered about adjusting it to run on older systems, but I think we're about ready to drop support for Windows < 10 anyway, so maybe I'll go and propose that separately, instead.

Re: Windows now has fdatasync()

2021-12-12 Thread Thomas Munro
On Sun, Dec 12, 2021 at 3:48 PM Thomas Munro wrote: > [...] I > tried out a quick POC patch and it runs a bit faster than fsync(), as > expected. I'm not sure if it's worth bothering with or not given the > other options, but figured it was worth sharing. One reason to consider developing this

Re: Windows now has fdatasync()

2021-12-11 Thread Sascha Kuhl
Great. File sync is a Nice extension for me, as I don't know all file structures. Thomas Munro schrieb am So., 12. Dez. 2021, 03:48: > Hi, > > While porting some new IO code to lots of OSes I noticed in passing > that there is now a way to do synchronous fdatasync() on Windows. > This mechanism

Windows now has fdatasync()

2021-12-11 Thread Thomas Munro
Hi, While porting some new IO code to lots of OSes I noticed in passing that there is now a way to do synchronous fdatasync() on Windows. This mechanism doesn't have an async variant, which is what I was actually looking for (which turns out to doable with bleeding edge IoRings, more on that