Steffen Daode Nurpmeso <sdao...@googlemail.com> added the comment:
I'm convinced. And i've rewritten the patch.
Now fsync(2) is always called first *regardless* of the new
optional argument. The documentation is (a little bit) better
now. And i've added support for NetBSD, AIX and Linux; though:
- for AIX i'm testing (O_SYNC && O_DSYNC) which is almost
definitely the wrong test for this, but (a) i don't know when
fsync_range() has been introduced (seems to be many years) and
(b) i've never really worked on AIX.
(I only have the documentation:
http://www.filibeto.org/unix/aix/lib/rel/5.3/basetrf1.pdf.)
- i've added sync_file_range() on Linux because of
SYNC_FILE_RANGE_WAIT_AFTER in the hope that it means something
even though the manual page says something else - but Linux and
documentation is something by itself.
http://lwn.net/Articles/178199/ states
Providing all three flags guarantees that those pages are
actually on disk when the call returns.
- it seems OpenBSD, FreeBSD, Solaris 11 and HP/UX provide data
reliability through fsync(2) alone, see e.g. the notes near EOF
of the following Solaris 11 file:
http://www.filibeto.org/sun/lib/solaris11-express-docs/2010.11/E19963_01/html/821-1464/fcntl.h-3head.html#fcntl.h-3head
----------
title: Mac OS X fsync() should really be fcntl(F_FULLFSYNC) -> Change
os.fsync() to support physical backing store syncs
Added file: http://bugs.python.org/file21748/11877.2.diff
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue11877>
_______________________________________diff --git a/Doc/library/os.rst b/Doc/library/os.rst
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -798,7 +798,7 @@
Availability: Unix.
-.. function:: fsync(fd)
+.. function:: fsync(fd, full_fsync=False)
Force write of file with filedescriptor *fd* to disk. On Unix, this calls
the
native :c:func:`fsync` function; on Windows, the MS :c:func:`_commit`
function.
@@ -807,6 +807,14 @@
``f.flush()``, and then do ``os.fsync(f.fileno())``, to ensure that all
internal
buffers associated with *f* are written to disk.
+ Note that on most operating systems :c:func:`fsync` only ensures that
+ the data is flushed to the disk device, *not* that it has been written
+ by the device itself. The optional *full_fsync* argument can be used to
+ issue a physical backing store synchronization request on operating
+ systems which do support such an operation, e.g. on NetBSD by calling
+ :c:func:`fsync_range` with the :data:`FDISKSYNC` flag, or on Mac OS X by
+ calling :c:func:`fcntl` with the :data:`F_FULLFSYNC` command.
+
Availability: Unix, and Windows.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -13,6 +13,9 @@
/* See also ../Dos/dosmodule.c */
+#ifdef __linux__
+# define _GNU_SOURCE
+#endif
#ifdef __APPLE__
/*
* Step 1 of support for weak-linking a number of symbols existing on
@@ -2119,13 +2122,74 @@
#ifdef HAVE_FSYNC
PyDoc_STRVAR(posix_fsync__doc__,
-"fsync(fildes)\n\n\
-force write of file with filedescriptor to disk.");
-
-static PyObject *
-posix_fsync(PyObject *self, PyObject *fdobj)
-{
- return posix_fildes(fdobj, fsync);
+"fsync(fildes, full_fsync=False)\n\n"
+"force write of file buffers with fildes to disk;\n"
+"if full_fsync is True it is tried to synchronize physical backing store.");
+
+static PyObject *
+posix_fsync(PyObject *self, PyObject *args, PyObject *kwargs)
+{
+ PyObject *retval;
+ auto PyObject *fdobj;
+ auto int full_fsync = 1;
+ static char *keywords[] = {"fd", "full_fsync", NULL };
+
+ retval = NULL;
+ if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|i", keywords,
+ &fdobj, &full_fsync))
+ goto jleave;
+
+ retval = posix_fildes(fdobj, fsync);
+ if (retval != Py_None)
+ goto jleave;
+
+ /* See issue 11877 discussion (and issue 11277 for OS X sparse file bug) */
+# if (((defined __AIX || defined _AIX) && \
+ defined O_SYNC && defined O_DSYNC) || \
+ (defined __APPLE__ && defined F_FULLFSYNC) || \
+ (defined __NetBSD__ && defined FDISKSYNC) || \
+ (defined __linux__ && defined SYNC_FILE_RANGE_WAIT_AFTER))
+ if (full_fsync != 0) {
+ int res, fd;
+ Py_DECREF(retval);
+
+ retval = NULL;
+ fd = PyObject_AsFileDescriptor(fdobj);
+ if (fd < 0)
+ goto jleave;
+ if (!_PyVerify_fd(fd)) {
+ retval = posix_error();
+ goto jleave;
+ }
+
+ Py_BEGIN_ALLOW_THREADS
+# if defined __AIX || defined _AIX
+ res = fsync_range(fd, O_SYNC, 0, 0);
+# endif
+# ifdef __APPLE__
+ res = fcntl(fd, F_FULLFSYNC);
+# endif
+# ifdef __NetBSD__
+ res = fsync_range(fd, FFILESYNC|FDISKSYNC, 0, 0);
+# endif
+# ifdef __linux__
+ res = sync_file_range(fd, 0, 0, (SYNC_FILE_RANGE_WAIT_BEFORE |
+ SYNC_FILE_RANGE_WRITE |
+ SYNC_FILE_RANGE_WAIT_AFTER));
+# endif
+ Py_END_ALLOW_THREADS
+
+ if (res < 0) {
+ retval = posix_error();
+ goto jleave;
+ }
+ Py_INCREF(Py_None);
+ retval = Py_None;
+ }
+# endif
+
+jleave:
+ return retval;
}
#endif /* HAVE_FSYNC */
@@ -9484,7 +9548,8 @@
{"fchdir", posix_fchdir, METH_O, posix_fchdir__doc__},
#endif
#ifdef HAVE_FSYNC
- {"fsync", posix_fsync, METH_O, posix_fsync__doc__},
+ {"fsync", (PyCFunction)posix_fsync, METH_VARARGS|METH_KEYWORDS,
+ posix_fsync__doc__},
#endif
#ifdef HAVE_SYNC
{"sync", posix_sync, METH_NOARGS, posix_sync__doc__},
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
