date:20170514

Re: stat-time tests fix for Windows

2017-05-14 Thread Bruno Haible

Paul Eggert wrote:
> This misfeature is by design. It's called "file system tunneling". See, for 
> example:
> 
> http://dfstream.blogspot.com/2012/02/file-system-tunneling-in-windows.html

Thanks. It's worth extending the comment:


2017-05-14  Bruno Haible  

stat-time tests: Improve comment.
* tests/test-stat-time.c: Add hyperlink, from Paul Eggert.

diff --git a/tests/test-stat-time.c b/tests/test-stat-time.c
index 25fe6c3..47849de 100644
--- a/tests/test-stat-time.c
+++ b/tests/test-stat-time.c
@@ -42,7 +42,9 @@ static char filename_stamp3[50];
 /* Use file names that are different at each run.
This is necessary for test_birthtime() to pass on native Windows:
On this platform, the file system apparently remembers the creation time
-   of a file even after it is removed and created anew.  */
+   of a file even after it is removed and created anew.  See
+   "Windows NT Contains File System Tunneling Capabilities"
+     */
 static void
 initialize_filenames (void)
 {

Re: stat-time tests fix for Windows

2017-05-14 Thread Paul Eggert


Bruno Haible wrote:

Strangely enough, when you create a file, delete it, create it again (with the
same name), then the st_ctime field will contain the time of the first creation.


This misfeature is by design. It's called "file system tunneling". See, for 
example:

http://dfstream.blogspot.com/2012/02/file-system-tunneling-in-windows.html

Re: plans for file related modules on Windows

2017-05-14 Thread Ben Pfaff

On Sun, May 14, 2017 at 06:06:03PM +0200, Bruno Haible wrote:
> > I did not realize that Windows could even support a proper
> > implementation of the struct stat st_dev and st_ino.  I'd find this
> > useful in multiple programs, although in some of them I might really
> > just use the code you write as an educational resource.
> 
> I've considered your wish. This functionality is now implemented as
> module 'windows-stat-inodes'.

Thank you!  I learned a lot of useful information from reading the
module, and in addition I do expect to use the module directly.

same-inode: adapt for windows-stat-inodes

2017-05-14 Thread Bruno Haible

With this adaptation of the 'same-inode' module, the 'stat' test
now passes on native Windows. (Well, it says "SKIP" because of missing
symbolic link support.)


2017-05-14  Bruno Haible  

same-inode: Adapt for windows-stat-inodes.
* lib/same-inode.h: Include .
(SAME_INODE) [_GL_WINDOWS_STAT_INODES]: Define specifically.
* modules/same-inode (Depends-on): Add sys_types.

diff --git a/lib/same-inode.h b/lib/same-inode.h
index 7cece6d..a08bc4e 100644
--- a/lib/same-inode.h
+++ b/lib/same-inode.h
@@ -18,6 +18,8 @@
 #ifndef SAME_INODE_H
 # define SAME_INODE_H 1
 
+# include 
+
 # ifdef __VMS
 #  define SAME_INODE(a, b) \
 ((a).st_ino[0] == (b).st_ino[0]\
@@ -25,9 +27,17 @@
  && (a).st_ino[2] == (b).st_ino[2] \
  && (a).st_dev == (b).st_dev)
 # elif (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN__
-/* On MinGW, struct stat lacks necessary info, so always return 0.
-   Callers can use !a.st_ino to deduce that the information is unknown.  */
-#  define SAME_INODE(a, b) 0
+   /* Native Windows.  */
+#  if _GL_WINDOWS_STAT_INODES
+/* stat() and fstat() set st_dev and st_ino to 0 if information about
+   the inode is not available.  */
+#   define SAME_INODE(a, b) \
+ (!((a).st_ino == 0 && (a).st_dev == 0) \
+  && (a).st_ino == (b).st_ino && (a).st_dev == (b).st_dev)
+#  else
+/* stat() and fstat() set st_ino to 0 always.  */
+#   define SAME_INODE(a, b) 0
+#  endif
 # else
 #  define SAME_INODE(a, b)\
 ((a).st_ino == (b).st_ino \
diff --git a/modules/same-inode b/modules/same-inode
index 27da5d2..53964d0 100644
--- a/modules/same-inode
+++ b/modules/same-inode
@@ -5,6 +5,7 @@ Files:
 lib/same-inode.h
 
 Depends-on:
+sys_types
 
 configure.ac:

windows-stat-inodes: new module

2017-05-14 Thread Bruno Haible

This patch implements the 'windows-stat-inodes' module, that provide good
st_dev and st_ino values on native Windows - at least for most files.

On files where it cannot do so (unreadable files or directories), it sets
both fields to 0. This is not as good as Cygwin does; Cygwin provides
non-zero st_ino in these cases too. I don't know whether this is because
Cygwin uses NtOpenFile and NtQueryInformationFile instead of Windows API,
or because Cygwin uses a hash code of the file name as a fallback.
I could have used a hash code of the file name as well (say, 64 bits
from an SHA-1 hash sum), but this could lead to bugs (e.g. with symbolic
links or hard links). Therefore I find it preferable to just set
st_dev = st_ino = 0 in this case, and let the users of st_ino deal with
it with care.


2017-05-14  Bruno Haible  

windows-stat-inodes: New module.
* m4/windows-stat-inodes.m4: New file.
* m4/sys_types_h.m4 (gl_SYS_TYPES_H): Set WINDOWS_STAT_INODES.
* modules/sys_types (Makefile.am): Substitute WINDOWS_STAT_INODES.
* lib/sys_types.in.h [WINDOWS_STAT_INODES]: Override dev_t and ino_t.
(_GL_WINDOWS_STAT_INODES): New macro.
* lib/stat-w32.c: Set _WIN32_WINNT. Include , verify.h.
(GetFileInformationByHandleExFunc): New variable.
(initialize): Initialize it.
(_gl_fstat_by_handle) [_GL_WINDOWS_STAT_INODES]: Initialize st_dev and
st_ino appropriately.
* lib/stat.c (rpl_stat): Use the directory entry based approach only as
a fallback, because it does not provide st_dev and st_ino values.
* modules/fstat (Depends-on): Add 'verify'.
* modules/windows-stat-inodes: New file.
* doc/windows-stat-inodes.texi: New file.
* doc/gnulib.texi: Include it.
* doc/posix-headers/sys_stat.texi: Mention the new module.

diff --git a/doc/gnulib.texi b/doc/gnulib.texi
index 5cadf46..3eb7a1d 100644
--- a/doc/gnulib.texi
+++ b/doc/gnulib.texi
@@ -6324,6 +6324,7 @@ to POSIX that it can be treated like any other Unix-like 
platform.
 @menu
 * Libtool and Windows::
 * Large File Support::
+* Inode numbers on Windows::
 * Precise file timestamps on Windows::
 * Avoiding the year 2038 problem::
 * Windows sockets::
@@ -6334,6 +6335,8 @@ to POSIX that it can be treated like any other Unix-like 
platform.
 
 @include largefile.texi
 
+@include windows-stat-inodes.texi
+
 @include windows-stat-timespec.texi
 
 @include year2038.texi
diff --git a/doc/posix-headers/sys_stat.texi b/doc/posix-headers/sys_stat.texi
index 4c176aa..42a27ba 100644
--- a/doc/posix-headers/sys_stat.texi
+++ b/doc/posix-headers/sys_stat.texi
@@ -5,7 +5,7 @@ POSIX specification:@* 
@url{http://www.opengroup.org/onlinepubs/9699919799/based
 
 Gnulib module: sys_stat
 
-Portability problems fixed by Gnulib:
+Portability problems fixed by Gnulib module @code{sys_stat}:
 @itemize
 @item
 The type @code{mode_t} is not defined on some platforms:
@@ -31,14 +31,18 @@ On some platforms, @code{struct stat} does not include 
@code{st_atim},
 @samp{stat-time} for accessors to portably get at subsecond resolution.
 @end itemize
 
+Portability problems fixed by Gnulib module @code{sys_stat}, together with 
module @code{windows-stat-inodes}:
+@itemize
+@item
+On Windows platforms (excluding Cygwin), @code{st_ino} is always 0.
+@end itemize
+
 Portability problems not fixed by Gnulib:
 @itemize
 @item
 The macro @code{S_IFBLK} is missing on some platforms:
 MSVC 9.
 @item
-On Windows platforms (excluding Cygwin), @code{st_ino} is always 0.
-@item
 On OpenVMS, @code{st_ino} is an array of three @code{ino_t} values,
 not a single value.
 @item
diff --git a/doc/windows-stat-inodes.texi b/doc/windows-stat-inodes.texi
new file mode 100644
index 000..f3c0581
--- /dev/null
+++ b/doc/windows-stat-inodes.texi
@@ -0,0 +1,14 @@
+@node Inode numbers on Windows
+@section Inode numbers on Windows
+
+The module @samp{windows-stat-inodes} ensures that,
+on native Windows platforms, @code{struct stat} contains
+@code{st_dev}, @code{st_ino} fields that are able to distinguish
+different inodes.
+
+Note: Such values can only be provided for most files on the
+file system.  For a few files (such as inaccessible files),
+@code{st_dev} and @code{st_ino} are set to 0.  Therefore,
+you should test whether @code{st_dev != 0 && st_ino != 0},
+before going to make inferences based on the file identity
+based on @code{st_dev} and @code{st_ino}.
diff --git a/lib/stat-w32.c b/lib/stat-w32.c
index 515311d..b4c762c 100644
--- a/lib/stat-w32.c
+++ b/lib/stat-w32.c
@@ -20,10 +20,15 @@
 
 #if (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN__
 
+/* Ensure that  defines FILE_ID_INFO.  */
+#undef _WIN32_WINNT
+#define _WIN32_WINNT _WIN32_WINNT_WIN8
+
 #include 
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -31,7 +36,16 @@
 #include "stat-w32.h"
 
 #include "pathmax.h"
+#include "verify.h"
 
+#if _GL_WINDOWS_STAT_INODES == 2
+/*

stat-time tests fix for Windows

2017-05-14 Thread Bruno Haible

On native Windows, the 'test-stat-time' test works fine the first time I run it,
but fails in subsequent runs in the same build directory.

Strangely enough, when you create a file, delete it, create it again (with the
same name), then the st_ctime field will contain the time of the first creation.
(See log appended below.) Reproduced with Windows 10 on NTFS.

As a workaround/fix: Let the stat-time test create files with different names
each time.


2017-05-14  Bruno Haible  

stat-time tests: Workaround for native Windows.
* tests/test-stat-time.c: Include , .
(filename_stamp1, filename_testfile, filename_stamp2, filename_stamp3):
New variables.
(initialize_filenames): New function.
(main): Invoke it.
(cleanup, prepare_test): Update.

diff --git a/tests/test-stat-time.c b/tests/test-stat-time.c
index 7b9cebf..25fe6c3 100644
--- a/tests/test-stat-time.c
+++ b/tests/test-stat-time.c
@@ -22,8 +22,10 @@
 
 #include 
 #include 
+#include 
 #include 
 #include 
+#include 
 
 #include "macros.h"
 
@@ -32,6 +34,25 @@
 
 enum { NFILES = 4 };
 
+static char filename_stamp1[50];
+static char filename_testfile[50];
+static char filename_stamp2[50];
+static char filename_stamp3[50];
+
+/* Use file names that are different at each run.
+   This is necessary for test_birthtime() to pass on native Windows:
+   On this platform, the file system apparently remembers the creation time
+   of a file even after it is removed and created anew.  */
+static void
+initialize_filenames (void)
+{
+  long t = (long) time (NULL);
+  sprintf (filename_stamp1,   "t-stt-%ld-stamp1", t);
+  sprintf (filename_testfile, "t-stt-%ld-testfile", t);
+  sprintf (filename_stamp2,   "t-stt-%ld-stamp2", t);
+  sprintf (filename_stamp3,   "t-stt-%ld-stamp3", t);
+}
+
 static int
 force_unlink (const char *filename)
 {
@@ -45,11 +66,10 @@ static void
 cleanup (int sig)
 {
   /* Remove temporary files.  */
-  force_unlink ("t-stt-stamp1");
-  force_unlink ("t-stt-testfile");
-  force_unlink ("t-stt-stamp2");
-  force_unlink ("t-stt-renamed");
-  force_unlink ("t-stt-stamp3");
+  force_unlink (filename_stamp1);
+  force_unlink (filename_testfile);
+  force_unlink (filename_stamp2);
+  force_unlink (filename_stamp3);
 
   if (sig != 0)
 _exit (1);
@@ -87,20 +107,20 @@ prepare_test (struct stat *statinfo, struct timespec 
*modtimes)
 {
   int i;
 
-  create_file ("t-stt-stamp1");
+  create_file (filename_stamp1);
   nap ();
-  create_file ("t-stt-testfile");
+  create_file (filename_testfile);
   nap ();
-  create_file ("t-stt-stamp2");
+  create_file (filename_stamp2);
   nap ();
-  ASSERT (chmod ("t-stt-testfile", 0400) == 0);
+  ASSERT (chmod (filename_testfile, 0400) == 0);
   nap ();
-  create_file ("t-stt-stamp3");
+  create_file (filename_stamp3);
 
-  do_stat ("t-stt-stamp1",  [0]);
-  do_stat ("t-stt-testfile", [1]);
-  do_stat ("t-stt-stamp2",  [2]);
-  do_stat ("t-stt-stamp3",  [3]);
+  do_stat (filename_stamp1,   [0]);
+  do_stat (filename_testfile, [1]);
+  do_stat (filename_stamp2,   [2]);
+  do_stat (filename_stamp3,   [3]);
 
   /* Now use our access functions. */
   for (i = 0; i < NFILES; ++i)
@@ -160,7 +180,7 @@ test_ctime (const struct stat *statinfo)
   if (statinfo[0].st_mtime != statinfo[0].st_ctime)
 return;
 
-  /* mtime(stamp2) < ctime(renamed) */
+  /* mtime(stamp2) < ctime(testfile) */
   ASSERT (statinfo[2].st_mtime < statinfo[1].st_ctime
   || (statinfo[2].st_mtime == statinfo[1].st_ctime
   && (get_stat_mtime_ns ([2])
@@ -183,11 +203,11 @@ test_birthtime (const struct stat *statinfo,
 return;
 }
 
-  /* mtime(stamp1) < birthtime(renamed) */
+  /* mtime(stamp1) < birthtime(testfile) */
   ASSERT (modtimes[0].tv_sec < birthtimes[1].tv_sec
   || (modtimes[0].tv_sec == birthtimes[1].tv_sec
   && modtimes[0].tv_nsec < birthtimes[1].tv_nsec));
-  /* birthtime(renamed) < mtime(stamp2) */
+  /* birthtime(testfile) < mtime(stamp2) */
   ASSERT (birthtimes[1].tv_sec < modtimes[2].tv_sec
   || (birthtimes[1].tv_sec == modtimes[2].tv_sec
   && birthtimes[1].tv_nsec < modtimes[2].tv_nsec));
@@ -200,6 +220,8 @@ main (void)
   struct timespec modtimes[NFILES];
   struct timespec birthtimes[NFILES];
 
+  initialize_filenames ();
+
 #ifdef SIGHUP
   signal (SIGHUP, cleanup);
 #endif


=== EXPERIMENT ==

$ gltests/test-stat-time.exe
i=0 modtime=1494769524.508897000 birthtime=1494769524.508897000
i=1 modtime=1494769524.509899300 birthtime=1494769524.509899300
i=2 modtime=1494769524.517378100 birthtime=1494769524.517378100
i=3 modtime=1494769524.535423400 birthtime=1494769524.535423400
File t-stt-stamp1: ret=0, st_ctim = 1494769524.508897000
File t-stt-testfile: ret=0, st_ctim = 1494769524.509899300
File t-stt-stamp2: ret=0, st_ctim = 1494769524.517378100
File t-stt-stamp3: ret=0, st_ctim = 1494769524.535423400

$ echo $?
0

$

stat-time: adapt for windows-stat-timespec

2017-05-14 Thread Bruno Haible

One more adaptation of module 'stat-time' for the 'struct stat'
with 'struct timespec' fields on Windows:


2017-05-14  Bruno Haible  

stat-time: Adapt for windows-stat-timespec.
* lib/stat-time.h (get_stat_birthtime) [_GL_WINDOWS_STAT_TIMESPEC]: Use
entire st_ctim field.

diff --git a/lib/stat-time.h b/lib/stat-time.h
index 88dcc7f..9e45e85 100644
--- a/lib/stat-time.h
+++ b/lib/stat-time.h
@@ -170,8 +170,12 @@ get_stat_birthtime (struct stat const *st)
   /* Native Windows platforms (but not Cygwin) put the "file creation
  time" in st_ctime (!).  See
  .  */
+# if _GL_WINDOWS_STAT_TIMESPEC
+  t = st->st_ctim;
+# else
   t.tv_sec = st->st_ctime;
   t.tv_nsec = 0;
+# endif
 #else
   /* Birth time is not supported.  */
   t.tv_sec = -1;

Re: stat-time tests fix for Windows

Re: stat-time tests fix for Windows

Re: plans for file related modules on Windows

same-inode: adapt for windows-stat-inodes

windows-stat-inodes: new module

stat-time tests fix for Windows

stat-time: adapt for windows-stat-timespec

7 matches

Site Navigation

Mail list logo

Footer information