On 18/01/16 19:27, Pádraig Brady wrote: > On 18/01/16 18:18, Assaf Gordon wrote: >> Sorry for the last-minute post, but found 4 failures that might relate to >> GPFS file-system ( >> https://en.wikipedia.org/wiki/IBM_General_Parallel_File_System ). >> >> The system is >> CentOS 7 >> kernel 3.10.0-229.el7.x86_64 >> gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) >> >> $ df --output=source,size,used,avail,pcent,target,fstype -h . >> Filesystem Size Used Avail Use% Mounted on Type >> /dev/commons 2.8P 2.2P 598T 79% /gpfs/commons gpfs >> >> tested with >> coreutils-8.24.163-3f02d >> >> The failed tests are: >> FAIL: tests/tail-2/F-headers >> FAIL: tests/tail-2/retry > > General problem with tail really on any remote file system. > > tail: 'a' has been replaced with a remote file. giving up on this name > > We should probably disable inotify if there are no files opened, > as otherwise tail -F is unusable on remote file systems in that use case. > > I'll do further testing on NFS. > >> FAIL: tests/misc/shred-passes > > We ran out of simulated random data > (maybe due to extra buffering or something): > > shred: 'Us': end of file > > I'll look into increasing it. > >> FAIL: tests/cp/preserve-slink-time > > -2016-01-18 12:21:28.294123000 -0500 > +2016-01-18 12:21:28.296528000 -0500 > > cp -Pp didn't seem to preserve the timestamp on GPFS. > I'd need access to the system to see why. > I'd test with touch -d '...' to see if the timestamp was honored
3 patches attached that hopefully address these issues. thanks, Pádraig.
>From dbff2508b85baf12d5e2eee4134eb17be69fdca9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?P=C3=A1draig=20Brady?= <[email protected]> Date: Tue, 19 Jan 2016 13:22:42 +0000 Subject: [PATCH 1/3] tail: disable inotify with only non existent files tests/tail-2/F-headers.sh and test/tail-2/retry.sh fail on on remote file systems due to tail going into inotify mode due to not being able to determine the remoteness of the non existent files. * src/tail.c (any_non_remote_file): A new function used to disable inotify when there are no open files, as we can't determine remoteness in that case. * NEWS: Mention the bug fix. --- NEWS | 3 +++ src/tail.c | 20 ++++++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/NEWS b/NEWS index 45ee103..5983ead 100644 --- a/NEWS +++ b/NEWS @@ -30,6 +30,9 @@ GNU coreutils NEWS -*- outline -*- that specify an offset for the first field. [bug introduced with the --debug feature in coreutils-8.6] + tail -F now works with initially non existent files on a remote file system. + [bug introduced in coreutils-7.5] + ** New commands base32 is added to complement the existing base64 command, diff --git a/src/tail.c b/src/tail.c index 781adf2..2a72a93 100644 --- a/src/tail.c +++ b/src/tail.c @@ -1270,6 +1270,20 @@ any_remote_file (const struct File_spec *f, size_t n_files) return false; } +/* Return true if any of the N_FILES files in F is non remote, i.e., has + an open file descriptor and is not on a network file system. */ + +static bool +any_non_remote_file (const struct File_spec *f, size_t n_files) +{ + size_t i; + + for (i = 0; i < n_files; i++) + if (0 <= f[i].fd && ! f[i].remote) + return true; + return false; +} + /* Return true if any of the N_FILES files in F is a symlink. Note we don't worry about the edge case where "-" exists, since that will have the same consequences for inotify, @@ -2313,6 +2327,11 @@ main (int argc, char **argv) in this case because it would miss any updates to the file that were not initiated from the local system. + any_non_remote_file() checks if the user has specified any + files that don't reside on remote file systems. inotify is not used + if there are no open files, as we can't determine if those file + will be on a remote file system. + any_symlinks() checks if the user has specified any symbolic links. inotify is not used in this case because it returns updated _targets_ which would not match the specified names. If we tried to always @@ -2339,6 +2358,7 @@ main (int argc, char **argv) for one name when a name is specified multiple times. */ if (!disable_inotify && (tailable_stdin (F, n_files) || any_remote_file (F, n_files) + || ! any_non_remote_file (F, n_files) || any_symlinks (F, n_files) || (!ok && follow_mode == Follow_descriptor))) disable_inotify = true; -- 2.5.0 >From 7c6652272bbf80832b3908769a377886cf29214c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?P=C3=A1draig=20Brady?= <[email protected]> Date: Tue, 19 Jan 2016 14:58:55 +0000 Subject: [PATCH 2/3] tests: avoid false failure in shred-passes.sh * tests/misc/shred-passes.sh: Specify an exact amount to shred, to avoid running out of simulated random data on file systems with a large st_blksize like GPFS for example. --- tests/misc/shred-passes.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/misc/shred-passes.sh b/tests/misc/shred-passes.sh index f237f5a..da0946a 100755 --- a/tests/misc/shred-passes.sh +++ b/tests/misc/shred-passes.sh @@ -76,7 +76,7 @@ shred: f: removing shred: f: renamed to 0 shred: f: removed" > exp || framework_failure_ -shred -v -u -n20 --random-source=Us f 2>out || fail=1 +shred -v -u -n20 -s4096 --random-source=Us f 2>out || fail=1 compare exp out || fail=1 -- 2.5.0 >From 8293b71e040eccb05e93f2a99b5aed091a7b733e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?P=C3=A1draig=20Brady?= <[email protected]> Date: Tue, 19 Jan 2016 15:51:00 +0000 Subject: [PATCH 3/3] tests: avoid false failure in preserve-slink-time.sh on GPFS * tests/cp/preserve-slink-time.sh: Add a delay between the ln and the cp so that there is enough difference between the timestamps so GPFS won't discard the update. Reported by Assaf Gordon. --- tests/cp/preserve-slink-time.sh | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/tests/cp/preserve-slink-time.sh b/tests/cp/preserve-slink-time.sh index 6c1835c..3054d79 100755 --- a/tests/cp/preserve-slink-time.sh +++ b/tests/cp/preserve-slink-time.sh @@ -31,11 +31,21 @@ case $(stat --format=%y dangle) in ??:??:??.000000000) sleep 2;; esac -# Can't use --format=%x, as lstat() modifies atime on some platforms. -cp -Pp dangle d2 || framework_failure_ -stat --format=%y dangle > t1 || framework_failure_ -stat --format=%y d2 > t2 || framework_failure_ - -compare t1 t2 || fail=1 +copy_timestamp_() { + sleep $1 + rm -f d2 + cp -Pp dangle d2 || framework_failure_ + # Can't use --format=%x, as lstat() modifies atime on some platforms. + stat --format=%y dangle > t1 || framework_failure_ + stat --format=%y d2 > t2 || framework_failure_ + compare t1 t2 +} + +# We retry with a delay at least 1.5s because on GPFS +# it was seen that the timestamp wasn't updated unless there +# was sufficient delay between the ln and cp. +# I.e., if there wasn't sufficient difference in +# the timestamp being updated, the update was discarded. +retry_delay_ copy_timestamp_ .1 4 || fail=1 Exit $fail -- 2.5.0
