On Thu, Mar 10, 2022 at 4:24 AM Anderson, Stuart B. <s...@caltech.edu> wrote: > > On Mar 9, 2022, at 4:04 PM, Ole Tange <o...@tange.dk> wrote: > > On Wed, Mar 9, 2022 at 7:21 PM Anderson, Stuart B. <s...@caltech.edu> wrote: > >>> On Mar 8, 2022, at 3:35 PM, Ole Tange <o...@tange.dk> wrote: > >>> On Tue, Mar 8, 2022 at 11:22 PM Anderson, Stuart B. <s...@caltech.edu> > >>> wrote: > >>>> parallel version 20190922 from EPEL 8 running a Rocky Linux 8 system > >>>> occasionally gets into an I/O spin loop writing 8193 bytes of "x" to a > >>>> deleted TMPDIR file and then immediately truncating it, e.g., > >>> This is by design: > >>> https://www.gnu.org/software/parallel/parallel_design.html#disk-full > >> Any reason not to call statvfs() to see if there is free disk space? > > It is unclear to me if statvfs is supported on all supported > > platforms. You should feel free to examine this. > > statvs is available for FreeBSD, OpenBSD, and Linux. What other > platforms need to be checked?
qnx solaris hpux aix scosysv macosx netbsd hurd miros unixware openindiana tru64 > > If it is not supported or if it requires installing more than the > > basic Perl package, you will need very convincing evidence. > > I forgot parallel is a Perl app, but the statvfs() function from [g]libc > that provides information about a mounted filesystem, including free space > is available from https://metacpan.org/pod/Filesys::Statvfs A key feature of GNU Parallel is the one-file-design: https://www.gnu.org/software/parallel/parallel_design.html#one-file-program and that it will run on even ancient hardware: https://www.gnu.org/software/parallel/parallel_design.html#old-perl-style Filesys::Statvfs is not part of a normal perl distribution - at least not for old systems. This makes it a potential nightmare, because I would have to include it into the one-file. If the interface changes over time (say, 10 years), I will have to support that, too. > And if that is problematic for some reason there must be a portable > perl module that reports on how much free space there is in a filesystem. The problem lies in that it must be a perl module included in a normal perl installation. Other than that, I would imagine https://metacpan.org/pod/Filesys::DfPortable might be a relevant choice, but from a one-file perspective this seems horrible, because it includes compiled code (XS). > I am not deliberately running any short lived jobs (see below). Great. The documentation you have provided convinces me. Can you also confirm the issue disappears if you change: sub exit_if_disk_full() { to: sub exit_if_disk_full() { return; which will disable the disk full check. Your environment is different from mine, and I cannot reproduce the issue on the systems I have access to. Therefore please see if you can follow: https://www.gnu.org/software/parallel/man.html#bug-dependent-on-environment Right now I have the feeling something funny is going on in your environment. E.g. /proc/sys/vm/dirty_*_centisecs is set to 0 or similar. /Ole
