On Tue, Mar 8, 2022 at 11:22 PM Anderson, Stuart B. <s...@caltech.edu> wrote: > > parallel version 20190922 from EPEL 8 running a Rocky Linux 8 system > occasionally gets into an I/O spin loop writing 8193 bytes of "x" to a > deleted TMPDIR file and then immediately truncating it, e.g.,
This is by design: https://www.gnu.org/software/parallel/parallel_design.html#disk-full GNU Parallel does an exponential back off: https://www.gnu.org/software/parallel/parallel_design.html#exponentially-back-off So for long running jobs, this test is run once per second. For short jobs (e.g. parallel true ::: {1..1000}) it will be run much more often. I find it unlikely that the CPU usage is solely caused by the checking of disk full. You can verify this by changing: sub exit_if_disk_full() { return; ... } This will, however, remove the protection that checks if $TMPDIR runs full during a run (and in this case output may be wrong). I ran strace -tt: [pid 1000762] 23:54:08.486547 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG, NULL) = 1000775 [pid 1000762] 23:54:08.486782 write(7, "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"..., 8192) = 8192 [pid 1000762] 23:54:08.486875 write(7, "b", 1) = 1 [pid 1000762] 23:54:08.486956 ftruncate(7, 0) = 0 so the write+truncate+seek takes at most 406 microseconds. When run on my system `vmstat 1` shows that the write never reaches the disk, which is the goal. > Any thoughts on how to avoid this seemingly useless spin loop with high CPU > and I/O resource consumption? Can you show that there really is an I/O consumption? Do the writes actually reach your disk? /Ole
