Hey, I've got 8 identical CentOS 6.5 clients that randomly keeps hanging fio when using --status-interval. I've tried fio 2.1.4 and fio 2.1.10 they both behave the same. I've also tried piping the output to tee instead of redirecting to a file. I also tried --output and specified output file, still same problem. My fio command runs through its tests flawlessly without --status-interval and exits cleanly every time. There could be anywhere from 0 to 5 clients that gets affected. Running strace on the process that seem hung yields the following output:
$ strace -p 31055 Process 31055 attached - interrupt to quit futex(0x7f346ede802c, FUTEX_WAIT, 1, NULL It will sit there forever. Excuse the splurge but this is the command (run 8 clients simultaneously where /fut1 and /fut2 is on shared storage (NFSv3)): $ fio --minimal --direct=1 --group_reporting --filesize=64m --norandommap --blocksize=4k --time_based --iodepth=1 --ramp_time=15 --ioengine=libaio --status-interval=5 --name=mytest-foo --rw=randread --numjobs=64 --filename=/fut1/6of4_1:/fut2/6of4_1:/fut1/6of4_2:/fut2/6of4_2:/fut1/6of4_3:/fut2/6of4_3:/fut1/6of4_4:/fut2/6of4_4:/fut1/6of4_5:/fut2/6of4_5:/fut1/6of4_6:/fut2/6of4_6:/fut1/6of4_7:/fut2/6of4_7:/fut1/6of4_8:/fut2/6of4_8:/fut1/6of4_9:/fut2/6of4_9:/fut1/6of4_10:/fut2/6of4_10:/fut1/6of4_11:/fut2/6of4_11:/fut1/6of4_12:/fut2/6of4_12:/fut1/6of4_13:/fut2/6of4_13:/fut1/6of4_14:/fut2/6of4_14:/fut1/6of4_15:/fut2/6of4_15:/fut1/6of4_16:/fut2/6of4_16:/fut1/6of4_17:/fut2/6of4_17:/fut1/6of4_18:/fut2/6of4_18:/fut1/6of4_19:/fut2/6of4_19:/fut1/6of4_20:/fut2/6of4_20:/fut1/6of4_21:/fut2/6of4_21:/fut1/6of4_22:/fut2/6of4_22:/fut1/6of4_23:/fut2/6of4_23:/fut1/6of4_24:/fut2/6of4_24:/fut1/6of4_25:/fut2/6of4_25:/fut1/6of4_26:/fut2/6of4_26:/fut1/6of4_27:/fut2/6of4_27:/fut1/6of4_28:/fut2/6of4_28:/fut1/6of4_29:/fut2/6of4_29:/fut1/6of4_30:/fut2/6of4_30:/fut1/6of4_31:/fut2/6of4_31:/fut1/6of4_32:/fut2/6of4_32 --runtime=60 Worth noting is that the output file is redirected to shared storage (NFSv4) on a different system than the one under test. These are the fio 2.1.4 compile options: $ ./configure Operating system Linux CPU x86_64 Big endian no Compiler gcc Cross compile no Wordsize 64 zlib no Linux AIO support yes POSIX AIO support yes POSIX AIO support needs -lrt yes POSIX AIO fsync yes Solaris AIO support no __sync_fetch_and_add yes libverbs no rdmacm no Linux fallocate yes POSIX fadvise yes POSIX fallocate yes sched_setaffinity(3 arg) yes sched_setaffinity(2 arg) no clock_gettime yes CLOCK_MONOTONIC yes CLOCK_MONOTONIC_PRECISE no gettimeofday yes fdatasync yes sync_file_range yes EXT4 move extent yes Linux splice(2) yes GUASI no Fusion-io atomic engine no libnuma no strsep yes strcasestr yes getopt_long_only() yes inet_aton yes socklen_t yes __thread yes gtk 2.18 or higher no RUSAGE_THREAD yes SCHED_IDLE yes TCP_NODELAY yes RLIMIT_MEMLOCK yes pwritev/preadv yes Regards Michael -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
