On 2014-07-21 22:25, Vasily Tarasov wrote:
Hi Jens,
I tried your patch, but it didn't help. Interestingly, the number of
threads changes in the end. At first, during the run:
# ps -eLf | grep fio
root 5224 4274 5224 1 2 11:12 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5225 0 2 11:12 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5231 5224 5231 60 1 11:12 ? 00:00:07 fio
--status-interval 10 --minimal fios/1.fio
root 5260 5237 5260 0 1 11:12 pts/0 00:00:00 grep fio
[root@bison01 vass]# ps -eLf | grep fio
root 5224 4274 5224 0 2 11:12 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5225 0 2 11:12 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5231 5224 5231 16 1 11:12 ? 00:00:21 fio
--status-interval 10 --minimal fios/1.fio
root 5293 5237 5293 0 1 11:14 pts/0 00:00:00 grep fio
[root@bison01 vass]# ps -eLf | grep fio
root 5224 4274 5224 0 2 11:12 pts/1 00:00:01 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5225 0 2 11:12 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5231 5224 5231 12 1 11:12 ? 00:01:13 fio
--status-interval 10 --minimal fios/1.fio
root 5411 5237 5411 0 1 11:22 pts/0 00:00:00 grep fio
Later, when the threads are stuck:
# ps -eLf | grep fio
root 5224 4274 5224 0 16 11:12 pts/1 00:00:02 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5225 0 16 11:12 pts/1 00:00:01 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5458 0 16 11:25 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5459 0 16 11:25 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5460 0 16 11:25 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5461 0 16 11:25 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5462 0 16 11:25 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5471 0 16 11:25 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5472 0 16 11:26 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5475 0 16 11:26 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5476 0 16 11:26 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5477 0 16 11:26 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5478 0 16 11:26 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5487 0 16 11:26 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5488 0 16 11:27 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 5224 4274 5489 0 16 11:27 pts/1 00:00:00 fio
--status-interval 10 --minimal fios/1.fio
root 6665 5237 6665 0 1 13:21 pts/0 00:00:00 grep fio
Is the number of threads supposed to change?..
Never answered this one... Yes, it'll change, since when you run the
job, you'll have one backend process, a number of IO workers, and one
disk util thread typically. When you get stuck, it's the backend that is
left waiting for that mutex.
In any case, I haven't been able to figure this one out yet. But it
should be safe enough to just ignore the stat mutex for the final
output, since the threads otherwise accessing it are gone. Can you see
if this one makes the issue go away?
--
Jens Axboe
diff --git a/backend.c b/backend.c
index 30f78b72f9d1..981625b61095 100644
--- a/backend.c
+++ b/backend.c
@@ -2068,7 +2068,7 @@ int fio_backend(void)
run_threads();
if (!fio_abort) {
- show_run_stats();
+ __show_run_stats();
if (write_bw_log) {
int i;
diff --git a/stat.h b/stat.h
index 2e46175053e8..90a7fb31a1bc 100644
--- a/stat.h
+++ b/stat.h
@@ -218,6 +218,7 @@ extern void show_group_stats(struct group_run_stats *rs);
extern int calc_thread_status(struct jobs_eta *je, int force);
extern void display_thread_status(struct jobs_eta *je);
extern void show_run_stats(void);
+extern void __show_run_stats(void);
extern void show_running_run_stats(void);
extern void check_for_running_stats(void);
extern void sum_thread_stats(struct thread_stat *dst, struct thread_stat *src, int nr);
