On 2010-07-08, at 14:01, Guy Coates wrote:
> Try this script; (It is from Bernd Schubert). It will parse the
> per-client proc stats on the mds/oss into something nice and
> humanly-readable. It is very useful.
I'm not sure I'd quite call it "human readable", but it does show that there is
a need for something to print out stats for all of the clients.
===================== /proc/fs/lustre/obdfilter/myth-OST0004/exports
============================
0...@lo read_bytes 123343 samples [bytes] 1 1048576 64498717397 write_bytes
18457 samples [bytes] 1 1048576 3200834973 get_info 2 samples [reqs]
set_info_async 1 samples [reqs] disconnect 3 samples [reqs] create 420 samples
[reqs] destroy 883 samples [reqs] setattr 13276 samples [reqs] punch 15 samples
[reqs] preprw 141800 samples [reqs] commitrw 141800 samples [reqs]
192.168.20....@tcp read_bytes 146 samples [bytes] 4096 1048576 114471161
write_bytes 7 samples [bytes] 163840 1048576 5244376 disconnect 6 samples
[reqs] preprw 153 samples [reqs] commitrw 153 samples [reqs]
192.168.20....@tcp read_bytes 550 samples [bytes] 4096 1048576 270017490
write_bytes 1126 samples [bytes] 32 1048576 614266996 disconnect 2 samples
[reqs] preprw 1676 samples [reqs] commitrw 1676 samples [reqs]
192.168.20....@tcp read_bytes 88745 samples [bytes] 0 1048576 61982699353
write_bytes 75428 samples [bytes] 16 1048576 27989934969 get_info 4 samples
[reqs] disconnect 22 samples [reqs] destroy 113 samples [reqs] setattr 1
samples [reqs] punch 154 samples [reqs] sync 81914 samples [reqs] preprw 164173
samples [reqs] commitrw 164173 samples [reqs]
=============================================================================================
Probably an equivalent script that produces more readable output would be like:
egrep -v "snapshot|ping" /proc/fs/lustre/{mds,obdfilter}/*/exports/*/stats |
cut -d/ -f 6,8,9
which will print something like:
myth-MDT0000/0...@lo/stats:open 10 samples [reqs]
myth-MDT0000/0...@lo/stats:close 2 samples [reqs]
myth-MDT0000/0...@lo/stats:getxattr 1 samples [reqs]
myth-MDT0000/192.168.20....@tcp/stats:open 3654 samples [reqs]
myth-MDT0000/192.168.20....@tcp/stats:close 1827 samples [reqs]
myth-MDT0000/192.168.20....@tcp/stats:unlink 1 samples [reqs]
myth-MDT0000/192.168.20....@tcp/stats:getxattr 15674 samples [reqs]
myth-OST0000/0...@lo/stats:read_bytes 2137 samples [bytes]
myth-OST0000/0...@lo/stats:preprw 2137 samples [reqs]
:
:
I would also recommend the "llstat" tool that is part of Lustre for ages
already, that will do mostly the same thing but can print it like "vmstat"
output with the current operation rates. The main difference is that the
"lustre_client_stats.sh" script prints the output for all of the clients at
once.
While we are on the topic, people may also be interested in "llobdstat", which
prints an IO-oriented status for any "stats" file containing the read_bytes and
write_bytes entries:
llobdstat myth-OST0000 2
/usr/bin/llobdstat on obdfilter/myth-OST0000
Processor counters run at 2800.419 MHz
Read: 4.08846e+11, Write: 9.0329e+10, create/destroy: 1133/1996, stat: 12128,
punch: 241
[NOTE: cx: create, dx: destroy, st: statfs, pu: punch ]
Timestamp Read-delta ReadRate Write-delta WriteRate
--------------------------------------------------------
1278622955 21.00MB 10.48MB/s 0.00MB 0.00MB/s
1278622957 23.00MB 11.48MB/s 0.00MB 0.00MB/s
1278622959 22.33MB 11.14MB/s 0.00MB 0.00MB/s
1278622961 11.68MB 5.83MB/s 0.00MB 0.00MB/s
1278622963 18.45MB 9.20MB/s 0.00MB 0.00MB/s st:1
1278622965 20.72MB 10.34MB/s 0.00MB 0.00MB/s st:1
It can also be used on a client stats file, like
/proc/fs/lustre/osc/myth-OST0000-osc-ffff81001f5d54d0/stats
Bernd, would you (or anyone) be interested to enhance those tools to be able to
show stats data from multiple files at once (each prefixed by the device name
and/or client NID)? I don't think it makes sense to create separate tools for
this.
Cheers, Andreas
--
Andreas Dilger
Lustre Technical Lead
Oracle Corporation Canada Inc.
_______________________________________________
Lustre-discuss mailing list
[email protected]
http://lists.lustre.org/mailman/listinfo/lustre-discuss
